Page 2 of 2

Re: Possible to stop SORT before EOF?

PostPosted: Thu Feb 17, 2011 9:08 am
by dick scherrer
Hello,

Please pay closer attention - it is more important to reply with information that fits the dialog rather than with personal preference. . . For many people/places, Easytrieve is their "go to" tool. Often a very good choice. But every coding opportunity is not a nail, so always using the hammer is not always the best (or even a good) solution.

Of course this is batch - rather silly to even consider "a few million records" in an online process (though i've had a few clients that were unhappy with the results when they did). Easytrieve will use more cpu because it is interpreted - most of the time. Of course i've only seen a few hundred thousand Easytrieve modules/jobs across more than 100 different data centers (been a "migrant data worker" for over 30 years - usually with multiple concurrent clients) . . . Once upon a time a couple of my clients ran some timing tests and determined that unless the data volume was "huge", the cost of the compile outweighed the cost of the interpreted run. Keep in mind that these were not little quick & dirty programs, but rather code that ran into the several thousand lines.

Previous post was commenting on good performance of Syncsort. Easytrieve Plus will use Syncsort (or whatever is installed).
Sort performance is not part of this consideration. The main cpu use in an Easytrieve process is used working on the data within the Easytrieve code - and it really is quite expensive.

the production department wanted to order us to write every program possible in Easytrieve rather than COBOL
Yup, i've clients who insisted that everythng to be promoted to Production had to be written in the Chosen Language. Others only need written permission to use a particular tool (like Easytrieve or whatever) rather than the Chosen Language.

Please confine replies to the part of the forum where the topic is posted. If someone is interested in some other solution, they usually ask for their topic to be relocated.

Re: Possible to stop SORT before EOF?

PostPosted: Fri Feb 18, 2011 5:22 am
by BillyBoyo
Hello,

Easytrieve Plus (in case anyone thought I was talking about Easytrieve 9.0d) is compiled. The following is from an Easytrieve manual:

You can better understand how CA-Easytrieve/Plus works by examining the
modes in which it operates, the structure of a program, and the syntax rules of
CA-Easytrieve/Plus statements.
CA-Easytrieve/Plus provides five modes of operation that facilitate production
as well as ad hoc programming. The modes of operation (determined by the
PARM statement parameters you choose) are:
 Syntax check CA-Easytrieve/Plus source statements.
 Syntax check and compile CA-Easytrieve/Plus source statements.
 Syntax check, compile, and execute CA-Easytrieve/Plus source statements
(the default).
 Syntax check and compile CA-Easytrieve/Plus source statements and
produce an object module.
 Execute previously link-edited programs.


OK, so in any Easytrieve you run with the source (the default option) you do have the overhead of sytax check/compile each time you run it. But if you stick to that for unit-testing and one-offs, then I don't see it being much of a problem.

If you want to save the CPU time (not much) of syntax/compile, then produce an object module and execute that. As to interpreting Easytrieve programs, I've never heard of it, know of no PARM option which does it, and it seems the same from looking at even a fairly recent manual. OK, CA could have taken it backwards since then and included an INTERPRET option, but I can't for the life of me think of a reason for them to do that.

OK, my evidence so far is just from the manual, and perhaps CA
would
say that, wouldn't they?

So, what I did, with a pretty big file, can't remember how many records but I suspect a couple of million, and I ran it the default way (syntax/compile/run) and then compiling to an object module, and executing that. I found no difference to worry about in either CPU or elapsed time (I was doing this in the evening, after the on-line day, before the production batch started (system was being slowly migrated from an HP-3000, so there was always a couple of hours before the IBM batch was able to start).

More evidence. On the syntax/compile/run option, add LIST to the PARM statement (or add "PARM LIST" as the first line of the code. Then look at your listing output. You should see the assembler code that your source has been compiled to. Interpreters don't generate 370-assembler (or whatever it is called these days).

OK, so you only execute EZTPA00 (or whatever it is called these days) - how does it run your compiled cobol program by default? After compiling I guess it "calls" your compiled code (number of ways that can be done, no real point in speculating).

Now what is happening in an Easytrieve sort? Easytrieve is handling the input file, and calling Synsort or whatever with each record that passes the selection criteria that you yourself write. All of the sorting, all of the work datasets, all of the output, is handled by Syncsort (or whatever is installed at your installation).

Does it take longer to sort this way? Yes, but not much. So why use it if it takes longer? For things that are not easy to do in Syncsort (or whatever is installed...).

So, looking idly through the forums there were three recent questions relating to SYNCSORT where people wanted answers. The particular answers could easily be provided by Easytrieve, in one case beating the pants off Syncsort in run time (as Syncsort would have to be run twice) and in another case doing something Syncsort is not capable of (header and trailer processing). My simple enquiries over whether the questioner had access to Easytrieve were deleted. OK, if I had been suggesting something outlandish (hey, you could do all this in REXX) then fair enough. But when all it is (except for the three-file join) is a way of using Syncsort for the sorting and something else for exactly what you want (the programming bit) then I think it is a bit much to delete it.

OK. So, of course, things could have changed since I used Easytrieve. But, like I said before, if anyone is having performance issues with Easytrieve Plus, make a post in the Easytrieve forum, including your PARM statement for starters.