IBM Mainframe Forum

again ...
get in touch with the IBM dfsort team...
they will tell what information to collect for the future

the chances of finding why You experienced a failure on that particular day are pretty slim

what usually happens on the first failure ( any software/hardware component ),
is that the info collected/available is insufficient for any problem determination

Note: whenever I say "you" I don't me you personally, I mean "whoever did that at your site"

Yes, well, basically it ran out of workspace. There is probably something in the informational messages for IBM Support. When you see lots of those, it should be easy to connect the dots and contact IBM Support. They may from those be able to say what you ran out of.

However, you have two messages which indicate the coming problem. "don't know how many records" and "I'm going to pause the sorting now and do a bit of merging, to see if that gets me to termination, sorry, it'll take a bit longer than otherwise".

There's a few ways you can specify an estimated number of records. 30% a quarter is quite rapid. And immediate fix is to hard-code an estimate of the number of records, probably most convenient in a DFSPARM DD. Then you need to change it "periodically". This can be a manual change of the hard-coded value, or output a formatted count from the COBOL program and feed that value into the next day's run (or if you have somewhere earlier that you've used the data set that day you can even go for the exact number of records if there are no insertions/exclusions).

This will probably remove the workspace problem, and the two informational messages alluded to will probably both disappear.

Expect improvement in elapsed, CPU and IO.

Best, most accurate advice, pointers, explanations and suggestions of things to look out for in the future, will come from DFSORT Support.

Dynamic allocation doesn't come into it, because you have SORTWKnn DDs in the JCL. So it is using those. From the use of Memory Objects and DASD work space, it looks to have dropped from using MO to DASD when the "intermediate merge" was selected, and the explanation of the message supports that.

Remember, the amount of "memory" require was greatly overestimated because the number of records is not known. It is here not a case of increasing memory available, but of aiding a better estimate of the workspace needed.

I don't think on your work data sets you are getting as much secondary allocation as you may expect. Secondary allocation for work datasets must be within the same volume (ie cannot be multi-volume).

I don't know why you specified SORTWKs. If the MO worked, you'd not use an DASD. Much simpler to allow dynamic allocation to support the potential lack of MO (unless DFSORT say otherwise...). With the SORTWKs, why 1000 cylinders primary? 6000 cylinders per SORTWK is being used, why not specify that as the primary? RLSE I don't see any point in (data set is going to be deleted), and DELETE,DELETE is the default for NEW, so why type it out even once?

Put in the FILSZ=En, remove the SORTWKs. Look to see how much memory is actually used.

However, better, consult DFSORT Support

THAT was helpful, thank you Billy.

From the SORTWK perspective, usage of only 6k Cyls is another burnt cookie; why weren't/aren't more cylinders allocated/being-allocated per volume even when it was hard-coded and ~9k cylinders per volume were all available (it could've gone upto 8 extents on secondary - 1000 + 7/8 * 1000 = 8k/9k cyls); why weren't more work datasets pulled in even when they were coded and available. Relatively low primary could mean, originally this job might have been executing parallel to other batch streams, which would mean the available storage would fluctuate, hence a safer-low value, and same was not revisited when the design changed at some point in the past. But that is a point for another discussion.

Thanks again; time to write to DFSORT team.

That is one of the great advantages of dynamic allocation. DFSORT, when it can, produces great estimates of required work space. No significant concern of under- or over-allocation. And if DFSORT can't make a good guess, you help it out and get the same effect.

These users thanked the author BillyBoyo for the post:

Aki88 (Wed Oct 26, 2016 11:10 am)

Hello,

Raised the query to Kolusu, and as-is-generally-the-case-with-Mr. Yaeger-and-him, the response was humbling.

The relevant excerpts:

DFSORT had to take a step back and re-evaluate the available resources and choose an alternate method to complete the sorting.
So once the Sortwork path isn't going to finish, then DFSORT reverted to using Memory objects as work space and was able to complete job.
The reason why DFSORT had to use Sortworks and Memory objects both is:
You have EXPMAX=12000 which is in Megabytes is a setting for maximum Hipersorting, memory object sorting, and data space sorting activity on a system.
ICE236I 0 OPTIONS: DYNAPCT=10 ,MOWRK=Y,TUNE=STOR,EXPMAX=12000 ,EXPOLD=10% ,EXPRES=10%

Based on the number of records you are sorting and LRECL you would roughly need about 78028MB. However your system is capped at 12000 MB

Is there a reason as to why you decided to cap EXPMAX?

...
...

Ideally we want to set MOSIZE=MAX and HIPRMAX=OPTIMAL but also set EXPOLD=5% and EXPMAX=25%. This would allow DFSORT to use up to 25% of the storage on the system only if it's available while also not allowing it to consider any old pages held by other workloads. This would be a conservative approach to allowing DFSORT to use some available storage to eliminate work data set I/O and improve performance. You could then monitor and gradually increase EXPMAX if resources are available.

...
...

So based on your current values you can pass an estimated file size of FILSZ=E720000000 which includes a 20% growth for future needs.

I'd never really have looked into EXPMAX.
There were discussion pieces about DYNALLOC and the relation to DYNAUTO, but that is already covered in the earlier posts on this topic.

Once again, Mr. Woodgar, Mr. Sample - thank you, you guys are amazing.

Best regards.

Thanks for the sharing the inputs from DFSORT expert.

Regards
Magesh