MOSIZE Calculation



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

MOSIZE Calculation

Postby Aki88 » Sat Oct 22, 2016 7:30 pm

Hello,

A few queries pertaining to DFSORT and storage:

a. How does DFSORT calculate MOSIZE?

For instance:
No SORTWK DD statement
MOSIZE override: 10%
ICE133I: MOSIZE=559
LRECL = 111
BLKSIZE = 27861
TYPE = FB
DYNAPCT=10
DYNALOC=(,004)
DYNSPC=256
For instance, number of rec: 31168769
ICE165I: ALLOCATED TKS: 81900
ICE299I gives: 3345 MB --> how was this number calculated.

b. What happens if SORTWK as well as DYNALLOC - both are provided; number > default of 4; allocation in SORTWK DD statement filling 1 volume (~ 9000 cyls) per DD statement?
My understanding is (if DYNAUTO=YES), SORTWK will be allocated first, if insufficient, then DYNALLOC will be utilized, when even that gets filled, then DYNAPCT calculated value will be used.

c. How are primary and secondary allocation values calculated for DYNALLOC DS using DYNSPC.

Any guidance and calculation explanation is really appreciated.

Thank you.
Aki88
 
Posts: 381
Joined: Tue Jan 28, 2014 1:52 pm
Has thanked: 33 times
Been thanked: 36 times

Re: MOSIZE Calculation

Postby Robert Sample » Sun Oct 23, 2016 7:53 pm

From the DFSORT Application Programming Guide:
p%
specifies that DFSORT determines dynamically the maximum size of a memory object to be used for memory object sorting, subject to a limit of p percent of the available central storage. p must be a value between 0 and 100. If p is 0, memory object sorting is not used. The value calculated for p% is limited to 2147483646 MB, and is rounded down to the nearest MB.
and this is relevant:
Some of these limits depend on system and other DFSORT activity during the time that a memory object sorting application runs. Consequently, the total amount of memory object storage that a memory object sorting application uses can vary from run to run.

MOSIZE=n specifies a fixed value for MOSIZE. MOSIZE=p% specifies a value for MOSIZE that varies as a percentage of the available central storage on the system at run-time. If the available central storage on a system changes, MOSIZE=p% will cause a corresponding change in the MOSIZE value selected by DFSORT, whereas MOSIZE=n will not. When sharing DFSORT installation options between systems, such as in a sysplex, MOSIZE=p% can be used to tailor the MOSIZE value to the system selected for the application, providing a more dynamic MOSIZE value than MOSIZE=n.

Your questions are best answered by looking -- CLOSELY at the DFSORT Application Programming Guide manual; you may need to read it several times for everything to sink in. And since some of what you are asking can be overridden by the site, you need to consult your site support group for answers as well. You may need to read some of the other DFSORT manuals as well.
Robert Sample
Global moderator
 
Posts: 3719
Joined: Sat Dec 19, 2009 8:32 pm
Location: Dubuque, Iowa, USA
Has thanked: 1 time
Been thanked: 279 times

Re: MOSIZE Calculation

Postby BillyBoyo » Sun Oct 23, 2016 8:41 pm

You'll find further information in the Installation and Customization and in the Tuning guides.

The stress is on "it is a system-wide thing" and DFSORT constantly monitors the use for concurrent Hipersorting - if you "use too much" it'll just swap out to DASD anyway.

Probably not something for "application people", like you and I, to mess with directly. If you have an idea that there may be either general improvements or specific improvements for critical JOBs, talk to your support people, as Robert has already suggested, and perhaps even encourage them to contact DFSORT for advice.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: MOSIZE Calculation

Postby Aki88 » Sun Oct 23, 2016 10:14 pm

BillyBoyo wrote:.... if you "use too much" it'll just swap out to DASD anyway....


Thank you Robert, Billy for the pointers.

Billy, reason for this query was, of late we've been noticing an aberrant behavior in the way storage is being allocated/used by various programs. For instance the stats that I showed for MOSIZE, they 'always' remained 'almost' constant for every run, for over years and then one fine day the thing simply conked. Now I do understand that 'something', 'somewhere', very definitely would've changed to cause such a thing to happen, but here are some numbers for a failure: daily MO usage was ~6 GB, and the day the thing conked, the successful run went as high as ~12 GB; whereas work datasets took around 1,350,000 tracks (this has been a constant number for all runs over years). It was a batch COBOL prog, doing a simple internal sort and then subsequent processing on the OUTPUT PROCEDURE. Increase in records was by around 0.7 mill, this guy was executing alone, neither CICS or any other batch process was performing heavy duty activity, entire work storage group was available (~ 70+ 3390-9 volumes) for allocation- yet something went wrong. The sysprogs tell me that the requirement was high hence the extra MO, but another test run brought the numbers back to the daily count; which doesn't support any of their argument. Hence the curious query. I couldn't salvage much on the lines of calculations from Tuning/Installation guides (I will revisit and look more closely); but the way the ICE messages speak, I am pretty sure that the maths can be done, just don't know how - yet.

If this is something to be raised to IBM DFSORT team, will write to them; but I thought if someone can point me to the algorithm, maybe I'll try doing some maths before I wrote to the big guns. :)

It'd be great if you can help me with point-c though, 'tis a tricky one for me.

Thank you.
Aki88
 
Posts: 381
Joined: Tue Jan 28, 2014 1:52 pm
Has thanked: 33 times
Been thanked: 36 times

Re: MOSIZE Calculation

Postby Robert Sample » Sun Oct 23, 2016 11:50 pm

Memory objects used in sorting are backed by central storage -- perhaps that became an issue? I suspect that MOSIZE, per se, is no more than a symptom of your problem since it merely gives an upper limit on the amount of 64-bit storage that can be used.

And, by the way, ICE299I provides a number that is NOT calculated -- it represents the actual amount of memory object storage used. So it was "calculated" by looking at the actual use during the sort. 3345M doesn't seem excessive -- one of our vendor products recommends running with MEMLIMIT=100G these days; CICS 5.2 recommends MEMLIMIT be set to at least 6 GB.
Robert Sample
Global moderator
 
Posts: 3719
Joined: Sat Dec 19, 2009 8:32 pm
Location: Dubuque, Iowa, USA
Has thanked: 1 time
Been thanked: 279 times

Re: MOSIZE Calculation

Postby BillyBoyo » Mon Oct 24, 2016 11:01 am

For your c., do you mean the old question of "how do I calculate the workspace" or do you mean "how does DFSORT know how to allocate the workspace"?

SORTWKn DDs in the JCL override the operation of DYNALLOC.

I think rather than speculating here, you need to contact DFSORT. The change you saw has to be associated with something changing somewhere, but the interaction of parameters and use of systems really means getting DFSORT to look at it is your only choice. Just a small misunderstanding of one thing, and your own (or anyone else's analysis) is dead.
telegraph india sudoku
Again, the MO figures are informational (not sure how one is bigger than the other, but I've never needed to look) and their presence does not indicate that DFSORT is even able to use that space.

From your description, it looks like the step had to drop out of hyperspace and run on impulse engines. Or is that Star Trek?

Why it would do that (and even if it really did that), with little running on the machine, is not something I've encountered or have a top-of-the head answer for if it did happen :-)

Contact DFSORT.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: MOSIZE Calculation

Postby Aki88 » Mon Oct 24, 2016 12:12 pm

Thank you Robert, Billy :)

Billy,

BillyBoyo wrote:For your c., do you mean ... "how does DFSORT know how to allocate the workspace"?

SORTWKn DDs in the JCL override the operation of DYNALLOC. ...

...Just a small misunderstanding of one thing, and your own (or anyone else's analysis) is dead.
telegraph india sudoku ...

... drop out of hyperspace and run on impulse engines. Or is that Star Trek? ...
....
Contact DFSORT.


Yup, while allocating space using DYNALLOC, how does DFSORT decide primary and secondary allocations OR an MB/GB allocation going hand in hand with DYNSPC parameter; I could see SORTWK DD being allocated in JESYSMSG, but couldn't figure out the logic behind the numbers being assigned. In the scenario when we give a hard-override from JCL, we are pretty aware of the amount of storage we are enforcing - correct or incorrect, this number IS going to be allocated. The interaction between DYNSPC and DYNALLOC is cloudy for me. If you can kindly give me a quick reckoner of how DYNALLOC allocates space in terms of how much to allocate, will suffice, even a quick theory lesson will help; for MO will be writing to DFSORT.

Reason for asking- generally during installation, we define primary/secondary allocation parameters for various utilities else they won't know How-much/What to allocate. In case of DFSORT I couldn't find any such value mentioned other than the DYNSPC (or maybe I overlooked the values, maybe I should look more closely at TMAX*/SIZE, and the correlations with DSA).

I completely agree with you on miscalculations; will try writing to the Holy creators for assistance. :D

Instead of impulse engines, I think it came down to a 'Sublight Drive' :mrgreen:

Thank you for everything.
Aki88
 
Posts: 381
Joined: Tue Jan 28, 2014 1:52 pm
Has thanked: 33 times
Been thanked: 36 times

Re: MOSIZE Calculation

Postby BillyBoyo » Mon Oct 24, 2016 9:26 pm

Well, if you want more specifics for the step, you'll have to post the sysout from the conked and an unconked step :-)
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: MOSIZE Calculation

Postby Aki88 » Tue Oct 25, 2016 12:10 pm

Hello Billy,

Here goes:

Relevant pieces of the legacy JCL (the [1000,1000] cyls are not optimal) are given below, have removed/modified the application JCL code piece:


25 XXRRRR#330 EXEC PGM=APPLE,COND=(0,NE)                                
......
45 XXDFSPARM  DD DSN=MONKEY.PARM(XXXPARM1),DISP=SHR            
   XX*                                                                  
46 XXSORTWK01 DD DSN=&&TEMP,DSNTYPE=LARGE,                              
   XX            DISP=(NEW,DELETE,DELETE),                              
   XX            SPACE=(CYL,(1000,1000),RLSE)                            
   XX*                                                                  
47 XXSORTWK02 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)
   XX*                                        
48 XXSORTWK03 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)
   XX*                                        
49 XXSORTWK04 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)
   XX*                                        
50 XXSORTWK05 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)
   XX*                                        
51 XXSORTWK06 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)
   XX*                                          
52 XXSORTWK07 DD DSN=&&TEMP,DSNTYPE=LARGE,      
   XX            DISP=(NEW,DELETE,DELETE),      
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                          
53 XXSORTWK08 DD DSN=&&TEMP,DSNTYPE=LARGE,      
   XX            DISP=(NEW,DELETE,DELETE),      
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                          
54 XXSORTWK09 DD DSN=&&TEMP,DSNTYPE=LARGE,      
   XX            DISP=(NEW,DELETE,DELETE),      
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                          
55 XXSORTWK10 DD DSN=&&TEMP,DSNTYPE=LARGE,      
   XX            DISP=(NEW,DELETE,DELETE),      
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                          
56 XXSORTWK11 DD DSN=&&TEMP,DSNTYPE=LARGE,      
   XX            DISP=(NEW,DELETE,DELETE),      
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                        
57 XXSORTWK12 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                        
58 XXSORTWK13 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                        
59 XXSORTWK14 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                        
60 XXSORTWK15 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                        
 


Clean job run with above code:


1ICE201I 1 RECORD TYPE IS F - DATA STARTS IN POSITION 1
 ICE118I 0 UNKNOWN FILE SIZE - RECOMMEND SPECIFYING FILSZ=EN TO REDUCE RISK OF FAILURE OR DEGRADED PERFORMANCE
 ICE751I 0 C5-BASE   C6-BASE   C7-BASE   C8-I29500 E4-BASE   C9-BASE   E5-I29500 E7-BASE
 ICE143I 0 BLOCKSET     SORT  TECHNIQUE SELECTED
 ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
 ICE000I 0 - CONTROL STATEMENTS FOR 5650-ZOS, Z/OS DFSORT V2R2  - NN:MM ON XXX YYY ZZ, 2016 -
0           OPTION MOSIZE=10%,HIPRMAX=10%,DYNALLOC=(SYSDA,15),DSPSIZE=300         -XXXXXXX
                    EXPMAX=10%,EXPOLD=10%,EXPRES=10%                               XXXXXXX
 ICE140I 0 END OF PARAMETERS FROM DFSPARM  - SYSIN OR SORTCNTL/PARAMETER LIST CONTROL STATEMENTS FOLLOW
           SORT FIELDS=(0001,0023,CH,A)
           RECORD TYPE=F,LENGTH=(000111,,)
 ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
 ICE252I 1 PARMLIB OPTIONS WERE MERGED WITH INSTALLATION MODULE DEFAULTS
 ICE089I 1 XXXXXX  .ZZZZZZZ .RRRR#330, INPUT LRECL = 111, TYPE = F
 ICE093I 0 MAIN STORAGE = (MAX,6291456,6291456)
 ICE156I 0 MAIN STORAGE ABOVE 16MB = (6234096,6234096)
 ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
 ICE128I 0 OPTIONS: SIZE=6291456,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT
 ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO   ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=N             ,ABCODE=MSG
 ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
 ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW=Y,DSA=0
 ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
 ICE133I 0 OPTIONS: HIPRMAX=1200   ,DSPSIZE=300 ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=1200
 ICE235I 0 OPTIONS: NULLOUT=RC0
 ICE236I 0 OPTIONS: DYNAPCT=10 ,MOWRK=Y,TUNE=STOR,EXPMAX=12000  ,EXPOLD=10%    ,EXPRES=10%
 ICE750I 0 DC 0 TC 0 CS DSVVV KSZ 23 VSZ 23
 ICE752I 0 FSZ=0 RE  IGN=0 C  AVG=112 0  WSP=0 E  DYN=0 0
 ICE247I 0 INTERMEDIATE MERGE ENTERED - PERFORMANCE MAY BE DEGRADED
 (AAAAAAA) AAAAAA - XXXX - OPENED: SEQ INPUT        NNN READS,         0 WRITES,         0 REWRITES,         0 DELETES
 ICE751I 1 DE-BASE   D5-I29741 D3-BASE   E1-BASE   D6-BASE   D3-BASE   D7-BASE   E8-I29741
 ICE091I 0 OUTPUT LRECL = 111, TYPE = F
 ICE055I 0 INSERT 540334515, DELETE 540334515
 ICE054I 0 RECORDS - IN: 0, OUT: 0
 ICE134I 0 NUMBER OF BYTES SORTED: 59977131165
 ICE253I 0 RECORDS SORTED - PROCESSED: 540334515, EXPECTED: 0
 ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 1350000 , TRACKS USED: 1342680
 ICE199I 0 MEMORY OBJECT USED AS MAIN STORAGE = 0M BYTES
 ICE299I 0 MEMORY OBJECT USED AS WORK STORAGE = 6758M BYTES
 ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES
 ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
 ICE052I 0 END OF DFSORT
 (AAAAAAA) ARRRRR   - XXXX - OPENED: RDM INPUT          3 READS,         0 WRITES,         0 REWRITES,         0 DELETES
 (AAAAAAA) ARRRRR   - YYYY - OPENED: SEQ INPUT  540334517 READS,         0 WRITES,         0 REWRITES
 (CCSBBBB) ARRRRR - ZZZZ - OPENED: DYN I-O            0 READS,         1 WRITES,         0 REWRITES,         0 DELETES
 


Failed run, the next day:


1ICE201I 1 RECORD TYPE IS F - DATA STARTS IN POSITION 1
 ICE118I 0 UNKNOWN FILE SIZE - RECOMMEND SPECIFYING FILSZ=EN TO REDUCE RISK OF FAILURE OR DEGRADED PERFORMANCE
 ICE751I 0 C5-BASE   C6-BASE   C7-BASE   C8-I29500 E4-BASE   C9-BASE   E5-I29500 E7-BASE
 ICE143I 0 BLOCKSET     SORT  TECHNIQUE SELECTED
 ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
 ICE000I 0 - CONTROL STATEMENTS FOR 5650-ZOS, Z/OS DFSORT V2R2  - NN:MM ON XXX YYY ZZ, 2016 -
0           OPTION MOSIZE=10%,HIPRMAX=10%,DYNALLOC=(SYSDA,15),DSPSIZE=300         -XXXXXXX
                    EXPMAX=10%,EXPOLD=10%,EXPRES=10%                               XXXXXXX
 ICE140I 0 END OF PARAMETERS FROM DFSPARM  - SYSIN OR SORTCNTL/PARAMETER LIST CONTROL STATEMENTS FOLLOW
           SORT FIELDS=(0001,0023,CH,A)
           RECORD TYPE=F,LENGTH=(000111,,)
 ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
 ICE252I 1 PARMLIB OPTIONS WERE MERGED WITH INSTALLATION MODULE DEFAULTS
 ICE089I 1 XXXXXX  .ZZZZZZZ .RRRR#330, INPUT LRECL = 111, TYPE = F
 ICE093I 0 MAIN STORAGE = (MAX,6291456,6291456)
 ICE156I 0 MAIN STORAGE ABOVE 16MB = (6234096,6234096)
 ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
 ICE128I 0 OPTIONS: SIZE=6291456,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT
 ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO   ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=N             ,ABCODE=MSG
 ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
 ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW=Y,DSA=0
 ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
 ICE133I 0 OPTIONS: HIPRMAX=1200   ,DSPSIZE=300 ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=1200
 ICE235I 0 OPTIONS: NULLOUT=RC0
 ICE236I 0 OPTIONS: DYNAPCT=10 ,MOWRK=Y,TUNE=STOR,EXPMAX=12000  ,EXPOLD=10%    ,EXPRES=10%
 ICE750I 0 DC 0 TC 0 CS DSVVV KSZ 23 VSZ 23
 ICE752I 0 FSZ=0 RE  IGN=0 C  AVG=112 0  WSP=0 E  DYN=0 0
 ICE247I 0 INTERMEDIATE MERGE ENTERED - PERFORMANCE MAY BE DEGRADED
0ICE805I 0 JOBNAME: XXXXXX   , STEPNAME: ZZZZZZZ
 ICE802I 0 BLOCKSET     TECHNIQUE IN CONTROL
 ICE906I 0 ST=ABOVE,SR=6291456,RC=0
 ICE907I 0 ST=ABOVE,SA=6291440,NF=1,LF=6291440,SF=6291440
 ICE906I 0 ST=BELOW,SR=1024000,RC=0
 ICE907I 0 ST=BELOW,SA=1023984,NF=1,LF=1023984,SF=1023984
 ICE992I 0 RA 0 WR 0 TR 0
 ICE887I 0 CSES 0,0,0 ES 0,0,0
 ICE886I 0 SYS 0 TSTG 0 FS 0 INIT 0 MAX 0 LEN 0
 ICE915I 0 MOFSZ=31,MOSZ=0,MOSYS=1200(1),MOSTG=12000,MEML=17592186039801(3)
 ICE916I 0 MOFR=0402,MOVR=VV
 ICE996I 0 ESM=3072000,ESO=230235,ESR=1439203,ESP=7936,ESS=16384,CES=15975680,HSZ=16777216
 ICE997I 0 HWSP=0,HMAX=0,MOMAX=307200,ASV=3072000,EQ=I3,HN=1
 ICE898I 0 OMAX=12089681,NMAX=14392035,ENQT=1872896,CMAX=190464,HU=99,BUN=16352,MD=NK,M2,DU=83,DR=0,HN=1
 ICE880I 0 QP=3 QA=155 HI=236 LI=234 MI=388 TZ=23808 N1=16384 N2=16384 SZ=25 HN=1
 ICE889I 0 CT=MAX     , SB=3, L=0, D=0000, CCW=1MAM
 ICE901I 0 W 01PP15 02PP15 03PP15 04PP15 05PP15 06PP15 07PP15 08PP15
 ICE901I 0 W 09PP15 10PP15 11PP15 12PP15 13PP15 14PP15 15PP15
 ICE906I 1 ST=ABOVE,SR=6234096,RC=0
 ICE907I 1 ST=ABOVE,SA=6234080,NF=1,LF=6234080,SF=6234080
 ICE906I 1 ST=BELOW,SR=52672,RC=0
 ICE907I 1 ST=BELOW,SA=44464,NF=1,LF=44464,SF=44464
 ICE996I 1 ESM=3072000,ESO=230073,ESR=1415042,ESP=7936,ESS=16384,CES=15975680,HSZ=16777216
 ICE897I 0 AVAL=11849690,BVAL=14150429,ENQT=1658624,ASV=3072000,HSZ=190464,HM=MH,HN=2
 ICE880I 0 QP=3 QA=176 HI=215 LI=213 MI=388 TZ=23808 N1=16384 N2=16384 SZ=25 HN=2
 ICE996I 1 ESM=3072000,ESO=230059,ESR=1394336,ESP=7936,ESS=16384,CES=15975680,HSZ=16777216
 ICE897I 0 AVAL=11642773,BVAL=13943364,ENQT=1444352,ASV=3072000,HSZ=190464,HM=MH,HN=3
 ICE880I 0 QP=3 QA=209 HI=182 LI=180 MI=388 TZ=23808 N1=16384 N2=16384 SZ=25 HN=3
 ICE996I 1 ESM=3072000,ESO=230195,ESR=1373972,ESP=7936,ESS=16384,CES=15975680,HSZ=16777216
 ICE897I 0 AVAL=11437768,BVAL=13739721,ENQT=1253888,ASV=3072000,HSZ=190464,HM=MH,HN=4
1ICE880I 0 QP=3 QA=233 HI=158 LI=156 MI=388 TZ=23808 N1=16384 N2=16384 SZ=25 HN=4
 ICE996I 1 ESM=3072000,ESO=230064,ESR=1353349,ESP=7936,ESS=16384,CES=15975680,HSZ=16777216
 ICE897I 0 AVAL=11232850,BVAL=13533491,ENQT=1039616,ASV=3072000,HSZ=190464,HM=MH,HN=5
 ICE880I 0 QP=3 QA=260 HI=131 LI=129 MI=388 TZ=23808 N1=16384 N2=16384 SZ=25 HN=5
 ICE996I 1 ESM=3072000,ESO=230176,ESR=1346187,ESP=7936,ESS=16384,CES=15975680,HSZ=16777216
 ICE897I 0 AVAL=11160105,BVAL=13461872,ENQT=960256,ASV=3072000,HSZ=190464,HM=MH,HN=6
 ICE880I 0 QP=3 QA=141 HI=250 LI=248 MI=388 TZ=23808 N1=16384 N2=16384 SZ=25 HN=6
 ICE996I 1 ESM=3072000,ESO=230056,ESR=1325778,ESP=7936,ESS=16384,CES=15975680,HSZ=16777216
 ICE897I 0 AVAL=10957225,BVAL=13257788,ENQT=769792,ASV=3072000,HSZ=190464,HM=MH,HN=7
 ICE880I 0 QP=3 QA=294 HI=97 LI=95 MI=388 TZ=23808 N1=16384 N2=16384 SZ=25 HN=7
 ICE996I 1 ESM=3072000,ESO=230179,ESR=1305719,ESP=7936,ESS=16384,CES=15975680,HSZ=16777216
 ICE897I 0 AVAL=10755401,BVAL=13057197,ENQT=555520,ASV=3072000,HSZ=190464,HM=MH,HN=8
 ICE880I 0 QP=3 QA=321 HI=70 LI=68 MI=388 TZ=23808 N1=16384 N2=16384 SZ=25 HN=8
 ICE996I 1 ESM=3072000,ESO=230099,ESR=1285031,ESP=7936,ESS=16384,CES=15975680,HSZ=16777216
 ICE897I 0 AVAL=10549320,BVAL=12850318,ENQT=365056,ASV=3072000,HSZ=190464,HM=MH,HN=9
 ICE880I 0 QP=3 QA=345 HI=46 LI=44 MI=388 TZ=23808 N1=16384 N2=16384 SZ=25 HN=9
 ICE996I 1 ESM=3072000,ESO=230078,ESR=1251187,ESP=7936,ESS=16384,CES=15975680,HSZ=16777216
 ICE897I 0 AVAL=10211089,BVAL=12511870,ENQT=15872,ASV=3072000,HSZ=15872,HM=MH,HN=10
 ICE880I 0 QP=2 QA=388 HI=2 LI=1 MI=388 TZ=15872 N1=15872 N2=15872 SZ=3 HN=10
 ICE996I 1 ESM=3072000,ESO=230227,ESR=1249696,ESP=7936,ESS=16384,CES=15975680,HSZ=16777216
 ICE897I 0 AVAL=10194689,BVAL=12496966,ENQT=0,ASV=3072000,HSZ=0,HM=M1,HN=11
 ICE046A 5 SORT CAPACITY EXCEEDED - RECORD COUNT 540733625
 ICE253I 0 RECORDS SORTED - PROCESSED: 540733625, EXPECTED: 0
 ICE751I 1 DE-BASE   D5-I29741 D3-BASE   E1-BASE   D6-BASE   E8-I29741
 ICE052I 0 END OF DFSORT
 



Modified JCL on the day of failure (in my opinion, modification was not really required, and a rerun would have got the job to run clean):


25 XXRRRR#330 EXEC PGM=APPLE,COND=(0,NE)                                
......
45 XXDFSPARM  DD DSN=MONKEY.PARM(XXXPARM1),DISP=SHR            
   XX*                                                                  
46 XXSORTWK01 DD DSN=&&TEMP,DSNTYPE=LARGE,                              
   XX            DISP=(NEW,DELETE,DELETE),                              
   XX            SPACE=(CYL,(1000,1000),RLSE)                            
   XX*                                                                  
47 XXSORTWK02 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)
   XX*                                        
48 XXSORTWK03 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)
   XX*                                        
49 XXSORTWK04 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)
   XX*                                        
50 XXSORTWK05 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)
   XX*                                        
51 XXSORTWK06 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)
   XX*                                          
52 XXSORTWK07 DD DSN=&&TEMP,DSNTYPE=LARGE,      
   XX            DISP=(NEW,DELETE,DELETE),      
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                          
53 XXSORTWK08 DD DSN=&&TEMP,DSNTYPE=LARGE,      
   XX            DISP=(NEW,DELETE,DELETE),      
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                          
54 XXSORTWK09 DD DSN=&&TEMP,DSNTYPE=LARGE,      
   XX            DISP=(NEW,DELETE,DELETE),      
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                          
55 XXSORTWK10 DD DSN=&&TEMP,DSNTYPE=LARGE,      
   XX            DISP=(NEW,DELETE,DELETE),      
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                          
56 XXSORTWK11 DD DSN=&&TEMP,DSNTYPE=LARGE,      
   XX            DISP=(NEW,DELETE,DELETE),      
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                        
57 XXSORTWK12 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                        
58 XXSORTWK13 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                        
59 XXSORTWK14 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                        
60 XXSORTWK15 DD DSN=&&TEMP,DSNTYPE=LARGE,    
   XX            DISP=(NEW,DELETE,DELETE),    
   XX            SPACE=(CYL,(1000,1000),RLSE)  
   XX*                                        
61 XXSORTWK16 DD DSN=&&TEMP,DSNTYPE=LARGE,  
   XX            DISP=(NEW,DELETE,DELETE),  
   XX            SPACE=(CYL,(1000,1000),RLSE)
   XX*                                      
62 XXSORTWK17 DD DSN=&&TEMP,DSNTYPE=LARGE,  
   XX            DISP=(NEW,DELETE,DELETE),  
   XX            SPACE=(CYL,(1000,1000),RLSE)
   XX*                                      
63 XXSORTWK18 DD DSN=&&TEMP,DSNTYPE=LARGE,  
   XX            DISP=(NEW,DELETE,DELETE),  
   XX            SPACE=(CYL,(1000,1000),RLSE)
   XX*                                      
64 XXSORTWK19 DD DSN=&&TEMP,DSNTYPE=LARGE,  
   XX            DISP=(NEW,DELETE,DELETE),  
   XX            SPACE=(CYL,(1000,1000),RLSE)
   XX*                                      
65 XXSORTWK20 DD DSN=&&TEMP,DSNTYPE=LARGE,  
   XX            DISP=(NEW,DELETE,DELETE),  
   XX            SPACE=(CYL,(1000,1000),RLSE)
 


Clean run post modification:


1ICE201I 1 RECORD TYPE IS F - DATA STARTS IN POSITION 1
 ICE118I 0 UNKNOWN FILE SIZE - RECOMMEND SPECIFYING FILSZ=EN TO REDUCE RISK OF FAILURE OR DEGRADED PERFORMANCE
 ICE751I 0 C5-BASE   C6-BASE   C7-BASE   C8-I29500 E4-BASE   C9-BASE   E5-I29500 E7-BASE
 ICE143I 0 BLOCKSET     SORT  TECHNIQUE SELECTED
 ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
 ICE000I 0 - CONTROL STATEMENTS FOR 5650-ZOS, Z/OS DFSORT V2R2  - NN:MM ON XXX YYY ZZ, 2016 -
0           OPTION MOSIZE=10%,HIPRMAX=10%,DYNALLOC=(SYSDA,15),DSPSIZE=300         -XXXXXXX
                    EXPMAX=10%,EXPOLD=10%,EXPRES=10%                               XXXXXXX
 ICE140I 0 END OF PARAMETERS FROM DFSPARM  - SYSIN OR SORTCNTL/PARAMETER LIST CONTROL STATEMENTS FOLLOW
           SORT FIELDS=(0001,0023,CH,A)
           RECORD TYPE=F,LENGTH=(000111,,)
 ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
 ICE252I 1 PARMLIB OPTIONS WERE MERGED WITH INSTALLATION MODULE DEFAULTS
 ICE089I 1 XXXXXX  .ZZZZZZZ .RRRR#330, INPUT LRECL = 111, TYPE = F
 ICE093I 0 MAIN STORAGE = (MAX,6291456,6291456)
 ICE156I 0 MAIN STORAGE ABOVE 16MB = (6234096,6234096)
 ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
 ICE128I 0 OPTIONS: SIZE=6291456,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT
 ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO   ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=N             ,ABCODE=MSG
 ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
 ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW=Y,DSA=0
 ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
 ICE133I 0 OPTIONS: HIPRMAX=1200   ,DSPSIZE=300 ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=1200
 ICE235I 0 OPTIONS: NULLOUT=RC0
 ICE236I 0 OPTIONS: DYNAPCT=10 ,MOWRK=Y,TUNE=STOR,EXPMAX=12000  ,EXPOLD=10%    ,EXPRES=10%
 ICE750I 0 DC 0 TC 0 CS DSVVV KSZ 23 VSZ 23
 ICE752I 0 FSZ=0 RE  IGN=0 C  AVG=112 0  WSP=0 E  DYN=0 0
 ICE247I 0 INTERMEDIATE MERGE ENTERED - PERFORMANCE MAY BE DEGRADED
 (AAAAAAA) AAAAAA - XXXX - OPENED: SEQ INPUT        NNN READS,         0 WRITES,         0 REWRITES,         0 DELETES
 ICE751I 1 DE-BASE   D5-I29741 D3-BASE   E1-BASE   D6-BASE   D3-BASE   D7-BASE   E8-I29741
 ICE091I 0 OUTPUT LRECL = 111, TYPE = F
 ICE055I 0 INSERT 540733625, DELETE 540733625
 ICE054I 0 RECORDS - IN: 0, OUT: 0
 ICE134I 0 NUMBER OF BYTES SORTED: 60021432375
 ICE253I 0 RECORDS SORTED - PROCESSED: 540733625, EXPECTED: 0
 ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 1350000 , TRACKS USED: 1344315
 ICE199I 0 MEMORY OBJECT USED AS MAIN STORAGE = 0M BYTES
 ICE299I 0 MEMORY OBJECT USED AS WORK STORAGE = 11916M BYTES
 ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES
 ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
 ICE052I 0 END OF DFSORT
 (AAAAAAA) ARRRRR   - XXXX - OPENED: RDM INPUT          3 READS,         0 WRITES,         0 REWRITES,         0 DELETES
 (AAAAAAA) ARRRRR   - YYYY - OPENED: SEQ INPUT  540733627 READS,         0 WRITES,         0 REWRITES
 (CCSBBBB) ARRRRR - ZZZZ - OPENED: DYN I-O            0 READS,         1 WRITES,         0 REWRITES,         0 DELETES
 


Clean run a day later:


1ICE201I 1 RECORD TYPE IS F - DATA STARTS IN POSITION 1
 ICE118I 0 UNKNOWN FILE SIZE - RECOMMEND SPECIFYING FILSZ=EN TO REDUCE RISK OF FAILURE OR DEGRADED PERFORMANCE
 ICE751I 0 C5-BASE   C6-BASE   C7-BASE   C8-I29500 E4-BASE   C9-BASE   E5-I29500 E7-BASE
 ICE143I 0 BLOCKSET     SORT  TECHNIQUE SELECTED
 ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
 ICE000I 0 - CONTROL STATEMENTS FOR 5650-ZOS, Z/OS DFSORT V2R2  - NN:MM ON XXX YYY ZZ, 2016 -
0           OPTION MOSIZE=10%,HIPRMAX=10%,DYNALLOC=(SYSDA,20),DSPSIZE=300         -XXXXXXX
                    EXPMAX=10%,EXPOLD=10%,EXPRES=10%                               XXXXXXX
 ICE140I 0 END OF PARAMETERS FROM DFSPARM  - SYSIN OR SORTCNTL/PARAMETER LIST CONTROL STATEMENTS FOLLOW
           SORT FIELDS=(0001,0023,CH,A)
           RECORD TYPE=F,LENGTH=(000111,,)
 ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
 ICE252I 1 PARMLIB OPTIONS WERE MERGED WITH INSTALLATION MODULE DEFAULTS
 ICE089I 1 XXXXXX  .ZZZZZZZ .RRRR#330, INPUT LRECL = 111, TYPE = F
 ICE093I 0 MAIN STORAGE = (MAX,6291456,6291456)
 ICE156I 0 MAIN STORAGE ABOVE 16MB = (6234096,6234096)
 ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
 ICE128I 0 OPTIONS: SIZE=6291456,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT
 ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO   ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=N             ,ABCODE=MSG
 ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
 ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW=Y,DSA=0
 ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
 ICE133I 0 OPTIONS: HIPRMAX=1200   ,DSPSIZE=300 ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=1200
 ICE235I 0 OPTIONS: NULLOUT=RC0
 ICE236I 0 OPTIONS: DYNAPCT=10 ,MOWRK=Y,TUNE=STOR,EXPMAX=12000  ,EXPOLD=10%    ,EXPRES=10%
 ICE750I 0 DC 0 TC 0 CS DSVVV KSZ 23 VSZ 23
 ICE752I 0 FSZ=0 RE  IGN=0 C  AVG=112 0  WSP=0 E  DYN=0 0
 ICE247I 0 INTERMEDIATE MERGE ENTERED - PERFORMANCE MAY BE DEGRADED
 (AAAAAAA) AAAAAA - XXXX - OPENED: SEQ INPUT        NNN READS,         0 WRITES,         0 REWRITES,         0 DELETES
 ICE751I 1 DE-BASE   D5-I29741 D3-BASE   E1-BASE   D6-BASE   D3-BASE   D7-BASE   E8-I29741
 ICE091I 0 OUTPUT LRECL = 111, TYPE = F
 ICE096I 0 SUCCESSFUL RECOVERY FROM B37 ABEND(S) FOR WORK DATA SET(S)
 ICE055I 0 INSERT 540903202, DELETE 540903202
 ICE054I 0 RECORDS - IN: 0, OUT: 0
 ICE134I 0 NUMBER OF BYTES SORTED: 60040255422
 ICE253I 0 RECORDS SORTED - PROCESSED: 540903202, EXPECTED: 0
 ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 1350000 , TRACKS USED: 1344420
 ICE199I 0 MEMORY OBJECT USED AS MAIN STORAGE = 0M BYTES
 ICE299I 0 MEMORY OBJECT USED AS WORK STORAGE = 6696M BYTES
 ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES
 ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
 ICE052I 0 END OF DFSORT
 (AAAAAAA) ARRRRR   - XXXX - OPENED: RDM INPUT          3 READS,         0 WRITES,         0 REWRITES,         0 DELETES
 (AAAAAAA) ARRRRR   - YYYY - OPENED: SEQ INPUT  540903204 READS,         0 WRITES,         0 REWRITES
 (CCSBBBB) ARRRRR - ZZZZ - OPENED: DYN I-O            0 READS,         1 WRITES,         0 REWRITES,         0 DELETES
 



As you can see, the work-storage usage spiked considerably on the day of failure.

Nevertheless, it'd be great if you can help me with point-c.

Thank you.
Aki88
 
Posts: 381
Joined: Tue Jan 28, 2014 1:52 pm
Has thanked: 33 times
Been thanked: 36 times

Re: MOSIZE Calculation

Postby Aki88 » Tue Oct 25, 2016 4:20 pm

Missed adding:

The B37 (ICE096I) of the last screenshot subsided the next day and the subsequent runs- to date.

Would be great if a recommendation is made on FILSZ usage as well- should we use it or can we live without it; knowing the fact that the data is going to grow over-time, by around 30% per quarter.
Aki88
 
Posts: 381
Joined: Tue Jan 28, 2014 1:52 pm
Has thanked: 33 times
Been thanked: 36 times

Next

Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post