Improve time with a large file
Posted: Tue Dec 27, 2011 12:49 pm
I have a COBOL program using INPUT-PROCEDURE OUTPUT-PROCEDURE to do an internal sort (z/OS DFSORT 1.10) for around 7 million records and I am trying to get a performance improvement. The program is not compiled with FASTSRT. The job JCL has REGION=0M
Operations claim that with our previous sort product, it took less than half the time and used just one SORTWK file, whereas with DFSORT now it requires 22.
I have attached the SORT log below. Is there anything there that may give a clue on performance improvement and reduce SORTWK files.
- 21.30 MINUTES EXECUTION TIME
22 //SORTWK01 DD DSN=&&TEMP01,UNIT=3390,SPACE=(CYL,(3200,50),RLSE),
// DISP=(,DELETE)
//SORTWK02....... etc
43 //SORTWK22 DD DSN=DCBK.T4,UNIT=3390,SPACE=(CYL,(3100,50),RLSE),
// DISP=(,DELETE)
44 //SORTDIAG DD DUMMY
45 //OTSFMEX DD DSN=DBP2.SFF005.DA.RPTTEST,DISP=(,CATLG,DELETE),
// UNIT=3390,SPACE=(CYL,(600,150),RLSE),DATACLAS=COMPS,
// DCB=(RECFM=VB,LRECL=7004,BLKSIZE=0)
46 //SYSTSIN DD *
47 //SYSIN DD *
48 //IGZSRTCD DD *
OPTION FILSZ=E7000000,AVGRLEN=7006,MAINSIZE=MAX,DYNSPC=512
1ICE805I 0 JOBNAME: SFJF10DX , STEPNAME: SFAF105P
ICE802I 0 BLOCKSET TECHNIQUE IN CONTROL
ICE201I G RECORD TYPE IS V - DATA STARTS IN POSITION 5
ICE992I 0 RA 0 WR 0 TR 0
ICE751I 0 C5-K51707 C6-K51707 C7-K54603 C8-K51707 E4-K51707 C9-BASE E5-K55337 B0-K48417 B0-K48417 E7-K51707
ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
ICE000I 0 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R10 - 07:29 ON TUE DEC 27, 2011 -
0 SORT FIELDS=(0083,0002,CH,A,0011,0042,CH,A)
RECORD TYPE=V,LENGTH=(007006,,,000005,)
OPTION FILSZ=E7000000,AVGRLEN=7006,MAINSIZE=MAX,DYNSPC=512
ICE201I G RECORD TYPE IS V - DATA STARTS IN POSITION 5
ICE992I 0 RA 0 WR 0 TR 0
ICE751I 0 C5-K51707 C6-K51707 C7-K54603 C8-K51707 E4-K51707 C9-BASE E5-K55337 B0-K48417 B0-K48417 E7-K51707
ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
ICE089I 6 SFJF10DX.SFAF105P. , INPUT LRECL = 7006, TYPE = V
ICE093I 0 MAIN STORAGE = (MAX,67108864,67108864)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (67011824,67011824)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=512
ICE128I 0 OPTIONS: SIZE=67108864,MAXLIM=1048576,MINLIM=450560,EQUALS=Y,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT
ICE129I 0 OPTIONS: VIO=N,RESDNT=NONE,SMF=NO ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=N ,ABCODE=MSG
ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW=Y,DSA=64
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE750I 0 DC 0 TC 0 CS DSVXX KSZ 48 VSZ 48
ICE752I 0 FSZ=7000000 RE IGN=0 C AVG=7006 7006 WSP=63697130 E DYN=0 0
ICE915I 0 MOFSZ=62204,MOSZ=1418,MOSYS=1578(5),MOSTG=1578,MEML=17592186040320(3)
ICE880I 0 QP=89 QA=89 HI=99 LI=11 MI=99 TZ=363010 N1=363008 N2=363008 SZ=90 HN=1
ICE916I 0 MOFR=0002,MOVR=D0
ICE996I 0 ESM=762624,ESO=762624,ESR=0,ESP=4096,ESS=16384,CES=762624,HSZ=524288
ICE997I 0 HWSP=29932861,HMAX=403970,HES=0,ASV=403970,EQ=I1,HN=0
ICE898I 0 OMAX=213783,NMAX=403970,TMAX=403970,CMAX=0,HU=0,BUN=0,MD=N6,DU=0,DR=0,HN=0
ICE889I 0 CT=MAX , SB=3, L=0, D=0000, CCW=1MAM
ICE901I 0 W 01NP17 02NP17 03NP17 04NP17 05NP17 06NP17 07NP17 08NP17
ICE901I 0 W 09NP17 10NP17 11NP17 12NP17 13NP17 14NP17 15NP17 16NP17
ICE901I 0 W 17NP17 18NP17 19NP17 20NP17 21NP17 22NP17
ICE751I 1 B1-K47755 B2-K47755 FA-K47755 B3-K47755 EA-BASE F1-K38900 E8-K51707
ICE091I 0 OUTPUT LRECL = 7006, TYPE = V
ICE055I 0 INSERT 6891774, DELETE 6891774
ICE054I 0 RECORDS - IN: 0, OUT: 0
ICE134I 0 NUMBER OF BYTES SORTED: 48283768644
ICE253I 0 RECORDS SORTED - PROCESSED: 6891774, EXPECTED: 7000000
ICE098I 0 AVERAGE RECORD LENGTH - PROCESSED: 7006, EXPECTED: 7006
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 896250 , TRACKS USED: 853515
ICE199I 0 MEMORY OBJECT STORAGE USED = 1418M BYTES
ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES
ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
ICE891I 1 67057192 WMAIN, 11992 CMAIN, MAX CALLOC, N SCN, B BA, 0 AZ, 0 BZ, NN QC, 0 CZ, 0 DZ, 1 PLE
ICE892I 1 7006 RIN 0 BLI 7010 BLO 7006 RUN 56664 BUN 2097 CPU 00 CVC
ICE893I 1 0 XIN 510 WIN 197119 GIN NDEY PFP3 B00 CM000 CIX UPTH LMV VS RUX
ICE894I 0 12 STR 12 MOR 1052 IPB 0 OPB 56901 CYL O MN
ICE881I 1 EQ=I1 DX=0 D2=0 D3=0 D4=89 AS=0 SA=0 SB=0 SC=0 HN=1
ICE896I 0 57009 SET 2563586 DEXTOT 852478 BLK 7000000 CSZ 115129 WE
ICE804I 1 SORTWK22 EXCP COUNT: 6096
ICE804I 1 SORTWK21 EXCP COUNT: 6087
ICE804I 1 SORTWK20 EXCP COUNT: 6089
1ICE804I 1 SORTWK19 EXCP COUNT: 6095
ICE804I 1 SORTWK18 EXCP COUNT: 2329
ICE804I 1 SORTWK17 EXCP COUNT: 2328
ICE804I 1 SORTWK16 EXCP COUNT: 2326
ICE804I 1 SORTWK15 EXCP COUNT: 2327
ICE804I 1 SORTWK14 EXCP COUNT: 2332
ICE804I 1 SORTWK13 EXCP COUNT: 6085
ICE804I 1 SORTWK12 EXCP COUNT: 6087
ICE804I 1 SORTWK11 EXCP COUNT: 6083
ICE804I 1 SORTWK10 EXCP COUNT: 6084
ICE804I 1 SORTWK09 EXCP COUNT: 6087
ICE804I 1 SORTWK08 EXCP COUNT: 6091
ICE804I 1 SORTWK07 EXCP COUNT: 6081
ICE804I 1 SORTWK06 EXCP COUNT: 6087
ICE804I 1 SORTWK05 EXCP COUNT: 6091
ICE804I 1 SORTWK04 EXCP COUNT: 6084
ICE804I 1 SORTWK03 EXCP COUNT: 6088
ICE804I 1 SORTWK02 EXCP COUNT: 6082
ICE804I 1 SORTWK01 EXCP COUNT: 6090
ICE052I 0 END OF DFSORT
Operations claim that with our previous sort product, it took less than half the time and used just one SORTWK file, whereas with DFSORT now it requires 22.
I have attached the SORT log below. Is there anything there that may give a clue on performance improvement and reduce SORTWK files.
- 21.30 MINUTES EXECUTION TIME
22 //SORTWK01 DD DSN=&&TEMP01,UNIT=3390,SPACE=(CYL,(3200,50),RLSE),
// DISP=(,DELETE)
//SORTWK02....... etc
43 //SORTWK22 DD DSN=DCBK.T4,UNIT=3390,SPACE=(CYL,(3100,50),RLSE),
// DISP=(,DELETE)
44 //SORTDIAG DD DUMMY
45 //OTSFMEX DD DSN=DBP2.SFF005.DA.RPTTEST,DISP=(,CATLG,DELETE),
// UNIT=3390,SPACE=(CYL,(600,150),RLSE),DATACLAS=COMPS,
// DCB=(RECFM=VB,LRECL=7004,BLKSIZE=0)
46 //SYSTSIN DD *
47 //SYSIN DD *
48 //IGZSRTCD DD *
OPTION FILSZ=E7000000,AVGRLEN=7006,MAINSIZE=MAX,DYNSPC=512
1ICE805I 0 JOBNAME: SFJF10DX , STEPNAME: SFAF105P
ICE802I 0 BLOCKSET TECHNIQUE IN CONTROL
ICE201I G RECORD TYPE IS V - DATA STARTS IN POSITION 5
ICE992I 0 RA 0 WR 0 TR 0
ICE751I 0 C5-K51707 C6-K51707 C7-K54603 C8-K51707 E4-K51707 C9-BASE E5-K55337 B0-K48417 B0-K48417 E7-K51707
ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
ICE000I 0 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R10 - 07:29 ON TUE DEC 27, 2011 -
0 SORT FIELDS=(0083,0002,CH,A,0011,0042,CH,A)
RECORD TYPE=V,LENGTH=(007006,,,000005,)
OPTION FILSZ=E7000000,AVGRLEN=7006,MAINSIZE=MAX,DYNSPC=512
ICE201I G RECORD TYPE IS V - DATA STARTS IN POSITION 5
ICE992I 0 RA 0 WR 0 TR 0
ICE751I 0 C5-K51707 C6-K51707 C7-K54603 C8-K51707 E4-K51707 C9-BASE E5-K55337 B0-K48417 B0-K48417 E7-K51707
ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
ICE089I 6 SFJF10DX.SFAF105P. , INPUT LRECL = 7006, TYPE = V
ICE093I 0 MAIN STORAGE = (MAX,67108864,67108864)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (67011824,67011824)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=512
ICE128I 0 OPTIONS: SIZE=67108864,MAXLIM=1048576,MINLIM=450560,EQUALS=Y,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT
ICE129I 0 OPTIONS: VIO=N,RESDNT=NONE,SMF=NO ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=N ,ABCODE=MSG
ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW=Y,DSA=64
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE750I 0 DC 0 TC 0 CS DSVXX KSZ 48 VSZ 48
ICE752I 0 FSZ=7000000 RE IGN=0 C AVG=7006 7006 WSP=63697130 E DYN=0 0
ICE915I 0 MOFSZ=62204,MOSZ=1418,MOSYS=1578(5),MOSTG=1578,MEML=17592186040320(3)
ICE880I 0 QP=89 QA=89 HI=99 LI=11 MI=99 TZ=363010 N1=363008 N2=363008 SZ=90 HN=1
ICE916I 0 MOFR=0002,MOVR=D0
ICE996I 0 ESM=762624,ESO=762624,ESR=0,ESP=4096,ESS=16384,CES=762624,HSZ=524288
ICE997I 0 HWSP=29932861,HMAX=403970,HES=0,ASV=403970,EQ=I1,HN=0
ICE898I 0 OMAX=213783,NMAX=403970,TMAX=403970,CMAX=0,HU=0,BUN=0,MD=N6,DU=0,DR=0,HN=0
ICE889I 0 CT=MAX , SB=3, L=0, D=0000, CCW=1MAM
ICE901I 0 W 01NP17 02NP17 03NP17 04NP17 05NP17 06NP17 07NP17 08NP17
ICE901I 0 W 09NP17 10NP17 11NP17 12NP17 13NP17 14NP17 15NP17 16NP17
ICE901I 0 W 17NP17 18NP17 19NP17 20NP17 21NP17 22NP17
ICE751I 1 B1-K47755 B2-K47755 FA-K47755 B3-K47755 EA-BASE F1-K38900 E8-K51707
ICE091I 0 OUTPUT LRECL = 7006, TYPE = V
ICE055I 0 INSERT 6891774, DELETE 6891774
ICE054I 0 RECORDS - IN: 0, OUT: 0
ICE134I 0 NUMBER OF BYTES SORTED: 48283768644
ICE253I 0 RECORDS SORTED - PROCESSED: 6891774, EXPECTED: 7000000
ICE098I 0 AVERAGE RECORD LENGTH - PROCESSED: 7006, EXPECTED: 7006
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 896250 , TRACKS USED: 853515
ICE199I 0 MEMORY OBJECT STORAGE USED = 1418M BYTES
ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES
ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
ICE891I 1 67057192 WMAIN, 11992 CMAIN, MAX CALLOC, N SCN, B BA, 0 AZ, 0 BZ, NN QC, 0 CZ, 0 DZ, 1 PLE
ICE892I 1 7006 RIN 0 BLI 7010 BLO 7006 RUN 56664 BUN 2097 CPU 00 CVC
ICE893I 1 0 XIN 510 WIN 197119 GIN NDEY PFP3 B00 CM000 CIX UPTH LMV VS RUX
ICE894I 0 12 STR 12 MOR 1052 IPB 0 OPB 56901 CYL O MN
ICE881I 1 EQ=I1 DX=0 D2=0 D3=0 D4=89 AS=0 SA=0 SB=0 SC=0 HN=1
ICE896I 0 57009 SET 2563586 DEXTOT 852478 BLK 7000000 CSZ 115129 WE
ICE804I 1 SORTWK22 EXCP COUNT: 6096
ICE804I 1 SORTWK21 EXCP COUNT: 6087
ICE804I 1 SORTWK20 EXCP COUNT: 6089
1ICE804I 1 SORTWK19 EXCP COUNT: 6095
ICE804I 1 SORTWK18 EXCP COUNT: 2329
ICE804I 1 SORTWK17 EXCP COUNT: 2328
ICE804I 1 SORTWK16 EXCP COUNT: 2326
ICE804I 1 SORTWK15 EXCP COUNT: 2327
ICE804I 1 SORTWK14 EXCP COUNT: 2332
ICE804I 1 SORTWK13 EXCP COUNT: 6085
ICE804I 1 SORTWK12 EXCP COUNT: 6087
ICE804I 1 SORTWK11 EXCP COUNT: 6083
ICE804I 1 SORTWK10 EXCP COUNT: 6084
ICE804I 1 SORTWK09 EXCP COUNT: 6087
ICE804I 1 SORTWK08 EXCP COUNT: 6091
ICE804I 1 SORTWK07 EXCP COUNT: 6081
ICE804I 1 SORTWK06 EXCP COUNT: 6087
ICE804I 1 SORTWK05 EXCP COUNT: 6091
ICE804I 1 SORTWK04 EXCP COUNT: 6084
ICE804I 1 SORTWK03 EXCP COUNT: 6088
ICE804I 1 SORTWK02 EXCP COUNT: 6082
ICE804I 1 SORTWK01 EXCP COUNT: 6090
ICE052I 0 END OF DFSORT