DFSORT differences with short SMF records



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

DFSORT differences with short SMF records

Postby belairdino » Tue Jun 10, 2008 7:14 pm

I built a DFSORT step to select SMF type 30 records for selected jobnames -- sent to a coworker to run against his data. The step works on my system, but fails on his with
ICE218A 6 18 BYTE VARIABLE RECORD IS SHORTER THAN ...


Neither of us had specified OPTION VLSHRT -- why does this work on my system but not on his? Both are running V1R5, but it's clear there are differences in our ICE751I msgs.

(btw, it does work for him with VLSHRT specified)

Note that we have differences in our condition for the jobname location -- based on our specific SMF30IOF offsets -- but I don't think that should matter.

My SYSOUT msgs:

ICE143I 0 BLOCKSET     COPY  TECHNIQUE SELECTED                                                                         
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE                                   
ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R5 - 06:56 ON TUE JUN 10, 2008 -                             
           SORT FIELDS=COPY                                                                                             
           INCLUDE COND=((6,1,BI,EQ,30),AND,                                                                             
                         (203,1,CH,EQ,C'A'))                                                                             
          * 1st change----***                                                                                           
          * 2nd change--------*-------*******                                                                           
ICE201I E RECORD TYPE IS V - DATA STARTS IN POSITION 5                                                                   
ICE751I 0 C5-K26318 C6-K90007 C7-K90000 C8-K23476 E9-K90007 C9-BASE   E5-K24705 E7-K24705                               
ICE193I 0 ICEAM1 ENVIRONMENT IN EFFECT - ICEAM1 INSTALLATION MODULE SELECTED                                             
ICE088I 0 CIMS02XX.STEP020 .        , INPUT LRECL = 32767, BLKSIZE = 27998, TYPE = VBS                                   
ICE093I 0 MAIN STORAGE = (MAX,6291456,6291456)                                                                           
ICE156I 0 MAIN STORAGE ABOVE 16MB = (6141226,6141226)                                                                   
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256             
ICE128I 0 OPTIONS: SIZE=6291456,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT                   
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO   ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=N             ,ABCODE=MSG     
ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,STIMER=Y,COBEXIT=COB2                 
ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=Y,DSA=0                                 
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N           
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX                           
ICE235I 0 OPTIONS: NULLOUT=RC0   
ICE084I 0 EXCP ACCESS METHOD USED FOR SORTOUT                                         
ICE084I 0 EXCP ACCESS METHOD USED FOR SORTIN                                           
ICE751I 1 EF-K10929 F0-Q84357 E8-K24705                                               
ICE090I 0 OUTPUT LRECL = 32767, BLKSIZE = 27998, TYPE = VBS  (SDB)                     
ICE055I 0 INSERT 0, DELETE 3050                                                       
ICE054I 0 RECORDS - IN: 3359, OUT: 309                                                 
ICE052I 0 END OF DFSORT


His SYSOUT msgs:

1ICE143I 0 BLOCKSET     COPY  TECHNIQUE SELECTED
 ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
 ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R5 - 14:16 ON TUE JUN 10, 2008 -
0           SORT FIELDS=COPY
            INCLUDE COND=((6,1,BI,EQ,30),AND,
                          (215,4,CH,EQ,C'CX41'))
           * 1st change----***
           * 2nd change--------*-------*******
 ICE201I 0 RECORD TYPE IS V - DATA STARTS IN POSITION 5
 ICE751I 0 C5-K05352 C6-Q95214 C7-K90000 C8-K05352 E9-K06751 C9-BASE   E5-K10929 E7-K11698
 ICE193I 0 ICEAM1 ENVIRONMENT IN EFFECT - ICEAM1 INSTALLATION MODULE SELECTED
 ICE088I 0 YS84CIMS.STEP020 .        , INPUT LRECL = 32767, BLKSIZE = 4096, TYPE = VBS
 ICE093I 0 MAIN STORAGE = (MAX,6291456,6274142)
 ICE156I 0 MAIN STORAGE ABOVE 16MB = (6074866,6074866)
 ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
 ICE128I 0 OPTIONS: SIZE=6291456,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT
 ICE129I 0 OPTIONS: VIO=N,RESDNT=NONE,SMF=NO   ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=N             ,ABCODE=MSG
 ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,STIMER=Y,COBEXIT=COB2
 ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=Y,DSA=0
 ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=N,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
 ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
 ICE235I 0 OPTIONS: NULLOUT=RC0
 ICE084I 0 EXCP ACCESS METHOD USED FOR SORTOUT
 ICE084I 0 BSAM ACCESS METHOD USED FOR SORTIN
 ICE218A 6 18 BYTE VARIABLE RECORD IS SHORTER THAN 218 BYTE MINIMUM FOR          FIELDS
 ICE751I 1 EF-K10929 F0-Q84357 E8-K11698
 ICE052I 0 END OF DFSORT


Thanks in advance,

Scott
belairdino
 
Posts: 4
Joined: Tue Jun 10, 2008 6:43 pm
Has thanked: 0 time
Been thanked: 0 time

Re: DFSORT differences with short SMF records

Postby Frank Yaeger » Tue Jun 10, 2008 8:37 pm

The reason why he gets the message and you don't is that in his case, the input data set has a record shorter than 218 bytes, whereas in your case the input data set does not have a record shorter than 203 bytes. That's the furthest out position for each of your runs, respectively.

BTW, Since you're not sorting, you might want to use the VLSCMP option rather than the VLSHRT option. See the "How can I use INCLUDE/OMIT with "short" fields?" Ask Professor Sort item at:

http://www.ibm.com/servers/storage/supp ... ssor_sort/

for more information.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1079
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 15 times

Re: DFSORT differences with short SMF records

Postby belairdino » Tue Jun 10, 2008 9:54 pm

Frank,

Thanks for your note (and for the pointer on VLSCMP).

I should have mentioned that I do have short records -- my input dataset contains SMF type 2/3 records which are only 18 bytes long, and I'm certain his input dataset does also -- that's why I don't understand the two different behaviors.

I'm probably hoping something in our separate DFSORT environments (defaults, maintenance levels, etc.) proves to be the culprit -- but willing to dig around for anything.

Scott
belairdino
 
Posts: 4
Joined: Tue Jun 10, 2008 6:43 pm
Has thanked: 0 time
Been thanked: 0 time

Re: DFSORT differences with short SMF records

Postby Frank Yaeger » Wed Jun 11, 2008 6:21 am

I looked at your messages and his messages and you both have the defaults of VLSCMP=N and VLSHRT=N.

I have to leave now. Perhaps you can use ICETOOL STATS with ON(VLEN) to check the minimum length of your records and we can proceed from there.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1079
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 15 times

Re: DFSORT differences with short SMF records

Postby belairdino » Thu Jun 12, 2008 5:15 pm

Here's the STATS ON(VLEN) result against my data:

           STATS   FROM(SYSUT1) ON(VLEN)                               
ICE627I 0 DFSORT CALL 0001 FOR COPY FROM SYSUT1   TO E35 EXIT COMPLETED
ICE628I 0 RECORD COUNT:  000000000011855                               
ICE607I 0 STATISTICS FOR (VLEN)        :                               
ICE608I 0   MINIMUM:  +000000000000018, MAXIMUM:  +000000000010015     
ICE609I 0   AVERAGE:  +000000000001998, TOTAL  :  +000000023696293     
ICE602I 0 OPERATION RETURN CODE:  00                                                                       


My co-worker's runs are from a client site in another country/timezone - trying to get him to run STATS ON(VLEN) when he can (but we already know he has short records!).

Scott
belairdino
 
Posts: 4
Joined: Tue Jun 10, 2008 6:43 pm
Has thanked: 0 time
Been thanked: 0 time

Re: DFSORT differences with short SMF records

Postby Frank Yaeger » Thu Jun 12, 2008 8:21 pm

Scott,

This doesn't make sense to me. I can see from the messages you posted that your version of DFSORT has service applied that your friend doesn't, but I don't see why that would make a difference. I'm on vacation for a couple of days. Here's what I'd like you to do while I'm gone:

Run your SMF step with the STATS step right after it. Make sure the SORTIN of the SMF step and SYSUT1 of the STATS step are pointing to the same data set. Send the source for your JCL and control statements, the //SYSOUT messages from the SMF step and the //TOOLMSG and //DFSMSG messages from the STATS step to the following e-mail ids:

yaeger@us.ibm.com
skolusu@us.ibm.com

Please put "DFSORT" somewhere in your Subject line to catch my attention.

We'll take a look to see what's going on.

(We don't need anything from your friend.)
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1079
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 15 times

Re: DFSORT differences with short SMF records

Postby belairdino » Fri Jun 13, 2008 1:28 am

Frank,

It's time for me to be sheepish. Turns out my SYSUT1 in the first step is a concatenated dd, with the short records in the second dd. My second (SORT) step does a referback to firststep.SYSUT1, but that picks up only the first dd.

So my SORT step never sees the short records -- entirely my bad :oops: .

I've spent too many years playing in the distributed world -- I guess I should be happy that I've retained whatever JCL skills I might still have.

Thanks again for your efforts,

Scott
belairdino
 
Posts: 4
Joined: Tue Jun 10, 2008 6:43 pm
Has thanked: 0 time
Been thanked: 0 time

Re: DFSORT differences with short SMF records

Postby Frank Yaeger » Mon Jun 16, 2008 3:51 am

Scott,

That's ok. That makes me feel better. I was beginning to wonder if we had a "bug" in one of our DFSORT PTFs. I'm glad to hear we don't.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1079
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 15 times


Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post