Different behavior when SYSIN's content is inside a dataset.

[dick scherrer]

Hello,

How does one read the same file 2 times and Not get the same filesize?

[tivrfoa]

Hello,

How does one read the same file 2 times and Not get the same filesize?

Hi Dick. Thanks for trying to help. I really appreciated the help from you guys, but you are only talking about the size.
Why there's no mention in the sysout about INREC and OUTFIL when using the dataset in the SYSIN?
Have anyone tried to reproduce what happened?
I was expecting that someone could try to reproduce what happened and if it shows the same thing for you with both runs, then there's something wrong here.

SYNCSORT FOR Z/OS 1.4.1.0N

sortin:

Code: Select all

0header
10000000124.09.201300000491700001234500020SNSN00006SSS00000S0000000002S08:00:00001002013-10-18-19.28.00.00000009:00:0014:00:0012:00:0013:00:000010031.12.99992013-10-18-19.28.00.00000000624455S000202013-10-18-19.28.00.00000012345678
10000000224.09.201300000491700001234500020SNSN00006SSS00000S0000000002S08:00:00001002013-10-18-19.28.00.00000009:00:0014:00:0012:00:0013:00:000010031.12.99992013-10-18-19.28.00.00000000624455S000202013-10-18-19.28.00.00000012345678
9trailer

edited: it wasn't showing all content of the sortin

[BillyBoyo]

You have to first realise that replacing DD * data with a DSN containing the Control Cards (usually a member of a PDS/PDSE) is a completely nornal thing to do, and it works. We can't recreate it, because it is something particular with your DSN.

I suspected, and hoped to get some confirmation from the sysout, that your DSN is variable, rather than fixed. No confirmation, at least not direct.

Some of your Control Cards are actioned. Some are not. Perhaps those which are not have line-numbers on?

Look at your DSN. Is it F or FB, 80? If not, it should be.

Your Control Cards themselves need some work, but one thing at a time.

[tivrfoa]

Hi BillyBoyo. Thanks a lot.

Looking at the sysouts I posted when you asked me, I realized that only the inline sysin shows this in the sysout:

Code: Select all

WER257I  INREC RECORD LENGTH =  239
WER238I  POTENTIALLY INEFFICIENT USE OF INREC
...
and
...
WER416B  OUTFIL WAS USED FOR SORTOUT
...

This is not shown when using the DSN. Do you have any thoughts on why this is happening? If it executes only SORT, but not INREC nor OUTFIL, then of course that's why I get different results.
Sorry, but what do you mean by "actioned" and "line-numbers on"?

I'll check tomorrow if it is FB/80 (without access now...), but I'm almost sure that it is. I think it gives an error if the lrecl is not 80.

Other test that I'll do is to delete the content of the DSN that contains the SYSIN and type everything, because this DSN is generated automatically by a previous SORT step, so maybe there's some garbage ...

[Robert Sample]

Hi Dick. Thanks for trying to help. I really appreciated the help from you guys, but you are only talking about the size.
Why there's no mention in the sysout about INREC and OUTFIL when using the dataset in the SYSIN?
Have anyone tried to reproduce what happened?
I was expecting that someone could try to reproduce what happened and if it shows the same thing for you with both runs, then there's something wrong here.

You keep trying to tell us you are getting different results running the same sort. We are telling you that since the size is different, you do NOT have the same sort -- the size is the accumulated length of all records being passed to the sort. If the sizes are different, the ONLY choices are that the record lengths are different, or that the record count is different. Hence, the sorts are not the same.

And why should ANYONE look at INREC / OUTFIL or anything else when you have different sorts? We know that different sorts will give different results so why try to reproduce that? And those of us with many years of experience know that the computer rarely lies to us -- if it tells us that the sorts are different (because the file sizes are different) then the sorts are not the same, no matter how often nor how loudly you proclaim otherwise. Furthermore, I have (as most likely have all of the senior people on this forum) converted inline code (for SORT or whatever) into a data set and gotten the exact same results so we know the behavior you are claiming is due to something you have done, not the system behaving differently with data sets or inline data.

[tivrfoa]

Hi Robert, please don't be mad with me.

I'm not being insane (I think so, I only drank on Saturday)

Here I posted the sysouts from both runs:
syncsort-synctool/topic9538.html#p47283

The sysin shows the INREC and OUTREC, but there's no mention about them in the rest of the sysout, while it's showed in the inline SYSIN's sysout. I'm just curious about that.

But that's ok. Nobody needs to reply anymore. I don't wanna cause headache in anyone. Have a nice week folks.

[BillyBoyo]

Robert,

The INREC is adding eight bytes to each record. 70002 records * eight gives the difference.

The OUTFIL has INCLUDE/OMIT to cut down the size of the output file.

If the INREC is not operating and the OUTFIL is not operating the you get the results we are seeing.

Yes, this shouldn't be happening and doesn't usually.

tivrfoa,

Please show the sysout from the step which creates the Control Cards. That step is what is causing the problem, somehow. Yes, I'd have expected a failure with a wrong-length record, but there is something about your DSN which is allowing some of the Control Cards to work, and ignoring others.

[tivrfoa]

Hi BillyBoyo,

I tested deleting the rows and pasting the logic and it worked =). So it really is the SORT that generates this DSN that's causing the problem, as you said.
The DSN is FB/80:
ALLOCATED TRACKS ===> 1 DSORG ===> PS
USED TRACKS ===> 1 RECFM ===> FB
ALLOCATED EXTENTS ===> 1 LRECL ===> 80

I have one dataset with two values, one date and one number, and I need to compare these values with the records in another dataset.
I first generate with LRECL 45, because with 80 it says “INCOMPATIBLEE LRECL”. Then in another step I copy the content to dataset that has LRECL equals 80.

Code: Select all

//SORT001   EXEC PGM=SORT,COND=(0,NE,SYS112)                         
//SYSOUT    DD SYSOUT=*                                             
//SORTIN    DD DSN=sortin-dataset,DISP=SHR       
//SORTOUT   DD DSN=temp-sysin-45,               
//          DISP=(,CATLG),RECFM=FB,DSORG=PS,LRECL=045,AVGREC=K,     
//          SPACE=(045,(100,100),RLSE)                               
//SYSIN     DD *                                                     
  SORT FIELDS=(01,16,ZD,D)                                           
  OUTFIL ENDREC=1,                                                   
         BUILD=(C'  SORT FIELDS=(01,01,ZD,A,   ** TIP-REG',/,       
                C'               16,04,ZD,A,   ** ANO DT-JTRB',/,   
                C'               13,02,ZD,A,   ** MES DT-JTRB',/,   
                C'               10,02,ZD,A,   ** DIA DT-JTRB',/,   
                C'               02,08,ZD,A)   ** NR-PSSL',/,       
                C'  INREC FIELDS=(01,231,16,04,13,02,10,02)',/,     
                C'  OUTFIL INCLUDE=((232,08,ZD,EQ,',01,08,C',AND,',/,
                C'                 02,08,ZD,GT,',09,08,C'),OR,',/,   
                C'                 232,08,ZD,GT,',01,08,C',OR,',/,   
                C'                 1,1,ZD,EQ,0,OR,1,1,ZD,EQ,9),',/, 
                C'       OUTREC=(01,231)')       

sysout:

Code: Select all

SYSIN :                                                                 
  SORT FIELDS=(01,16,ZD,D)                                               
  OUTFIL ENDREC=1,                                                       
         BUILD=(C'  SORT FIELDS=(01,01,ZD,A,   ** TIP-REG',/,           
                C'               16,04,ZD,A,   ** ANO DT-JTRB',/,       
                C'               13,02,ZD,A,   ** MES DT-JTRB',/,       
                C'               10,02,ZD,A,   ** DIA DT-JTRB',/,       
                C'               02,08,ZD,A)   ** NR-PSSL',/,           
                C'  INREC FIELDS=(01,231,16,04,13,02,10,02)',/,         
                C'  OUTFIL INCLUDE=((232,08,ZD,EQ,',01,08,C',AND,',/,   
                C'                 02,08,ZD,GT,',09,08,C'),OR,',/,       
                C'                 232,08,ZD,GT,',01,08,C',OR,',/,       
                C'                 1,1,ZD,EQ,0,OR,1,1,ZD,EQ,9),',/,     
                C'       OUTREC=(01,231)')                               
WER276B  SYSDIAG= 11146638, 15218035, 15218035, 5615775                 
WER164B  24,612K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,     
WER164B     0 BYTES RESERVE REQUESTED, 24,564K BYTES USED               
WER146B  12K BYTES OF EMERGENCY SPACE ALLOCATED                         
WER108I  SORTIN   : RECFM=FB   ; LRECL=    16; BLKSIZE= 27984           
WER073I  SORTIN   : DSNAME=xxx         
WER110I  SORTOUT  : RECFM=FB   ; LRECL=    45; BLKSIZE= 27990           
WER074I  SORTOUT  : DSNAME=xxx             
WER410B  23,584K BYTES OF VIRTUAL STORAGE AVAILABLE ABOVE THE 16MEG LINE,
WER410B     0 BYTES RESERVE REQUESTED, 23,568K BYTES USED               
WER036B  G=6996,B=3327                                                   
WER045C  END SORT PHASE                                                 
WER418I  DATASPACE(S) AND/OR ZSPACE USED                                 
WER405I  SORTOUT  :  DATA RECORDS OUT          1; TOTAL RECORDS OUT     
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000                           
WER416B  SORTIN   : EXCP'S=1,UNIT=3390,DEV=A32E,CHP=(D8D9DADB,1),VOL=DS0X
WER416B  OUTFIL WAS USED FOR SORTOUT                                     
WER246I  FILESIZE 1,120 BYTES                                           
WER054I  RCD IN         70, OUT         70                     
NOEQUALS, BALANCE IN EFFECT         
RELEASE 1.4 BATCH 0520 TPF LEVEL 1.0
END SYNCSORT - Jxxx,SORT001,,DIA         

Code: Select all

//SORT002   EXEC PGM=SORT,COND=(0,NE,SYS112)                   
//SYSOUT    DD SYSOUT=*                                       
//SORTIN    DD DSN=temp-sysin-45,DISP=SHR
//SORTOUT   DD DSN=sysin-80,         
//          DISP=(,CATLG),RECFM=FB,DSORG=PS,LRECL=080,AVGREC=K,
//          SPACE=(080,(100,100),RLSE)                         
//SYSIN     DD *                                               
  OPTION COPY   

sysout:

Code: Select all

SYSIN :                                                                 
  OPTION COPY                                                           
WER276B  SYSDIAG= 11144149, 15210510, 15210510, 5615775                 
WER164B  8,884K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,       
WER164B     0 BYTES RESERVE REQUESTED, 2,420K BYTES USED                 
WER146B  12K BYTES OF EMERGENCY SPACE ALLOCATED                         
WER108I  SORTIN   : RECFM=FB   ; LRECL=    45; BLKSIZE= 27990           
WER073I  SORTIN   : DSNAME=xxx             
WER110I  SORTOUT  : RECFM=FB   ; LRECL=    80; BLKSIZE= 27920           
WER074I  SORTOUT  : DSNAME=xxx             
WER462I  OUTPUT LRECL DIFFERS FROM SORTOUT LRECL                         
WER410B  7,856K BYTES OF VIRTUAL STORAGE AVAILABLE ABOVE THE 16MEG LINE,
WER410B     0 BYTES RESERVE REQUESTED, 2,276K BYTES USED                 
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000                           
WER416B  SORTIN   : EXCP'S=1,UNIT=3390,DEV=A0A7,CHP=(D8D9DADB,1),VOL=DS0X
WER416B  SORTOUT  : EXCP'S=1,UNIT=3390,DEV=A2CA,CHP=(D8D9DADB,1),VOL=DS0X
WER416B  TOTAL OF 2 EXCP'S ISSUED FOR COPYING                           
WER054I  RCD IN         11, OUT         11                               
WER169I  RELEASE 1.4 BATCH 0520 TPF LEVEL 1.0                           
WER052I  END SYNCSORT - Jxxx,SORT002,,DIAG=AE00,710A,8284,00CE,E66A,4

[tivrfoa]

Hi BillyBoyo,

Just to let you know that meanwhile I found a workaround, and probably it's a better aproach.
Instead of writing all the sort logic to the dataset, I write only the SYMNAMES, and then use it in the next step. It worked.

Code: Select all

//SYSIN     DD *                 
  SORT FIELDS=(01,16,ZD,D)       
  OUTFIL ENDREC=1,               
         BUILD=(C'DTJTRB,',1,8,/,
                C'NRPSSL,',9,8) 
//*

...
//SYMNAMES DD DSN=xxx,DISP=SHR
...

Code: Select all

SYSIN :                                                               
  INREC FIELDS=(01,231,16,04,13,02,10,02)                             
  SORT FIELDS=(01,01,ZD,A,   ** TIP-REG                               
               16,04,ZD,A,   ** ANO DT-JTRB                           
               13,02,ZD,A,   ** MES DT-JTRB                           
               10,02,ZD,A,   ** DIA DT-JTRB                           
               02,08,ZD,A)   ** NR-PSSL                               
  OUTFIL INCLUDE=((232,08,ZD,EQ,DTJTRB,AND,                           
                 02,08,ZD,GT,NRPSSL),OR,                               
                 232,08,ZD,GT,DTJTRB,OR,                               
                 1,1,ZD,EQ,0,OR,1,1,ZD,EQ,9),                         
       OUTREC=(01,231)                                                 
DATA DICTIONARY SYMBOLS SUBSTITUTED :         ohh yeahh             
INREC FIELDS=(01,231,16,04,13,02,10,02)                               
SORT FIELDS=(01,01,ZD,A,16,04,ZD,A,13,02,ZD,A,10,02,ZD,A,02,08,ZD,A)   
OUTFIL INCLUDE=((232,08,ZD,EQ,20130924,AND,02,08,ZD,GT,70000),OR,232,08,ZD,GT,20
130924,OR,1,1,ZD,EQ,0,OR,1,1,ZD,EQ,9),OUTREC=(01,231)                 

[BillyBoyo]

That's good. Some further things.

On INREC, OUTREC and OUTFIL it is better to use BUILD rather than FIELDS (FIELDS has multiple meanings on multiple statements) or OUTREC on OUTFIL (people get confused at the difference between OUTREC and OUTFIL OUTREC=).

I can see no reason not to use INCLUDE/OMIT rather than OUTFIL INCLUDE=/OMIT=. You SORT all the data and then throw away what you don't want, rather than throwing away what you don't want and then SORTing the data. Less resources the second way.

I see no need to extend the record with INREC. The SORT already uses the specific positions within the record, there is absolutely no reason that the INCLUDE= can't do the same.

It does require three symbols for DTJTRB, but given the amount of data (unknown to us) that may not need to be sorted, I'd strongly consider it. OK. I lie. I'd do it (make three symbols).

I like symbols/SYMNAMES. Why don't you:

Code: Select all

TIP-REG,01,01,ZD                               
ANO-DT-JTRB,16,04,ZD                           
MES-DT-JTRB,13,02,ZD                           
DIA-DT-JTRB,10,02,ZD                           
NR-PSSL,02,08,ZD 

and

Code: Select all

  SORT FIELDS=(TIP-REG,A,                               
               ANO-DT-JTRB,A,                           
               MES-DT-JTRB,A,                           
               DIA-DT-JTRB,A,                           
               NR-PSSL,A)

With an appropriate INCLUDE, you can have INCLUDE, SORT, no INREC, no OUTREC, and more documentation than your comments provided