SORT a Split record



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

SORT a Split record

Postby krishnakumar15 » Thu Jul 17, 2014 12:10 pm

Scenario:
I have a student information in flat file. Each Student record is of 100 bytes/Length.I have a file length constraint in PS file LRECL=50 (stud.file1). So each student record is stored in two rows (50+50) rows per student.

Layout of student record (100 bytes):
01 student.
05 Regno pic 9(10)
05 Addr pic X(40)
05 descr pic X(50)

Task do:
Sort the Student record file(stud.file1) based on REGNO and store output in 2 different files formats

outputfile1(LRECL=50)
outputfile2 (LRECL=100)

Thanks in advance
krishnakumar15
 
Posts: 3
Joined: Thu Jul 17, 2014 11:55 am
Has thanked: 0 time
Been thanked: 0 time

Re: SORT a Split record

Postby Magesh23586 » Thu Jul 17, 2014 1:26 pm

You question is confusing

Input file is of Length 100 ?

You need output (Length 50) in one file and (length of 100) in other file ?

If so what is the layout of the output file for Length 50.

Specify your requirements clearly to suggest.

Thanks
Magesh
Magesh23586
 
Posts: 36
Joined: Sat Jul 05, 2014 5:36 pm
Has thanked: 1 time
Been thanked: 3 times

Re: SORT a Split record

Postby krishnakumar15 » Thu Jul 17, 2014 6:23 pm

Clarification

Length/LRECL
Input file =>Lrecl=50
outputfile1=>LRECL=50
outputfile2=> LRECL=100


I need two sorted output files having all records but different file size to be handled.

E.g:
Input file1:(Lrecl=50)
999999999sdfsdfsdfsdfsdfsdfsdfsdfsdfsdf...
dfsdfsdf..........................................
525252525sdfsdfsdfsdfsdfsdfsdfsdfsdfsdf...
dfsdfsdf..........................................

Output file1:(Lrecl=50)
525252525sdfsdfsdfsdfsdfsdfsdfsdfsdfsdf...
dfsdfsdf..........................................
999999999sdfsdfsdfsdfsdfsdfsdfsdfsdfsdf...
dfsdfsdf..........................................

Output file2:(Lrecl=100)
525252525sdfsdfsdfsdfsdfsdfsdfsdfsdfsdf...dfsdfsdf..........................................
999999999sdfsdfsdfsdfsdfsdfsdfsdfsdfsdf...dfsdfsdf..........................................

Hope this clears what is needed
krishnakumar15
 
Posts: 3
Joined: Thu Jul 17, 2014 11:55 am
Has thanked: 0 time
Been thanked: 0 time

Re: SORT a Split record

Postby Magesh23586 » Thu Jul 17, 2014 9:11 pm

Try this and let us know the results.
//CPYJK EXEC PGM=ICETOOL                   
//TOOLMSG DD SYSOUT=*                     
//SYSOUT DD SYSOUT=*                       
//DFSMSG DD SYSOUT=*                       
//IN1 DD  DSN=INPUT1 OF LENGTH 50         
//OUT1 DD DSN=OUT1 OF LENGTH 50           
//OUT2 DD DSN=OUT2 OF LENGTH 100           
//TOOLIN DD *                             
  COPY FROM(IN1) TO(OUT1)                 
  RESIZE FROM(OUT1) TO(OUT2) TOLEN(100)   
/*                                         


Thanks
Magesh
Magesh23586
 
Posts: 36
Joined: Sat Jul 05, 2014 5:36 pm
Has thanked: 1 time
Been thanked: 3 times

Re: SORT a Split record

Postby BillyBoyo » Fri Jul 18, 2014 1:33 am

Maghesh,

No need to read the file twice. You can have two OUTFILs for two output files.

krishnakumar15,

You need to make pairs of records into one record and output that, and also output the pairs. Both output files need to be in key sequence, which is data on the first record only.

Amongst other ways, you can pass data from one record to another using PUSH on IFTHEN=(WHEN=GROUP.

How do you define the GROUP? Since they are in pairs, the easiest way is with a SEQUENCE number appended to the record, and start a GROUP when the sequence number is odd, using RECORDS=2 for the pairs.

PUSH the whole record to the new end of the record. You will know have to records, both with a key in the same place. With OPTION EQUALS, SORT on the key.

Have two OUTFILs. One to take all the records, and just BUILD for the original record length (50).

The second OUTFIL only to take the even-numbered records, and use BUILD to put those into the format needed (2nd-extension-start,50,1,50).

There's quite a lot there. Work on it one piece at a time, once you know and have one thing working, move on to the next.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: SORT a Split record

Postby Magesh23586 » Fri Jul 18, 2014 11:17 am

Is this what you are telling, This solution is complex and little tough to understand for new people but yes I agree 1 pass - two copy...

//STEP01   EXEC  PGM=SORT                                       
 //SYSOUT  DD SYSOUT=*                                           
 //SORTIN     DD DSN=...  input file (FB/50)                       
 //OUT DD DSN=...  output file (FB/100)     
 //OUT1 DD DSN=...  output file (FB/50)     
 //SYSIN  DD *                                                   
  OPTION COPY                                                   
  INREC IFTHEN=(WHEN=GROUP,RECORDS=2,PUSH=(51:1,50,101:SEQ=1))
  OUTFIL FNAMES=OUT,INCLUDE=(101,1,ZD,EQ,2),                     
      BUILD=(51,50,1,50)
    OUTFIL FNAMES=OUT1,INCLUDE=(101,1,ZD,EQ,1),                     
      BUILD=(1,50)
/*
Magesh23586
 
Posts: 36
Joined: Sat Jul 05, 2014 5:36 pm
Has thanked: 1 time
Been thanked: 3 times

Re: SORT a Split record

Postby BillyBoyo » Fri Jul 18, 2014 12:05 pm

That's close, but that's going to pair each consecutive record.

With SORT, it is a complex task to give to a beginner. The easier way to achieve it is in separate steps. The way to mimic the easier solution in the development of the complex solution is to do it a bit at a time, to break it into separate steps, but instead of keeping them separate, to build up each time.
KEY1 DATA1A
DATA1B
KEY2 DATA2A
DATA2B


There's a representation of the data. First thing is to arrange it like this:

KEY1 DATA1A KEY1 DATA1A
DATA1B      KEY1 DATA1A
KEY2 DATA2A KEY2 DATA2A
DATA2B      KEY2 DATA2A


The KEY1 group now has one record with data duplicated, and one record with all the data. On each record, the value of KEY1 is in the same place.

The file can now be SORTed on the key (with OPTION EQUALS, important).

Now we have the same data, but in the order we want. Need to turn it into the output we want.

We need "short" records. These are easy, as we have pairs of records per key, with order they were in on input retained (through OPTION EQUALS).

OUTFIL FNAMES=SHORT,BUILD=(1,50)

On SHORT, we have sorted pairs of 50-byte records now.

For the long records, we don't need the first record of each pair, as all the data is already in the second:

OUTFIL FNAMES=LONG,INCLUDE=/OMIT=(condition which gives us first record)

This is where we say "wouldn't it be handy if we had a count on each record which showed its position in the group"? That is the SEQ on the PUSH on the WHEN=GROUP, which will give either 1 or 2. Then just pick the 2s (or exclude the 1s).

Then, with the selection out of the way, just have to unscramble the order from the second record, BUILD=(51,50,1,50) and it is done.

That leaves the question of "but how do I get the data to look that way, the rest looks OK"? You know you need WHEN=GROUP, it is just that they are separate groups of two records, not intersecting groups. If there were a sequence number on the records, each record which starts the group would be an odd-numbered one.

OK, there isn't a sequence number, but we can make one. IFTHEN=(WHEN=NONE and stick a sequence number long enough for the size of the file at position 101 (after where we will put the data).

How to test the sequence number for being odd is then actually the trickiest part of the whole thing...
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: SORT a Split record

Postby Magesh23586 » Fri Jul 18, 2014 12:40 pm

It would be great, if you could provide a sample code demonstrating what you are telling.

Thanks
Magesh
Magesh23586
 
Posts: 36
Joined: Sat Jul 05, 2014 5:36 pm
Has thanked: 1 time
Been thanked: 3 times

Re: SORT a Split record

Postby BillyBoyo » Fri Jul 18, 2014 12:54 pm

Well, you've got yours really close already. Take off the INCLUDE where you are outputting the 50-byte records.

Then there's just the fun part, the most interesting part as an exercise. How to get the WHEN=GROUP to work how I want it to. There are several ways. By experimentation you get to learn a whole lot about WHEN=GROUP. If I cough-up now, you just get to see what I've already done.

Take both the OUTFILs away from your code and run it with some sample data.

Add a WHEN=NONE to put a sequence number on. And to start with, test for the sequence-field being one to set the GROUP. Does that give correct results for the first pair of records? If it does, how about another WHEN=NONE with a bit of maths to try to get all odd numbers to be one (or some other single value).
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times


Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post