How to Merge records in 2 ps files alternately into 3rd?



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

Re: How to Merge records in 2 ps files alternately into 3rd?

Postby BillyBoyo » Wed Jan 11, 2012 9:38 pm

What I am assuming is that:

Initially the first record is read from each of the SORTINxx files.

INREC, when present, intervenes and effectively the input records from the files become new input records to the merge process.

The merge process decides which key is lowest and passes that for output, via OUTFIL, OUTREC processing, whatever, if present.

The merge process then requests the next record from the file which had the lowest key.

The file to which that corresponds is read, INREC gets the record and makes a new record to pass to the merge process.

The flip-flopping is only happening because as each new input is processed, INREC assigns the next highest sequence number. The INREC will have been processed with each initial record from the files, prior to the merge processing being entered.

Internally to DFSORT, I'm not saying that I know how the "requests the next record" or anything works (is it pushed or pulled, as it were) but my solution was on the above assumptions. If it doesn't work like that, my solution is a bad one.

I had looked at the EQUALS processing for the MERGE to see if that would "get in the way", but since the keys are unique/unique-when the-merge-is-operating (like the suggestion that it would still work with a single digit) I used NOEQUALS, but with EQUALS merge would respect the order of the record appearing on the files when keys are equal - so, if SORTIN01 and SORTIN03 have equal merge keys, EQUALS will ensure the SORTIN01 record appears on the output before the SORTIN03 (assuming multiple-more-than-two SORTINxx's) record, whereas NOEQUALS would take whichever was the most convenient at that time.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: How to Merge records in 2 ps files alternately into 3rd?

Postby enrico-sorichetti » Wed Jan 11, 2012 10:25 pm

seems reasonable !
cheers
enrico
When I tell somebody to RTFM or STFW I usually have the page open in another tab/window of my browser,
so that I am sure that the information requested can be reached with a very small effort
enrico-sorichetti
Global moderator
 
Posts: 2994
Joined: Fri Apr 18, 2008 11:25 pm
Has thanked: 0 time
Been thanked: 164 times

Re: How to Merge records in 2 ps files alternately into 3rd?

Postby Frank Yaeger » Wed Jan 11, 2012 11:34 pm

BillyBoyo has it essentially right.

For a MERGE, DFSORT starts out by reading the first record from SORTIN01 and the first record from SORTIN02 in that order.

INREC would thus assign seqnum 1 to SORTIN01 record 1 and seqnum 2 to SORTIN02 record 1.

DFSORT selects the lowest MERGE key of the pair (the "winner") which would be seqnum 1 and writes that record to the output data set. DFSORT then goes back to the same file which the winner came from (SORTIN01) and reads the next record (SORTIN01 record 2). INREC would thus assign seqnum 3 to SORTIN01 record 2. Now we have records with seqnum 2 and seqnum 3, so seqnum 2 is the winner and DFSORT writes that record to the output data set. Then it goes to SORTIN02 (the winner) to get the next record, and so on. So it works because DFSORT handles pairs of records from the two files based on the winner for each pair and the winner for each pair is always SORTIN01 and then SORTIN02 since they have alternating sequence numbers.

EQUALS has no effect in this case because all of the seqnums are unique.

So the process is:

READ RECORD1 from SORTIN01 - assign seqnum 1.
READ RECORD4 from SORTIN02 - assign seqnum 2.
Select winner for pair (seqnum 1/seqnum 2 -> seqnum 1) and write RECORD1 (winner) to SORTOUT.
READ RECORD2 from SORTIN01 (previous winner) - assign seqnum 3.
Select winner for pair (seqnum 2/seqnum3 -> seqnum2) and write RECORD4 (winner) to SORTOUT.
READ RECORD5 from SORTIN02 (previous winner) - assign seqnum 4.
Select winner for pair (seqnum 3/seqnum4 -> seqnum3) and write RECORD2 (winner) to SORTOUT.
and so on.

Does that help?
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1079
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 15 times

Re: How to Merge records in 2 ps files alternately into 3rd?

Postby enrico-sorichetti » Thu Jan 12, 2012 12:28 am

Thank You Frank!
as usual Your explanations are very clear.
cheers
enrico
When I tell somebody to RTFM or STFW I usually have the page open in another tab/window of my browser,
so that I am sure that the information requested can be reached with a very small effort
enrico-sorichetti
Global moderator
 
Posts: 2994
Joined: Fri Apr 18, 2008 11:25 pm
Has thanked: 0 time
Been thanked: 164 times

Re: How to Merge records in 2 ps files alternately into 3rd?

Postby Nik22Dec » Thu Jan 12, 2012 10:27 am

Hi Billy, Dick, Enrico & Frank,

Thank you all for your quick replies. Now I understand!! Really Appreciate!!! :)
Thanks,
Nik
User avatar
Nik22Dec
 
Posts: 68
Joined: Mon Dec 26, 2011 6:38 pm
Has thanked: 2 times
Been thanked: 0 time

Previous

Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post