Combining multiple records



Support for NetApp SyncSort for z/OS, Visual SyncSort, SYNCINIT, SYNCLIST and SYNCTOOL

Combining multiple records

Postby Don_W » Thu Feb 05, 2015 4:21 am

I'm trying to figure out a way with Syncsort to combine some data from multiple records. I've searched and I can find some solutions, but I can't find one that will seem to work with my situation.

The input file looks like this:

----+----1----+----2----+----3----+
NAME1111  #######  FILENAME.GOES.HE
                   RE_AND_CONTINUES
                   _DOWN_HERE_AND_K
                   EEPS_GOING.TXT
NAME2222  #######  NEXT_FILE.TXT
NAME2222  #######  ANOTHER_FILE_YET
                   _AGAIN.TXT
NAME3333  #######  THIRD_FILE_NAME_
                   TWO_LINES.TXT


And I need the output file to look like this.

NAME1111  FILENAME.GOES.HERE_AND_CONTINUES_DOWN_HERE_AND_KEEPS_GOING.TXT
NAME2222  NEXT_FILE.TXT
NAME2222  ANOTHER_FILE_YET_AGAIN.TXT
NAME3333  THIRD_FILE_NAME_TWO_LINES.TXT


The filename field in this file can continue on several lines and this where I have the issues. The first 8 character name column can repeat many times and is not unique and neither are the files names so there is no unique key. I was hoping to do what I need with a quick and dirty sort rather than having to write a program.

Thanks in advance for any replies.
Don_W
 
Posts: 1
Joined: Thu Feb 05, 2015 4:10 am
Has thanked: 0 time
Been thanked: 0 time

Re: Combining multiple records

Postby Thampy » Fri Feb 06, 2015 5:29 pm

I have tested the syncsort jcl with the assumption that the maximum number of combined records is 5. If you want more than 5, then code IFTHEN conditions accordingly.

//STEP010   EXEC PGM=SORT                                         
//SYSOUT    DD   SYSOUT=*                                         
//SORTOUT   DD   SYSOUT=*                                         
//SORTIN     DD *                                                 
NAME1111  #######  FILENAME.GOES.HE                               
                   RE_AND_CONTINUES                               
                   _DOWN_HERE_AND_K                               
                   EEPS_GOING.TXT                                 
NAME2222  #######  NEXT_FILE.TXT                                   
NAME2222  #######  ANOTHER_FILE_YET                               
                   _AGAIN.TXT                                     
NAME3333  #######  THIRD_FILE_NAME_                               
                   TWO_LINES.TXT                                   
//SYSIN     DD   *                                                 
  INREC IFTHEN=(WHEN=GROUP,BEGIN=(1,8,CH,NE,C' '),PUSH=(1:1,8,     
               390:SEQ=3)),                                             
        IFTHEN=(WHEN=INIT,OVERLAY=(400:20,16,11:16Z,27:16Z,43:16Z,     
                 59:16Z,75:16Z)),                                       
        IFTHEN=(WHEN=(390,3,ZD,EQ,1),OVERLAY=(11:400,16)),             
        IFTHEN=(WHEN=(390,3,ZD,EQ,2),OVERLAY=(27:400,16)),             
        IFTHEN=(WHEN=(390,3,ZD,EQ,3),OVERLAY=(43:400,16)),             
        IFTHEN=(WHEN=(390,3,ZD,EQ,4),OVERLAY=(59:400,16)),             
        IFTHEN=(WHEN=(390,3,ZD,EQ,5),OVERLAY=(75:400,16))               
  SORT FIELDS=(1,8,CH,A)                                               
  SUM FIELDS=(11,8,19,8,27,8,35,8,43,8,51,8,59,8,67,8,75,8,             
              83,8),FORMAT=BI                                           
  OUTREC FINDREP=(IN=X'00',OUT=X'40')                                   
  OUTFIL BUILD=(1,100)                                                 
Thampy
 
Posts: 36
Joined: Sat Sep 26, 2009 2:27 pm
Has thanked: 0 time
Been thanked: 3 times

Re: Combining multiple records

Postby BillyBoyo » Mon Feb 09, 2015 6:17 am

For up to five elements to the file-name:

Firstly, extends the record to 119 bytes, filling the new bytes with blanks.

A group with an ID (up to 99 groups will work, make your length one more than can possibly be needed) and a sequence number (here code deals with one to five in a group).

For a group of five records, starting with the first record, copy the reference to part of the record we don't need, and copy the text to a new position to be appended to.

For a group of four records, starting with the second record, copy the reference to part of the record we don't need, and copy the text to a new position to be appended to.

Continue like that through to and including the fourth record.

For the fifth record, append the data.

For each group of records defined by sequence, the first record of the next group will be clobbered, so clear all but the first part of the new file-name to space.

Use OUTFIL reporting features to output the last record of the group.

Adjust for the maximum number of records you need, and for the maximum number of different groups (keys can be the same, contiguous or not, no problem).

//SYSIN    DD *
 OPTION COPY
                                           
 INREC IFTHEN=(WHEN=INIT,
               OVERLAY=(119:X)),
                                           
       IFTHEN=(WHEN=GROUP,
               BEGIN=(1,1,CH,NE,C' '),
               PUSH=(36:ID=2,SEQ=1)),
                                           
       IFTHEN=(WHEN=GROUP,
               BEGIN=(38,1,CH,EQ,C'1'),
               PUSH=(11:1,8,39:20,16),
               RECORDS=5),
                                           
       IFTHEN=(WHEN=GROUP,
               BEGIN=(38,1,CH,EQ,C'2'),
               PUSH=(55:20,16),
               RECORDS=4),
       IFTHEN=(WHEN=GROUP,
               BEGIN=(38,1,CH,EQ,C'3'),
               PUSH=(71:20,16),
               RECORDS=3),
       IFTHEN=(WHEN=GROUP,
               BEGIN=(38,1,CH,EQ,C'4'),
               PUSH=(87:20,16),
               RECORDS=2),
                                           
       IFTHEN=(WHEN=(38,1,CH,EQ,C'1'),
               OVERLAY=(55:64X)),
                                           
       IFTHEN=(WHEN=(38,1,CH,EQ,C'5'),
               OVERLAY=(103:20,16))
                                           
  OUTFIL REMOVECC,
         NODETAIL,
         SECTIONS=(36,2,
                   TRAILER3=(11,8,X,39,80))
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: Combining multiple records

Postby BillyBoyo » Mon Feb 09, 2015 6:23 am

Thampy,

If the data does not require SORTing, I always suggest not SORTing it. You could include an IFOUTLEN in the INREC to cut down the size of the record, rather than SORTing the 400+ bytes anyway, and you'd not need the OUTFIL.

If you want to remove the SORT, you should be able to use the same type of solution with TOT/TOTAL on TRAILER3 within SECTIONS in OUTFIL using reporting features. Except for...

What's the FINDREP for?
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: Combining multiple records

Postby Thampy » Mon Feb 09, 2015 8:05 pm

Thanks for the better solution. In my solution, I am filling the positions with binary zeros and then move the records in the group into these binary fields. If the number of records in the group is less than 5, then those associated fields will have binary zeros value. So I am using a FINDREP to replace binary zeros with spaces.
Thampy
 
Posts: 36
Joined: Sat Sep 26, 2009 2:27 pm
Has thanked: 0 time
Been thanked: 3 times

Re: Combining multiple records

Postby BillyBoyo » Mon Feb 09, 2015 8:36 pm

OK. I'd limit the FINDREP to only the fields in question. I can't think that the binary zeros can be changed after the TOT/TOTAL, else yours with the OUTFIL would be more efficient than the multiple GROUPs in mine.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times


Return to Syncsort/Synctool

 


  • Related topics
    Replies
    Views
    Last post