IBM Mainframe Forum

Mainframe Technical Support Forums

Skip to content


Split file, SORT then append records.



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER
Post a reply

Split file, SORT then append records.

Postby Esmayeelhusen » Fri Aug 04, 2023 6:37 pm

Hi All,
I am working on a requirement to sort the files in to 2 different sort sequences and append the records.
First sort should be done as, SORT FIELDS=(1,1,BI,A,27,6,BI,A,11,1,BI,A)
Then i have to separate the '3' in first byte and sort like as, SORT FIELDS=(80,12,BI,A,11,1,BI,D)
Then I have to append these records at the end of the first file before trailer record.
Files is PS,FB, LRECL=227

Please suggest if its possible in a single step. Your help will be much appreciated.


Select all
VIEW       SHOG.W3SODS.CHS.ODSCHSOE                                                                            Columns 00001 00124
Command ===>                                                                                                      Scroll ===> CSR
=COLS> ----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0----+----1----+----2----
000001 11230804
000002 255356090515102700007500000000063290000000000000230804000000B0000000000000000000CH21 29Apr2022  SGB CHS SCN014 OFI
000003 210021180915102700007400000000002530000000000000230804000000B0000000000000000000Ofi dd
000004 210021180913207700008400000000002530000000000000230804000000B0000000000000000000INWARD PAYMENT Muhammed CBA ACC
000005 210015030513207700007600000000005920000000000000230804000000B0000000000000000000MD06 20 Jun 2022 INWARD PAYMENT Muhammed
000006 3130067297ADD00000000004012308041440143361657001093240104411                   230804114456                             0000
000007 3130067297DEL00000000004012308041440143361657001093240104411                   230804114605                             0000
000008 3130067297ADD00000000008022308041440143361657001093240104411                   230804114605                             0000
000009 3130067297DEL00000000008332308041505143361657001093240150511                   230804115255                             0000
000010 3130067297ADD00000000008332308041505143361657001093240150511                   230804115117                             0000
000011 210015030513207700007600000000022930000000000000230804000000B0000000000000000000WBC API
000012 210015030563202700008600000000022930000000000000230804000000B0000000000000000000  PARTNERSHIP SECURITIES PTY LIMITED
000013 210010562715102700007800000007331000000000000000230804000000B0000000000000000000 FTH ODS CHS SCN001 Tania England
000014 210034530515102700007800000000079310000000000000230804000000B0000000000000000000Nazia Shaik OFI
000015 210034530513507700008800000000079310000000000000230804000000B0000000000000000000OFI 2.0
000016 3130067297ADD00000000005102308041251043361657001093240015211                   230804112559                             0000
000017 3130067297DEL00000000004012308041440143361657001093240104411                   230804114758                             0000
000018 3130067297ADD00000000009162308041440143361657001093240104411                   230804114945                             0000
000019 3130067297ADD00000000002112308041270343361657001093240307211                   230804112841                             0000
000020 3130067297ADD00000000003112308041302343361657001093240320311                   230804113045                             0000
000021 3130067297ADD00000000011352308041193543361657001093240539111                   230804112328                             0000
000022 3130067297ADD00000000003002308041234543361657001093240543211                   230804112456                             0000
000023 255356090513207700008500000000063290000000000000230804000000B0000000000000000000SGB CHS SCN014 OFI
000024 210021180915102700007400000000019160000000000000230804000000B0000000000000000000CH21 29Apr2022  SGB CHS SCN014 OFI
000025 3130067297ADD00000000004162308041341643361657001093240614311                   230804113538                             0000
000026 3130067297DEL00000000004162308041341643361657001093240614311                   230804113757                             0000
000027 3130067297ADD00000000005472308041341643361657001093240614311                   230804113757                             0000
000028 3130067297DEL00000000005472308041341643361657001093240614311                   230804113921                             0000
000029 3130067297ADD00000000006102308041341643361657001093240614311                   230804113921                             0000
000030 3130067297DEL00000000006102308041341643361657001093240614311                   230804114126                             0000
000031 3130067297ADD00000000009502308041341643361657001093240614311                   230804114126                             0000
000032 3130067297ADD00000000008562308041285643361657001093240658211                   230804112934                             0000
000033 9000000192
****** ****************************************************** Bottom of Data ******************************************************
Esmayeelhusen
 
Posts: 13
Joined: Thu May 04, 2023 1:57 pm
Has thanked: 3 times
Been thanked: 0 time
Top

Re: Split file, SORT then append records.

Postby sergeyken » Fri Aug 04, 2023 6:57 pm

More than one SORT is not possible in a single SORT step, but you can do it sequentially via two SORT statements within a single ICETOOL/SYNCTOOL step.
Select all
//TWINSORT EXEC PGM=ICETOOL
// . . . . .
//TOOLIN   DD  *
 SORT FROM(SORTIN)    TO(WORKFILE)  USING(SRT1)
 SORT FROM(WORKFILE)  TO(SORTOUT)   USING(SRT2)
//*
//WORKFILE DD  SPACE=(CYL,(10,10))
//*
//SRT1CNTL DD  *
 SORT FIELDS=...
 OUTFIL FNAMES=SORTOUT,INCLUDE=(first part)
 OUTFIL FNAMES=WORKFILE,SAVE    all unused yet records
//*
//SRT2CNTL DD *
 SORT FIELDS=...
 OUTFIL FNAMES=SORTOUT    - add resorted part to SORTOUT with DISP=MOD (!!)
//*
Javas and Pythons come and go, but JCL and SORT stay forever.
User avatar
sergeyken
 
Posts: 409
Joined: Wed Jul 24, 2019 10:12 pm
Has thanked: 6 times
Been thanked: 40 times
Top

Re: Split file, SORT then append records.

Postby sergeyken » Fri Aug 04, 2023 7:18 pm

1. If //SORTOUT DD SYSOUT=* then DISP=MOD is used by default.

2. If //SORTOUT DD DISP=(NEW,...) then it is equivalent to DISP=MOD

3. Using //SORTOUT DD DISP=MOD explicitly is doubtful: for existing dataset the data from the first SORT will be appended to its old, unpredictable content.

4. Using //SORTOUT DD DISP=OLD/SHR is wrong, because the second SORT will override the result of the first one.
Javas and Pythons come and go, but JCL and SORT stay forever.
User avatar
sergeyken
 
Posts: 409
Joined: Wed Jul 24, 2019 10:12 pm
Has thanked: 6 times
Been thanked: 40 times
Top

Re: Split file, SORT then append records.

Postby Esmayeelhusen » Sun Aug 06, 2023 7:23 pm

Hi Sergeyken,
Thanks for your reply.
As per your suggestions I have achieved the expected outcome. Appreciate your help.


Select all
//PS016     EXEC PGM=ICETOOL,COND=(4,LT)
//SORTIN    DD DSN=X.YYYYY.ZZZ.INFILE(0),DISP=SHR
//*            DCB=BUFNO=5
//SORTOUT   DD DSN=X.YYYYY.ZZZ.INFILE.SORTED(+1),
//             UNIT=DISK,
//             DCB=(&MDSCB..XXXX.MODEL.DSCB,
//             BLKSIZE=0,LRECL=227,RECFM=FB,BUFNO=5,DSORG=PS),
//             SPACE=(CYL,(02,2),RLSE),DISP=(NEW,CATLG,KEEP)
//*
//TOOLIN    DD  *
    SORT FROM(SORTIN)    TO(WRKFILE1)  USING(SRT1)
    SORT FROM(WRKFILE2)  TO(WRKFILE1)  USING(SRT2)
    SORT FROM(WRKFILE1)  TO(SORTOUT)   USING(SRT3)
//*
//WRKFILE1  DD  DISP=MOD,SPACE=(CYL,(10,10))
//WRKFILE2  DD  SPACE=(CYL,(10,10))
//TOOLMSG   DD SYSOUT=*
//DFSMSG    DD SYSOUT=*
//*
//SRT1CNTL  DD  *
 SORT FIELDS=(001,1,BI,A,
              027,6,BI,A,
              131,6,BI,A,
              011,1,BI,A),FILSZ=E1500 * SORT RECORDS AS PER CARD
    OUTFIL FNAMES=WRKFILE1,           * COPY SORTED EXCEPT TYPE 3 REC,
           INCLUDE=(1,1,CH,NE,C'3')   * INCLUDING H & T TO WRKFILE1
    OUTFIL FNAMES=WRKFILE2,SAVE       * COPY SORTED ONLY TYPE 3 RECORDS
//*                                   * TO WRKFILE2
//SRT2CNTL DD *
    SORT FIELDS=(163,12,BI,A,         * SORT WRKFILE2 AS PER CARD
                 011,01,BI,D)         * APPEND WRKFILE2 REC TO WRKFILE1
    OUTFIL FNAMES=WRKFILE1,SAVE
//*
//SRT3CNTL DD *                       * NOW TYPE 3 REC ARE AFTER TRILER
    SORT FIELDS=(1,1,BI,A)            * SORT AS PER REC TYPE TO SORTOUT
    OUTFIL FNAMES=SORTOUT
//*
Esmayeelhusen
 
Posts: 13
Joined: Thu May 04, 2023 1:57 pm
Has thanked: 3 times
Been thanked: 0 time
Top

Re: Split file, SORT then append records.

Postby sergeyken » Mon Aug 07, 2023 1:09 am

There is no need to run it in three stages. Two stages (and only one work file) is enough to get the same result. I suggested you the solution, but you prefer to ignore it.
Javas and Pythons come and go, but JCL and SORT stay forever.
User avatar
sergeyken
 
Posts: 409
Joined: Wed Jul 24, 2019 10:12 pm
Has thanked: 6 times
Been thanked: 40 times
Top

Re: Split file, SORT then append records.

Postby Esmayeelhusen » Mon Aug 07, 2023 7:29 am

Hi Sergeyken,
As per your solution record type 3 are appended at the end, which is after trailer record (Record type 9)
To bring the trailer at the end I have introduced third stage.
Please suggest, if this can be done without stage 3.
Thanks
Husen
Esmayeelhusen
 
Posts: 13
Joined: Thu May 04, 2023 1:57 pm
Has thanked: 3 times
Been thanked: 0 time
Top

Re: Split file, SORT then append records.

Postby sergeyken » Mon Aug 07, 2023 5:35 pm

Select all
//TWINSORT EXEC PGM=ICETOOL                                                    
//*                                                                            
//SSMSG    DD  SYSOUT=*                                                        
//TOOLMSG  DD  SYSOUT=*                                                        
//SORTIN   DD  *                                                                
1         D               ZZZZZZ                           YYYYYYYYYYYY        
2         D               ZZZZZZ                           YYYYYYYYYYYY        
3         D               ZZZZZZ                           YYYYYYYYYYYY        
1         C               KKKKKK                           LLLLLLLLLLLL        
2         C               KKKKKK                           LLLLLLLLLLLL        
3         C               KKKKKK                           LLLLLLLLLLLL        
1         A               BBBBBB                           CCCCCCCCCCCC        
2         A               BBBBBB                           CCCCCCCCCCCC        
3         A               BBBBBB                           CCCCCCCCCCCC        
1         B               BBBBBB                           MMMMMMMMMMMM        
2         B               BBBBBB                           MMMMMMMMMMMM        
3         B               BBBBBB                           MMMMMMMMMMMM        
//*-+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8
//TOOLIN   DD  *                                                                
 SORT FROM(SORTIN)    TO(WORKFILE)  USING(SRT1)                                
 SORT FROM(WORKFILE)  TO(SORTOUT)   USING(SRT2)                                
//*                                                                            
//WORKFILE DD  SPACE=(CYL,(10,10))                                              
//*                                                                            
//SORTOUT  DD  SYSOUT=*                                                        
//*                                                        
//SRT1CNTL DD  *      
* Makes no sense to sort character data as 'BI"!!!                                    
 SORT FIELDS=(1,1,CH,A,                                    
             27,6,CH,A,                                    
             11,1,CH,A)                                    
 OUTFIL FNAMES=SORTOUT,INCLUDE=(1,1,CH,NE,C'3')            
 OUTFIL FNAMES=WORKFILE,SAVE    all unused yet records    
//*                                                        
//SRT2CNTL DD *
* Here position 1 is always equal to '3' - NO NEED TO RESORT AGAIN!                                          
 SORT FIELDS=(60,12,CH,A,                                  
              11,1,CH,D)                                  
 OUTFIL FNAMES=SORTOUT    add resorted part to SORTOUT    
//*                                                        

Select all
********************************* TOP OF DATA ******************************
1         A               BBBBBB                           CCCCCCCCCCCC    
1         B               BBBBBB                           MMMMMMMMMMMM    
1         C               KKKKKK                           LLLLLLLLLLLL    
1         D               ZZZZZZ                           YYYYYYYYYYYY    
2         A               BBBBBB                           CCCCCCCCCCCC    
2         B               BBBBBB                           MMMMMMMMMMMM    
2         C               KKKKKK                           LLLLLLLLLLLL    
2         D               ZZZZZZ                           YYYYYYYYYYYY    
3         A               BBBBBB                           CCCCCCCCCCCC    
3         C               KKKKKK                           LLLLLLLLLLLL    
3         B               BBBBBB                           MMMMMMMMMMMM    
3         D               ZZZZZZ                           YYYYYYYYYYYY    
******************************** BOTTOM OF DATA ****************************
Javas and Pythons come and go, but JCL and SORT stay forever.
These users thanked the author sergeyken for the post:
Esmayeelhusen (Tue Aug 08, 2023 9:36 am)
User avatar
sergeyken
 
Posts: 409
Joined: Wed Jul 24, 2019 10:12 pm
Has thanked: 6 times
Been thanked: 40 times
Top

Re: Split file, SORT then append records.

Postby Esmayeelhusen » Tue Aug 08, 2023 6:51 am

Hi Sergeyken,
Thanks for your reply.

Your input data looks good, however i have record types '6', '8' and '9' as well in the files.
I have modified the test data and executed your job, the outcome is as below.
The record type '3' are at the bottom of the file.
I believe we need stage 3 to sort on the first byte to bring everything in sequence.


Select all
********************************* TOP OF DATA ***************************
1         A               BBBBBB                           CCCCCCCCCCCC
1         B               BBBBBB                           MMMMMMMMMMMM
1         C               KKKKKK                           LLLLLLLLLLLL
1         D               ZZZZZZ                           YYYYYYYYYYYY
2         A               BBBBBB                           CCCCCCCCCCCC
2         B               BBBBBB                           MMMMMMMMMMMM
2         C               KKKKKK                           LLLLLLLLLLLL
2         D               ZZZZZZ                           YYYYYYYYYYYY
8         B               BBBBBB                           MMMMMMMMMMMM
8         B               BBBBBB                           MMMMMMMMMMMM
9         B               BBBBBB                           MMMMMMMMMMMM
9         B               BBBBBB                           MMMMMMMMMMMM
9         B               BBBBBB                           MMMMMMMMMMMM
3         A               BBBBBB                           CCCCCCCCCCCC
3         C               KKKKKK                           LLLLLLLLLLLL
3         B               BBBBBB                           MMMMMMMMMMMM
3         D               ZZZZZZ                           YYYYYYYYYYYY
******************************** BOTTOM OF DATA *************************
Esmayeelhusen
 
Posts: 13
Joined: Thu May 04, 2023 1:57 pm
Has thanked: 3 times
Been thanked: 0 time
Top

Re: Split file, SORT then append records.

Postby sergeyken » Tue Aug 08, 2023 5:07 pm

First: please list ALL your requirements IN FULL
Next: we can continue the discussion.
Javas and Pythons come and go, but JCL and SORT stay forever.
User avatar
sergeyken
 
Posts: 409
Joined: Wed Jul 24, 2019 10:12 pm
Has thanked: 6 times
Been thanked: 40 times
Top

Re: Split file, SORT then append records.

Postby sergeyken » Tue Aug 08, 2023 10:47 pm

In any case: there is no need to re-sort again the part of data already sorted once. Especially for mega-million datasets

Select all
//TOOLIN   DD  *                                                
 SORT FROM(SORTIN)    TO(SORTOUT,PART3,PARTGT3)  USING(SRT0)    
 SORT FROM(PART3)     TO(SORTOUT)                USING(SRT3)    
 COPY FROM(PARTGT3)   TO(SORTOUT)                              
//*                                                            
//PART3    DD  SPACE=(CYL,(10,10))                              
//PARTGT3  DD  SPACE=(CYL,(10,10))                              
//*                                                            
//SORTOUT  DD  SYSOUT=*                                        
//*                                                            
//SRT0CNTL DD  *                                                
* Sort whole input data; split into 3 parts                    
 SORT FIELDS=(1,1,CH,A,                                        
             27,6,CH,A,                                        
             11,1,CH,A)                                        
 OUTFIL FNAMES=SORTOUT,INCLUDE=(1,1,CH,LT,C'3')                
 OUTFIL FNAMES=PART3,INCLUDE=(1,1,CH,EQ,C'3')                  
 OUTFIL FNAMES=PARTGT3,SAVE                                    
//*                                                            
//SRT3CNTL DD *                                                
* Re-sort only group '3' data; append to SORTOUT                
 SORT FIELDS=(60,12,CH,A,                                      
              11,1,CH,D)                                                 
//*                                                            
Javas and Pythons come and go, but JCL and SORT stay forever.
These users thanked the author sergeyken for the post:
Esmayeelhusen (Wed Aug 09, 2023 6:57 am)
User avatar
sergeyken
 
Posts: 409
Joined: Wed Jul 24, 2019 10:12 pm
Has thanked: 6 times
Been thanked: 40 times
Top
Next
Post a reply

Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post
Forum Rules|FAQ|Subscriptions|Bookmarks