Merge multiple fixed length files of different record length



JES, JES2, JCL utilities, IDCAMS, Compile & Run JCLs, PROCs etc...

Merge multiple fixed length files of different record length

Postby corvette1982 » Mon Mar 01, 2021 11:46 pm

I would like to merge multiple files of different record length into a variable length file. The input files are either 88, 104, or 163 record (fixed) length. I am trying to merge them into a single variable length file of 1004.

I tried using IEBGENER where I concatenated the files as a part of the SYSUT1 statement. (there are 30+ files to merge, so I was trying to keep this simple with a single JCL step and concatenating the files to merge)..

My SYSUT2 statement defines a VB file of lrecl 1004.

When I check the JCL, I get the following errors:

***ERROR - DSS4880E - LRECL " 104" CANNOT BE CONCAT. WITH LRECL " 88"
***ERROR - DSS4880E - LRECL " 163" CANNOT BE CONCAT. WITH LRECL " 88"

I suppose I could IEBGENER each of the 30+ files into a VB file of length 1004, then merge all those at the end. This would require many more steps though.... Thoughts? Many thanks!
corvette1982
 
Posts: 3
Joined: Mon Mar 01, 2021 11:37 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Merge multiple fixed length files of different record le

Postby willy jensen » Tue Mar 02, 2021 12:11 am

Are those datasets (please don't call them files, you will upset some folks in this forum) large?
If not, I would write a small REXX pgm to do it. EXECIO is pretty good at record format conversion, you can read all input datasets to the stack and then write them in one go.
If they are big, I would look at SORT with the FB-to-VB option.
By the way, when you say 'merge' do you really mean that, or will a concatenation be ok?
willy jensen
 
Posts: 357
Joined: Thu Mar 10, 2016 5:03 pm
Has thanked: 0 time
Been thanked: 49 times

Re: Merge multiple fixed length files of different record le

Postby corvette1982 » Tue Mar 02, 2021 1:13 am

Hi Willy

Thank you for the clarifications. Sorry to all about using the wrong terminology... been 20 years since I have been on the mainframe :)

I did mean datasets. There are 37 datasets which fall in three different record lengths. I went ahead and used a 4 step JCL process, each step using IEBGENER. The first three concatenated the datasets of similar length in the SYSUT1 into a VB temp dataset for SYSUT2. The last step then did the same for all three temp datasets into the final dataset required. Worked like a charm.
corvette1982
 
Posts: 3
Joined: Mon Mar 01, 2021 11:37 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Merge multiple fixed length files of different record le

Postby sergeyken » Tue Mar 02, 2021 1:21 am

Do you really need TO MERGE your datasets (by merge key field), or you only need TO CONCATENATE datasets with different LRECL/RECFM, e.g. place records of the second DSN all after records of the first one?
Javas come and go, but JCL stays forever.
User avatar
sergeyken
 
Posts: 216
Joined: Wed Jul 24, 2019 10:12 pm
Has thanked: 5 times
Been thanked: 19 times

Re: Merge multiple fixed length files of different record le

Postby corvette1982 » Tue Mar 02, 2021 1:28 am

Concatenate. Sorry again for the terminology.
corvette1982
 
Posts: 3
Joined: Mon Mar 01, 2021 11:37 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Merge multiple fixed length files of different record le

Postby sergeyken » Tue Mar 02, 2021 1:34 am

In order to do this CONCATENATION(!) in one step, you can use MULTIIN option of SYNCSORT utility (PARM=‘MULTIIN’j:

1) use several //SORTINnn DD concatenations, each of them combining those DSNs with equal LRECL

2) use SORT statement
INREC IFTHEN=(WHEN=(&MULTIINDD,EQ,C’nn’),  input from lrecl #1?
              BUILD=(1,lrecl1,lllX)),      Extend record to the full desired size
         . . . . . .


3) if needed, convert the result to RECFM=VB:
OUTFIL FTOV,... - with extra conversion, truncation, whatever...
Javas come and go, but JCL stays forever.
User avatar
sergeyken
 
Posts: 216
Joined: Wed Jul 24, 2019 10:12 pm
Has thanked: 5 times
Been thanked: 19 times

Re: Merge multiple fixed length files of different record le

Postby willy jensen » Tue Mar 02, 2021 1:46 am

Away for 20 years? Welcome back.
So assuming reasonably small datasets, then consider this REXX (from the top of my head, I haven't tested it).
/*  rexx */                                                    
 ids.1='input.data.set.name.1'                                  
 ids.2='input.data.set.name.2'                                  
 ids.0=2     /* number of input datasets */                                                  
 ods  ='output.data.set.name'                                  
 dd='A'space(translate(time(),' ',':'),0)                      
 "newstack"                                                    
 do n=1 to ids.0   /* load all */                              
   cc=bpxwdyn('alloc da('ids.n') shr dd('dd') reuse')          
   if cc<>0 then zz=Quit('alloc' ods 'rc' cc)                  
   "execio * diskr" dd "(finis)"                                
   if rc<>0 then zz=Quit('read' ids.n 'rc' rc)                  
 end                                                            
 cc=bpxwdyn('alloc da('ods') shr dd('dd') reuse')   /* write */
 if cc<>0 then zz=Quit('alloc' ods 'rc' cc)                    
 "execio" queued() "diskw" dd "(finis)"                        
 if rc<>0 then zz=Quit('write' ods 'rc' rc)                    
 zz=Quit('All done')                                            
Quit:                                                          
 "delstack"                                                    
 say arg(1)                                                    
 cc=bpxwdyn('free dd('dd')')                                    
 exit 0  
willy jensen
 
Posts: 357
Joined: Thu Mar 10, 2016 5:03 pm
Has thanked: 0 time
Been thanked: 49 times

Re: Merge multiple fixed length files of different record le

Postby sergeyken » Tue Mar 02, 2021 1:48 am

Sorry, for MULTIIN option the used DD names must be:
//SORTMI01 DD
//SORTMI02 DD
. . . .
//SORTMInn DD
Javas come and go, but JCL stays forever.
User avatar
sergeyken
 
Posts: 216
Joined: Wed Jul 24, 2019 10:12 pm
Has thanked: 5 times
Been thanked: 19 times

Re: Merge multiple fixed length files of different record le

Postby willy jensen » Tue Mar 02, 2021 3:02 am

Slight modification to handle bigger datasets.....
/*  rexx */                                                        
 ids.1='input.data.set.name.1'                                      
 ids.2='input.data.set.name.2'                                      
 ids.0=2                                                            
 ods  ='output.data.set.name'                                      
 ddi  ='I'space(translate(time(),' ',':'),0)                        
 ddo  ='O'space(translate(time(),' ',':'),0)                        
 cc=bpxwdyn('alloc da('ods') shr dd('ddo') reuse')   /* output */  
 if cc<>0 then zz=Quit('alloc' ods 'rc' cc)                        
 "newstack"                                                        
 do n=1 to ids.0   /* load all */                                  
   cc=bpxwdyn('alloc da('ids.n') shr dd('ddi') reuse')              
   if cc<>0 then zz=Quit('alloc' ods 'rc' cc)                      
   "execio * diskr" dd "(finis)"                                    
   if rc<>0 then zz=Quit('read' ids.n 'rc' rc)                      
   "execio" queued() "diskw" ddo                                    
   if rc<>0 then zz=Quit('write' ods 'rc' rc)                      
 end                                                                
 zz=Quit('All done')                                                
XMsg:                                                              
 "execio 0 diskw" ddo "(finis)"       /* close */                  
 "delstack"                                                        
 say arg(1)                                                        
 cc=bpxwdyn('free dd('ddi')')                                      
 cc=bpxwdyn('free dd('ddo')')                                      
 exit 0  
willy jensen
 
Posts: 357
Joined: Thu Mar 10, 2016 5:03 pm
Has thanked: 0 time
Been thanked: 49 times


Return to JCL

 


  • Related topics
    Replies
    Views
    Last post