Page 1 of 1
Merge multiple fixed length files of different record length
Posted:
Mon Mar 01, 2021 11:46 pmby corvette1982
I would like to merge multiple files of different record length into a variable length file. The input files are either 88, 104, or 163 record (fixed) length. I am trying to merge them into a single variable length file of 1004.
I tried using IEBGENER where I concatenated the files as a part of the SYSUT1 statement. (there are 30+ files to merge, so I was trying to keep this simple with a single JCL step and concatenating the files to merge)..
My SYSUT2 statement defines a VB file of lrecl 1004.
When I check the JCL, I get the following errors:
***ERROR - DSS4880E - LRECL " 104" CANNOT BE CONCAT. WITH LRECL " 88"
***ERROR - DSS4880E - LRECL " 163" CANNOT BE CONCAT. WITH LRECL " 88"
I suppose I could IEBGENER each of the 30+ files into a VB file of length 1004, then merge all those at the end. This would require many more steps though.... Thoughts? Many thanks!
Re: Merge multiple fixed length files of different record le
Posted:
Tue Mar 02, 2021 12:11 amby willy jensen
Are those datasets (please don't call them files, you will upset some folks in this forum) large?
If not, I would write a small REXX pgm to do it. EXECIO is pretty good at record format conversion, you can read all input datasets to the stack and then write them in one go.
If they are big, I would look at SORT with the FB-to-VB option.
By the way, when you say 'merge' do you really mean that, or will a concatenation be ok?
Re: Merge multiple fixed length files of different record le
Posted:
Tue Mar 02, 2021 1:13 amby corvette1982
Hi Willy
Thank you for the clarifications. Sorry to all about using the wrong terminology... been 20 years since I have been on the mainframe
I did mean datasets. There are 37 datasets which fall in three different record lengths. I went ahead and used a 4 step JCL process, each step using IEBGENER. The first three concatenated the datasets of similar length in the SYSUT1 into a VB temp dataset for SYSUT2. The last step then did the same for all three temp datasets into the final dataset required. Worked like a charm.
Re: Merge multiple fixed length files of different record le
Posted:
Tue Mar 02, 2021 1:21 amby sergeyken
Do you really need TO MERGE your datasets (by merge key field), or you only need TO CONCATENATE datasets with different LRECL/RECFM, e.g. place records of the second DSN all after records of the first one?
Re: Merge multiple fixed length files of different record le
Posted:
Tue Mar 02, 2021 1:28 amby corvette1982
Concatenate. Sorry again for the terminology.
Re: Merge multiple fixed length files of different record le
Posted:
Tue Mar 02, 2021 1:34 amby sergeyken
In order to do this CONCATENATION(!) in one step, you can use MULTIIN option of SYNCSORT utility (PARM=‘MULTIIN’j:
1) use several //SORTINnn DD concatenations, each of them combining those DSNs with equal LRECL
2) use SORT statement
INREC IFTHEN=(WHEN=(&MULTIINDD,EQ,C’nn’), input from lrecl #1?
BUILD=(1,lrecl1,lllX)), Extend record to the full desired size
. . . . . .
3) if needed, convert the result to RECFM=VB:
OUTFIL FTOV,... - with extra conversion, truncation, whatever...
Re: Merge multiple fixed length files of different record le
Posted:
Tue Mar 02, 2021 1:46 amby willy jensen
Away for 20 years? Welcome back.
So assuming reasonably small datasets, then consider this REXX (from the top of my head, I haven't tested it).
/* rexx */
ids.1='input.data.set.name.1'
ids.2='input.data.set.name.2'
ids.0=2 /* number of input datasets */
ods ='output.data.set.name'
dd='A'space(translate(time(),' ',':'),0)
"newstack"
do n=1 to ids.0 /* load all */
cc=bpxwdyn('alloc da('ids.n') shr dd('dd') reuse')
if cc<>0 then zz=Quit('alloc' ods 'rc' cc)
"execio * diskr" dd "(finis)"
if rc<>0 then zz=Quit('read' ids.n 'rc' rc)
end
cc=bpxwdyn('alloc da('ods') shr dd('dd') reuse') /* write */
if cc<>0 then zz=Quit('alloc' ods 'rc' cc)
"execio" queued() "diskw" dd "(finis)"
if rc<>0 then zz=Quit('write' ods 'rc' rc)
zz=Quit('All done')
Quit:
"delstack"
say arg(1)
cc=bpxwdyn('free dd('dd')')
exit 0
Re: Merge multiple fixed length files of different record le
Posted:
Tue Mar 02, 2021 1:48 amby sergeyken
Sorry, for MULTIIN option the used DD names must be:
//SORTMI01 DD
//SORTMI02 DD
. . . .
//SORTMInn DD
Re: Merge multiple fixed length files of different record le
Posted:
Tue Mar 02, 2021 3:02 amby willy jensen
Slight modification to handle bigger datasets.....
/* rexx */
ids.1='input.data.set.name.1'
ids.2='input.data.set.name.2'
ids.0=2
ods ='output.data.set.name'
ddi ='I'space(translate(time(),' ',':'),0)
ddo ='O'space(translate(time(),' ',':'),0)
cc=bpxwdyn('alloc da('ods') shr dd('ddo') reuse') /* output */
if cc<>0 then zz=Quit('alloc' ods 'rc' cc)
"newstack"
do n=1 to ids.0 /* load all */
cc=bpxwdyn('alloc da('ids.n') shr dd('ddi') reuse')
if cc<>0 then zz=Quit('alloc' ods 'rc' cc)
"execio * diskr" dd "(finis)"
if rc<>0 then zz=Quit('read' ids.n 'rc' rc)
"execio" queued() "diskw" ddo
if rc<>0 then zz=Quit('write' ods 'rc' rc)
end
zz=Quit('All done')
XMsg:
"execio 0 diskw" ddo "(finis)" /* close */
"delstack"
say arg(1)
cc=bpxwdyn('free dd('ddi')')
cc=bpxwdyn('free dd('ddo')')
exit 0