Page 1 of 1

Splitting a very large file

PostPosted: Thu Dec 24, 2020 10:35 pm
by socker_dad
I have need to split a very large file of roughly 17.8 million records into many "sub-files" of about 50,000 records each. It's for tax season and our mainframe isn't allowed to run unfettered.

I know I can use steps of successive "$$DD01 SPACE IN=50000 $$DD01 COPY OUT=50000", but the calculations indicate that this will have to be repeated about 355 times. Ouch.

Is there another way to get File Aid to split more efficiently? Would the be better in DFSORT? Or should I just pop out a quick and dirty program?

Re: Splitting a very large file

PostPosted: Fri Dec 25, 2020 2:42 pm
by NicC
Look up the DFSort Tricks publication which gives several ways to split datasets.

Re: Splitting a very large file

PostPosted: Thu Jan 28, 2021 12:09 am
by chaat
I've written an E15 sort exit to count the number of records, it then passes this count to an E35 sort exit which splits the sort out using Dynamic allocation to multiple output files. This number of files is controlled by an input split control file.

If you are interested in the source code, send me a private message and include your email address. I will send you the two COBOL Sort Exit programs.

Chuck Haatvedt