Splitting a very large file

[socker_dad]

I have need to split a very large file of roughly 17.8 million records into many "sub-files" of about 50,000 records each. It's for tax season and our mainframe isn't allowed to run unfettered.

I know I can use steps of successive "$$DD01 SPACE IN=50000 $$DD01 COPY OUT=50000", but the calculations indicate that this will have to be repeated about 355 times. Ouch.

Is there another way to get File Aid to split more efficiently? Would the be better in DFSORT? Or should I just pop out a quick and dirty program?

[NicC]

Look up the DFSort Tricks publication which gives several ways to split datasets.

[chaat]

I've written an E15 sort exit to count the number of records, it then passes this count to an E35 sort exit which splits the sort out using Dynamic allocation to multiple output files. This number of files is controlled by an input split control file.

If you are interested in the source code, send me a private message and include your email address. I will send you the two COBOL Sort Exit programs.

Chuck Haatvedt