Page 1 of 1

Split a file into x outfiles, max 20k in each

PostPosted: Fri Oct 05, 2012 2:33 pm
by LasseH
Read the DFSORT tricks but...

Problem:
Inputfile up to 200k rows (fb10) can also be 100 rows

Want that file to be split into x outfiles with maximum 20k rows in each.

(The "tricks" shows how to split into x files (x=hardcoded amount))
Can You help me with this?

//Lasse

Re: Split a file into x outfiles, max 20k in each

PostPosted: Fri Oct 05, 2012 9:03 pm
by skolusu
Lasse,

20k each file with an lrecl of 10 would only have (20 X 1024 = 20480 /10) = 2048 Records. You would be creating 100 datasets to accommodate all the rows. You can define all the 100 datasets and use SPLIT1R=2048 to split the records. If you are concerned about the number of empty datasets , you can run an IDCAMS step to check for an empty file and delete all the empty files in a single step.

Re: Split a file into x outfiles, max 20k in each

PostPosted: Fri Oct 05, 2012 9:59 pm
by Akatsukami
Is not the TS looking to create output data sets with a maximum of 20K records (which he miscalls "rows") rather than 20K bytes?

Re: Split a file into x outfiles, max 20k in each

PostPosted: Mon Oct 08, 2012 11:27 am
by LasseH
Sorry for the miscall, yes, I ment 20k records.
(200.000 records will end up in 10 outputfiles, with 20.000 records each, and 21.000 ends up in 2 files)

Re: Split a file into x outfiles, max 20k in each

PostPosted: Mon Oct 08, 2012 10:04 pm
by skolusu
LasseH wrote:Sorry for the miscall, yes, I ment 20k records.
(200.000 records will end up in 10 outputfiles, with 20.000 records each, and 21.000 ends up in 2 files)


It is still not clear as to how you want to split. Do you dynamically allocate the output files each with 20k records?

ex:
100,000 records  = SPLIT files =5 (5 x 20,000 )
200,000 records  = SPLIT files =5 (10 x 20,000 )
250,000 records  = SPLIT files =5 (12 x 20,000  + 1 X 10,000)
...


If that is indeed the case as I mentioned earlier define the max outfil files (100) and then in the next step use IDCAMS to delete all the empty datasets which is the simplest task.

Another solution is to read the file, get the count of records and then dynamically generate the ICETOOL JCL and submit it via INTRDR.

Re: Split a file into x outfiles, max 20k in each

PostPosted: Tue Oct 09, 2012 11:25 am
by LasseH
Thanks, I go with the "create a lot, delete after"
//Lasse