Split a file based on size



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

Split a file based on size

Postby msnirmal1487 » Tue Dec 20, 2011 7:14 pm

Hi all,

We have a requiremnt to split a file based on size. (i.e) we would get a file (whose size may vary) and we would require to split this file into smaller parts of equal size (say 50 KB each). From the manuals, i could understand that using SPLIT command we create "n" (fixed number) files.

Say, in Week 1 we got a file of 300 KB size we need to split it to 6 files of 50 kb each. In Week 2 we got a file of 200 KB size we need to split it to 2 files of 50 kb each.

I was just wonering if there is a possiblity to do this using DFSORT.

Regards,
Nirmal
msnirmal1487
 
Posts: 10
Joined: Sat Jan 29, 2011 10:39 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Split a file based on size

Postby Robert Sample » Tue Dec 20, 2011 7:22 pm

My first question has to be, WHY talk KB? Mainframe disk data sets are defined in terms of blocks, tracks, or cylinders yet you are referencing KB.

My second questiion has to be, WHY do it al all? A 300KB file on a mainframe is ridiculously tiny -- what possible business reason could your site have for splitting such a file into smaller files?
Robert Sample
Global moderator
 
Posts: 3719
Joined: Sat Dec 19, 2009 8:32 pm
Location: Dubuque, Iowa, USA
Has thanked: 1 time
Been thanked: 279 times

Re: Split a file based on size

Postby msnirmal1487 » Tue Dec 20, 2011 7:30 pm

Hi Robert,

The sizes i have mentioned are examples.

we would actually require to split files of size around 10 MB. the 50 KB restriction is on the FTP server where we are going to PUT the file.

Regards,
Nirmal
msnirmal1487
 
Posts: 10
Joined: Sat Jan 29, 2011 10:39 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Split a file based on size

Postby msnirmal1487 » Tue Dec 20, 2011 7:32 pm

Answering your first question, if there is a way to split file into files of say 100 tracks each it would be of great help to us.

Regards,
Nirmal
msnirmal1487
 
Posts: 10
Joined: Sat Jan 29, 2011 10:39 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Split a file based on size

Postby BillyBoyo » Tue Dec 20, 2011 8:27 pm

I wouldn't have though a 50kb limit on an ftp server would make the server very useful. If you have a 10mb file, that's 100 files to create, send, receive and verify everything on the receiving end as being what you started out with. That's a lot of JCL, a lot of ftp steps, lots of places to go wrong in writing/maintaining everything, lots of places to loose data/not put it together correctly at the other end (which is presumably out of your control).

If there is a 50kb limit, presumably it is for a reason, like it doesn't want big files... or what?

DFSORT can SPLIT on record counts, in various ways, on ranges of records, but if you want to split on size of data, you'd have to know beforehand how to express that in records.

I suggest you write out (computer or piece of paper) everything you'd need to do (and they at the other end need to do) to do that properly (including verifying that all the correct data arrives, and no extraneous data is included) and then set out how to do that as one shot with a 10MB file and then go to your boss with that.

If someone, somewhere, doesn't change there mind about something then you just have to spend a lot of time ensuring it is done this way, but properly.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: Split a file based on size

Postby Akatsukami » Tue Dec 20, 2011 9:00 pm

To expand on BillyBoyo's post, I would suggest using Rexx to calculate how many files you want your data set split into, and then use ISPF services to fabricate DFSORT and FTP JCL from skeletons.
"You have sat too long for any good you have been doing lately ... Depart, I say; and let us have done with you. In the name of God, go!" -- what I say to a junior programmer at least once a day
User avatar
Akatsukami
Global moderator
 
Posts: 1058
Joined: Sat Oct 16, 2010 2:31 am
Location: Bloomington, IL
Has thanked: 6 times
Been thanked: 51 times

Re: Split a file based on size

Postby Frank Yaeger » Tue Dec 20, 2011 11:09 pm

For ideas on how to do this with DFSORT, see the "Split a file to n output files dynamically" and "Five ways to split a data set" Smart DFSORT Tricks at:

http://www.ibm.com/support/docview.wss? ... g3T7000094
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1079
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 15 times


Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post