SORT 2 files with different record length



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

SORT 2 files with different record length

Postby shankar_dh » Mon Oct 20, 2014 8:09 pm

Hi,

I have 2 files with different record length (both are FB)
File 1 -- LRECL 100
File 2 -- LRECL 80

Can we sort (using IBM SORT utility) these 2 files and write out put to File 3 of record length 80.

I know the solution of 2 steps where
Step 1 --> Convert File 1 to record length 80
Step 2 --> SORT it in regular way.

But, Is there any way to achieve it in only one step avoid multiple reads?
Because I have millions of records in File 1 and Step1 will be only to reduce records from 100 to 80 and again need to read the million of records in step 2 to sort.

I want to use the below control card but its not holds good if the 2 files are of different length.

SUM FIELDS=(NONE)
SORT FIELDS=(1,10,CH,A)
OUTREC FIELDS=(1:1,80)
shankar_dh
 
Posts: 41
Joined: Fri Mar 22, 2013 1:00 pm
Has thanked: 1 time
Been thanked: 0 time

Re: SORT 2 files with different record length

 

Re: SORT 2 files with different record length

Postby BillyBoyo » Mon Oct 20, 2014 8:18 pm

Are File 1 and File 2 in sequence already?

You mention "millions". Is that on both files, or what?
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 264 times

Re: SORT 2 files with different record length

Postby shankar_dh » Mon Oct 20, 2014 8:27 pm

Thanks Billiboyo for the quick response.

Both are not in sorted order.
Only File 1 has millions of records.

I need to merge both files in to File 3 in sorted order.
shankar_dh
 
Posts: 41
Joined: Fri Mar 22, 2013 1:00 pm
Has thanked: 1 time
Been thanked: 0 time

Re: SORT 2 files with different record length

Postby BillyBoyo » Mon Oct 20, 2014 9:32 pm

Since you can't, generally, concatenate two FB of different LRECL, I think you have three approaches.

Firstly, is the two-step that you have. Cut the File 1 down to 80, then concatenate and SORT in the second step.

Secondly, there is ICETOOL, which can give you one step, but you will still have two passes of File 1 (which is what you want to avoid).

Thirdly, JOINKEYS.

The JOINKEYS would be straightforward if you have no duplicate keys.

So, the next question is whether you have duplicate keys. Internally on File 1? Internally on File 2? Between File 1 and File 2? Some combination of these.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 264 times

Re: SORT 2 files with different record length

Postby shankar_dh » Tue Oct 21, 2014 4:39 pm

Yes, I have duplicates.
I think, I would need to go with either option 1 or 2.
Thanks for your time and suggestion.
shankar_dh
 
Posts: 41
Joined: Fri Mar 22, 2013 1:00 pm
Has thanked: 1 time
Been thanked: 0 time

Re: SORT 2 files with different record length

Postby BillyBoyo » Tue Oct 21, 2014 5:25 pm

You didn't describe the types of duplicates as per my question.

The JOINKEYS may still be the most efficient. If the job is a once-off, use number 1. If a regular job, compare timings to the JOINKEYS, It does mean coding it out, but we can assist if necessary.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 264 times


Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post