DFSORT improve run times (ideas)



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

DFSORT improve run times (ideas)

Postby javivi » Wed Dec 14, 2011 9:39 pm

Hi I have two BIG files that I'm joining with the next sentences:

  SORT FIELDS=COPY                                                     
  JOINKEYS FILE=F1,FIELDS=(371,08,A,92,10,A,37,10,A)                   
  JOINKEYS FILE=F2,FIELDS=(29,8,A,37,10,A,47,10,A)                     
  JOIN UNPAIRED,F1                                                     
  REFORMAT FIELDS=(F1:01,389,F2:01,28,?),FILL=C'?'                     
  OUTFIL FNAMES=SALIDA1,                                               
       INCLUDE=(418,1,CH,EQ,C'B',AND,31,1,CH,EQ,C'X'),               
           BUILD=(001,389,C'REL',390,28,47,3)                         
  OUTFIL FNAMES=SALIDA2,                                               
       INCLUDE=((418,1,SS,EQ,C'1B',AND,31,1,CH,NE,C'X'),OR,           
                (418,1,CH,EQ,C'1',AND,31,1,CH,EQ,C'X')),             
           BUILD=(001,389,C'REL',371,08,92,10,37,10,47,3)             
  END                                                                   


It work very well but the problem is that the sort take about 30 minutes to end.

The files aren't sorted by the keys and each one has over 60 millions of records.

Any Idea to optimize the sort in order to improve the efficiency? it is possible?

Thanks.
javivi
 
Posts: 47
Joined: Fri Jan 21, 2011 2:53 pm
Has thanked: 0 time
Been thanked: 0 time

Re: DFSORT improve run times (ideas)

 

Re: DFSORT improve run times (ideas)

Postby Frank Yaeger » Wed Dec 14, 2011 11:19 pm

In order to comment, I would need to see your complete JES log. Send it to me as a plain text file offline (yaeger@us.ibm.com) if you want me to take a look.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1080
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 14 times

Re: DFSORT improve run times (ideas)

Postby javivi » Thu Dec 15, 2011 8:41 pm

Hi frank

I send you a E-mail with the log file

thanks.
javivi
 
Posts: 47
Joined: Fri Jan 21, 2011 2:53 pm
Has thanked: 0 time
Been thanked: 0 time

Re: DFSORT improve run times (ideas)

Postby javivi » Thu Dec 15, 2011 9:00 pm

Frank Yaeger wrote:In order to comment, I would need to see your complete JES log. Send it to me as a plain text file offline (yaeger@us.ibm.com) if you want me to take a look.


Hi Frank

I sent you a E-mail with the log of the job.

thanks.
javivi
 
Posts: 47
Joined: Fri Jan 21, 2011 2:53 pm
Has thanked: 0 time
Been thanked: 0 time

Re: DFSORT improve run times (ideas)

Postby Frank Yaeger » Thu Dec 15, 2011 10:57 pm

Well, actually you sent me an e-mail with some of the DFSORT messages. I need you to send me the COMPLETE JES log including all of the JES messages and DFSORT messages.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1080
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 14 times


Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post