DFSORT improve run times (ideas)
Posted: Wed Dec 14, 2011 9:39 pmHi I have two BIG files that I'm joining with the next sentences:
It work very well but the problem is that the sort take about 30 minutes to end.
The files aren't sorted by the keys and each one has over 60 millions of records.
Any Idea to optimize the sort in order to improve the efficiency? it is possible?
Thanks.
SORT FIELDS=COPY
JOINKEYS FILE=F1,FIELDS=(371,08,A,92,10,A,37,10,A)
JOINKEYS FILE=F2,FIELDS=(29,8,A,37,10,A,47,10,A)
JOIN UNPAIRED,F1
REFORMAT FIELDS=(F1:01,389,F2:01,28,?),FILL=C'?'
OUTFIL FNAMES=SALIDA1,
INCLUDE=(418,1,CH,EQ,C'B',AND,31,1,CH,EQ,C'X'),
BUILD=(001,389,C'REL',390,28,47,3)
OUTFIL FNAMES=SALIDA2,
INCLUDE=((418,1,SS,EQ,C'1B',AND,31,1,CH,NE,C'X'),OR,
(418,1,CH,EQ,C'1',AND,31,1,CH,EQ,C'X')),
BUILD=(001,389,C'REL',371,08,92,10,37,10,47,3)
END
JOINKEYS FILE=F1,FIELDS=(371,08,A,92,10,A,37,10,A)
JOINKEYS FILE=F2,FIELDS=(29,8,A,37,10,A,47,10,A)
JOIN UNPAIRED,F1
REFORMAT FIELDS=(F1:01,389,F2:01,28,?),FILL=C'?'
OUTFIL FNAMES=SALIDA1,
INCLUDE=(418,1,CH,EQ,C'B',AND,31,1,CH,EQ,C'X'),
BUILD=(001,389,C'REL',390,28,47,3)
OUTFIL FNAMES=SALIDA2,
INCLUDE=((418,1,SS,EQ,C'1B',AND,31,1,CH,NE,C'X'),OR,
(418,1,CH,EQ,C'1',AND,31,1,CH,EQ,C'X')),
BUILD=(001,389,C'REL',371,08,92,10,37,10,47,3)
END
It work very well but the problem is that the sort take about 30 minutes to end.
The files aren't sorted by the keys and each one has over 60 millions of records.
Any Idea to optimize the sort in order to improve the efficiency? it is possible?
Thanks.