Page 1 of 1

Need Help on Performance Improvement of SyncSort

PostPosted: Fri Feb 25, 2011 1:12 pm
by gopal_NKP
hi,

One of the step of my job is merges two files and writes to one output file in sorted order using SyncSort.

The number of records in each file is about 30 millions . it is taking 14 minutes 37 seconds.

Please help to decrease this CPU usage time to minimum level.

Thanks
VGL

Re: Need Help on Performance Improvement of SyncSort

PostPosted: Fri Feb 25, 2011 3:49 pm
by Akatsukami
You have posted nothing to indicate that this job is taking an excessive amount of CPU time. Show us some JCL, control cards, and sysout.

Re: Need Help on Performance Improvement of SyncSort

PostPosted: Fri Feb 25, 2011 3:52 pm
by Robert Sample
The sort packages are NOT noted for wasting a lot of time, so your Syncsort time may be close to the minimum already. Is the 14 minutes 37 seconds elapsed time or CPU time? And remember, handling 60 million records will take a good bit of time, no matter what you want.

Re: Need Help on Performance Improvement of SyncSort

PostPosted: Fri Feb 25, 2011 8:25 pm
by BillyBoyo
As Robert says, things like sorts are pretty nippy already. If you are just merging two files, that's a pretty simple thing which doesn't leave much room for optimisation. It reads a record on file 1, reads a record on file 2, compares the keys, writes out the lower key, reads the file that had the lower key, repeats until end.

How often will the job be running?

How many key fields are defined?

What type of files are the input and output? Ensure optimal blocking.

Try to dig up a manual. I'm sure it will have advice on treating big files.

Don't expect any big savings, so consider the first question, which is asking "is it worth it"?

Re: Need Help on Performance Improvement of SyncSort

PostPosted: Tue Mar 01, 2011 9:22 pm
by Alissa Margulies
Without seeing any of the JCL or sort messages, the only recommendation I can make to possibly improve CPU time would be to pass the CPU optimization parm...
//$ORTPARM DD *
 CPU

If you would like further assistance tuning this application, please email the COMPLETE job listing to me at alissa.margulies@syncsort.com and I would be happy to review it for you.

Regards,