Reducing CPU Time and making Sort Efficient



Support for NetApp SyncSort for z/OS, Visual SyncSort, SYNCINIT, SYNCLIST and SYNCTOOL

Reducing CPU Time and making Sort Efficient

Postby saurabh pune » Thu Jan 05, 2012 9:49 pm

Hi,
I am using the below JCL for.

1) Sorting the input files with KEY1, KEY2, KEY3 and KEY4 ( Already using 46 SortWK files)
2) Sorting the o/p of 1st step with KEY1, KEY2 and KEY3 using SUMFIELDS=NONE for removing duplicates( Already using 46 SortWK files)

3) Cobol program to compare the current file (O/P from 2nd step with previous version)
4) Cobol Program : Formatting of File
5) Sort Step : Adding a header to this file
6) Copying the O/P from 2nd step as previous file for comaring it next time.

The majority of time (80%) is taken by the 1st and 2nd steps to complete.

I/P File : RECFM=FB,LRECL=265
We are using SYNCSORT FOR Z/OS 1.4.0.1R

SYSOUT for the SORT in the first step is :
WER276B  SYSDIAG= 104402, 282114, 282114, 392325                             
WER164B  18,032K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,           
WER164B     0 BYTES RESERVE REQUESTED, 17,696K BYTES USED                     
WER146B  32K BYTES OF EMERGENCY SPACE ALLOCATED                               
WER108I  SORTIN   : RECFM=FB   ; LRECL=   265; BLKSIZE= 27825                 
WER073I  SORTIN   : DSNAME=HLQ.ABC.XYZ.G001V00                       
WER110I  SORTOUT  : RECFM=FB   ; LRECL=   265; BLKSIZE= 32595                 
WER074I  SORTOUT  : DSNAME= HLQ.XYZ.ABC1                         
WER410B  17,004K BYTES OF VIRTUAL STORAGE AVAILABLE ABOVE THE 16MEG LINE,     
WER410B     0 BYTES RESERVE REQUESTED, 16,980K BYTES USED                     
WER036B  G=61165,B=17,BIAS=01                                                 
WER162B  2,235,000 PREALLOCATED SORTWORK TRACKS, 0 DYNAMICALLY ALLOCATED,     
WER162B     0 ACQUIRED IN SECONDARY EXTENTS, 676,995 RELEASED, TOTAL OF 1,557,
WER124I  PREALLOCATED/USED SORTWORK SPACE USAGE FACTOR = 1.43                 
WER045C  END SORT PHASE                                                       
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000                                 
WER449I  SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE                                 
WER416B  BSAM WAS USED FOR SORTIN                                             
WER416B  BSAM WAS USED FOR SORTOUT                                           



Please let me know If I could provide you with any other information

Is there some way here to tune the performance of the first two sort steps and reduce the run time ?

Thanks,
S
Regards,
S
saurabh pune
 
Posts: 6
Joined: Sun Nov 20, 2011 6:16 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Reducing CPU Time and making Sort Efficient

 

Re: Reducing CPU Time and making Sort Efficient

Postby dick scherrer » Fri Jan 06, 2012 12:28 am

Hello,

Is there some way here to tune the performance of the first two sort steps and reduce the run time
Maybe. . .

On what media is the input data? How long does each step now run? How busy was the system (nearly empty or very busy). Running a test at some off-peak hour may change the run time.

Was there some reason to post only part of the Syncsort informational messages?
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6304
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 91 times

Re: Reducing CPU Time and making Sort Efficient

Postby saurabh pune » Fri Jan 06, 2012 12:50 am

Hi Dick,

Some More Info :
I/P File is on the Disk, Not on Tape
1st step runs for almost 4-5 hrs and 2nd step 1.5 - 2 hrs and other 4 in almost 30 mins.
System Busy or Empty? Could you please guide me on how to check this info.
Ran the test at Off-Peak hour improves the total run time by arnd 20-30 mins , marginally not much of a difference .

Posted all the Messages, Nothing intentional. Let me know if we would need other parts I would surely post those If that helps me .

Thanks a lot for your time Dick.

Regards,
Saurabh
Regards,
S
saurabh pune
 
Posts: 6
Joined: Sun Nov 20, 2011 6:16 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Reducing CPU Time and making Sort Efficient

Postby NicC » Fri Jan 06, 2012 1:09 am

I am not familiar with Syncsort but nowhere do I see the record count - nor do you mention it in your post. Also, nowhere do I see your sort cards so no one can comment on them. Have you tried letting Syncsort dynamically allocate its work files?
The problem I have is that people can explain things quickly but I can only comprehend slowly.
Regards
Nic
NicC
Global moderator
 
Posts: 2690
Joined: Sun Jul 04, 2010 12:13 am
Location: Pushing up the daisys (almost)
Has thanked: 4 times
Been thanked: 105 times

Re: Reducing CPU Time and making Sort Efficient

Postby dick scherrer » Fri Jan 06, 2012 1:12 am

Hello,

You're welcome :)

Posted all the Messages
Hmmm. . .

When i run Syncsort, i get:
The product release.
The sort control statements submitted.
Record counts.
And a bit of other info. . .

System Busy or Empty? Could you please guide me on how to check this info.
Best to check with your Scheduling or Operations people. They will know when the system is heavily loaded and when it is nearly empty (if it ever is). You can also display all of the active processes and when there are several screens of things running the system will most likely be busy while when there are few active, the system will be not so busy. At the top of the display there should be a line that shows how busy the system is overall. Some organizations also publish hour by hour statistics of how busy the system has been over the last day/week/month - whatever.

Is there some chance that either of the 2 large files are being used by some other process at the same time? If yes, this can be a killer.

It may help if you post the JCL used for the first 2 steps.
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6304
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 91 times

Re: Reducing CPU Time and making Sort Efficient

Postby BillyBoyo » Fri Jan 06, 2012 5:31 pm

From what you have said, but not yet shown, your second step is sorting an already-sorted file. You've used Cobol elsewhere, I'm surprised you didn't consider it for this. Even then, you have to think "it must be possible in sort" and start to look at the "tool" options and start to wonder if anything can even be built into the first step to do it.

But we'd need to see what your sort steps look like, as has already been asked.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 264 times

Re: Reducing CPU Time and making Sort Efficient

Postby Alissa Margulies » Tue Jan 31, 2012 2:57 am

Hello saurabh pune.

If you are still looking to tune this application, please forward the complete job listing (including the JES Job Log, JCL, SYSOUT, and SYNCSORT messages) to zos_tech@syncsort.com and we would be happy to further assist you.

Regards,
Alissa Margulies
Syncsort Mainframe Product Services
201-930-8260
zos_tech@syncsort.com
Alissa Margulies
Global moderator
 
Posts: 369
Joined: Tue Feb 26, 2008 11:15 pm
Location: USA
Has thanked: 1 time
Been thanked: 1 time

Re: Reducing CPU Time and making Sort Efficient

Postby saurabh pune » Wed Feb 01, 2012 11:21 pm

Thanks, Stopped working on this got a partial solution, Combined 1st and 2nd step, although not much successful in the results.

Thanks a ton though..
Regards,
S
saurabh pune
 
Posts: 6
Joined: Sun Nov 20, 2011 6:16 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Reducing CPU Time and making Sort Efficient

Postby BillyBoyo » Wed Feb 01, 2012 11:35 pm

Can't believe you're going to let the offer go by. It's not work you have to do, beyond sending an e-mail :-)
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 264 times

Re: Reducing CPU Time and making Sort Efficient

Postby anurag_87 » Tue Feb 07, 2012 3:10 am

Hi Saurabh,
Can you try this:-
1)Combine Step1 and Step2:-Also verify in the job log how many sort work volumes your job uses while sorting and remove if you are having additional volumes.Sort work volumes should be much more costlier than CPU sec.
Specify MINCORE parm too for such huge sorting.This will ensure your job does not fail with error mention.

2)Step3-Step6 may also be done by using syncosrt join keys and outrec etc.Sort is most of the time more effecient then cobol.

Also last but not the least,can you check if your job is more I/O bound or CPU bound.
If its I/O bound I would recommend to do all your step via a assembler/cobol program, it would complete faster.

Regards
Anurag
anurag_87
 
Posts: 7
Joined: Mon Feb 06, 2012 4:03 pm
Has thanked: 0 time
Been thanked: 0 time


Return to Syncsort/Synctool

 


  • Related topics
    Replies
    Views
    Last post