## How to calculate sortwork

IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

### How to calculate sortwork

Heelo,

i would like to know if there's a methode to calculate the space and the number of sortwork needed for a sort step.

For example,

my dataset in entry have 712000000 records. LRECL :626. RECFM : VB BLKSIZE : 27998

It takes 12 tapes.

The sort is very simple :

SORT FIELDS=(22,91,A),FORMAT=CH

The dataset resulting will be on tape two.

I would like to optimize this sort by using the best dynamic allocation.

Actually, i'am using 60 sortwork with :

SPACE=(CYL,(100,200))

I have this message :

ICE054I 0 RECORDS - IN: 712250276, OUT: 712250276
ICE134I 0 NUMBER OF BYTES SORTED: 145940009076
ICE253I 0 RECORDS SORTED - PROCESSED: 712250276, EXPECTED: 467886206
ICE098I 0 AVERAGE RECORD LENGTH - PROCESSED: 204, EXPECTED: 313
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 2662500 , TRACKS USED: 2661120

Thank's for your help.
samb01

Posts: 221
Joined: Mon Nov 16, 2009 7:24 pm
Has thanked: 0 time
Been thanked: 0 time

### Re: How to calculate sortwork

samb01 wrote:Heelo,

i would like to know if there's a methode to calculate the space and the number of sortwork needed for a sort step.

For example,

my dataset in entry have 712000000 records. LRECL :626. RECFM : VB BLKSIZE : 27998

It takes 12 tapes.

The sort is very simple :

SORT FIELDS=(22,91,A),FORMAT=CH

The dataset resulting will be on tape two.

I would like to optimize this sort by using the best dynamic allocation.

Actually, i'am using 60 sortwork with :

SPACE=(CYL,(100,200))

I have this message :

ICE054I 0 RECORDS - IN: 712250276, OUT: 712250276
ICE134I 0 NUMBER OF BYTES SORTED: 145940009076
ICE253I 0 RECORDS SORTED - PROCESSED: 712250276, EXPECTED: 467886206
ICE098I 0 AVERAGE RECORD LENGTH - PROCESSED: 204, EXPECTED: 313
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 2662500 , TRACKS USED: 2661120

Thank's for your help.
• I''m not a sort expert, but I'd guess you did not provide enough sort work space. I suspect you need to tell sort a good estimate of the number of records in your data, and, since the data contains variable length records, a good estimate of the average record size. The mechanism to do this is documented in the sort manuals.
• I do not understand this; the input appears to be on 12 tape volumes, but you appear to be saying you expect the output to require 2 tape volumes. You seem to be saying you want to compact 12 sausages into 2. Or, does, "The dataset resulting will be on tape two," mean something else?
steve-myers
Global moderator

Posts: 1773
Joined: Thu Jun 03, 2010 6:21 pm
Has thanked: 4 times
Been thanked: 168 times

### Re: How to calculate sortwork

steve-myers wrote:I do not understand this; the input appears to be on 12 tape volumes, but you appear to be saying you expect the output to require 2 tape volumes. You seem to be saying you want to compact 12 sausages into 2. Or, does, "The dataset resulting will be on tape two," mean something else?

"Two" = "too"?
"You have sat too long for any good you have been doing lately ... Depart, I say; and let us have done with you. In the name of God, go!" -- what I say to a junior programmer at least once a day

Akatsukami
Global moderator

Posts: 999
Joined: Sat Oct 16, 2010 2:31 am
Location: Bloomington, IL
Has thanked: 6 times
Been thanked: 48 times

### Re: How to calculate sortwork

Samb01 wrote:Actually, i'am using 60 sortwork with :
SPACE=(CYL,(100,200))

Since you have hard coded JCL SORTWORKs In your JCL,your allocation of 100 cylinders as primary and 200 as secondary spread across 60 sortwork datasets would result in

60 X (1500 + 45000 ) = 2,790,000

Now look at the message ICE165I
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 2662500 , TRACKS USED: 2661120

DFSORT allocated all the 60 sortworks with primary of 100 cylinders, the secondary extents will only come into picture as they are are needed.

But as steve-myers pointed out, you can provide the AVGLEN and estimated number of records.

Apart from that use DFSORT's Dynamic Allocation and get rid off the JCL sort work datasets.

Add the following line before your SORT statement and remove your JCL sortwork and re-run your job.

OPTION AVGRLEN=204,DYNALLOC=(,60),FILSZ=E730000000
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
skolusu

Posts: 586
Joined: Wed Apr 02, 2008 10:38 pm
Has thanked: 0 time
Been thanked: 39 times

### Re: How to calculate sortwork

I have very old notes (10+ years) on SORTWK I got from a systems programmer that you are welcome to play with. Have fun...

- SORTWK Calculation in Tracks
-- (A * B * 1.3) / C
|   |          |
|   |          |> track capacity (47,476 tracks for 3380)
|   |          |>                (56,664 tracks for 3390)
|   |> Average record length
|> Number of records to sort

-- notes...---max SORTWK space in tracks should be 2-3 k
---50,000 tracks per 3380 disk pack
c62ap90

Posts: 125
Joined: Thu Oct 11, 2012 10:24 pm
Has thanked: 1 time
Been thanked: 6 times

### Re: How to calculate sortwork

You may have had it for 10 years, but I think that the advice is at least 30 years old and a little out-of-date.

Compare it to the actual example. If using this to calculate static SORTWK datasets, I'd check on how much "overallocation" there is.

Samb01,

The reason the average record-length is so different is that the bulk of your records are "small" in comparison to the LRECL. The more skewed the figure is, the more useful the estimate of records and a good average record-length will be.
BillyBoyo
Global moderator

Posts: 3738
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 255 times

### Re: How to calculate sortwork

BillyBoyo wrote:You may have had it for 10 years, but I think that the advice is at least 30 years old and a little out-of-date.

Compare it to the actual example. If using this to calculate static SORTWK datasets, I'd check on how much "overallocation" there is.

Actually if you replace your shops track capacity (i.e. 3390, model 1,2,3,9 etc.) into the calculation it looks much better. Google for your track capacity (very easy). I assume you did that? To get the CYLinder value, just divide by 15.
c62ap90

Posts: 125
Joined: Thu Oct 11, 2012 10:24 pm
Has thanked: 1 time
Been thanked: 6 times

### Re: How to calculate sortwork

I used your calculation with your figures. Only the model 9 has a different bytes-per-track. Tracks-on-a-pack is irrelevant to the calculation. Optimum blocksize is relevant but is static across the disk models within a range.

I've got an old SyncSort "Reference Card" dated 1986. By that time they were suggesting 1.2 * file size. From memory, the figure I used in 1979 (but don't know the year of publication) was 1.3.

If you are happy with 1.3, and with your company/client paying for using that, it's OK with me. I just don't like it being suggested to others.

I don't need to know how many cylinders, proportion is same whether calculated in tracks, bytes, cylinders or bananas. It is proportion, that is the way it works - same units throughout calculation, result will be the same whatever unit is used in calculation.

For me it is significantly overallocating, more especially since dynamic allocation will get a much better figure without having to calculate much at all
BillyBoyo
Global moderator

Posts: 3738
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 255 times

### Re: How to calculate sortwork

Heelo.

I will try this :

OPTION AVGRLEN=204,DYNALLOC=(,60),FILSZ=E730000000

as skolusu said.

But i would like to know if the sort will work more quiclkly ?

Actually i have 12 tape on entry et 12 tape out.

I think it will be work fatser if the dataset is on disk. What do you think about it.

The problem is the size of the dataset : 712 000 000 records with 624 lrecl.

I calculate 50 disk model 9 (our disk are model 9). Actually we don't have enought disk... but we can ask for having more disk if we can win time by doing the sort ondisk. Only if it worth it.
samb01

Posts: 221
Joined: Mon Nov 16, 2009 7:24 pm
Has thanked: 0 time
Been thanked: 0 time

### Re: How to calculate sortwork

Hello,

Only if it worth it.

Who decides this "worth"?
Are the tapes physical or virtual?
How long will the "output" be needed?

Hope this helps,
d.sch.

dick scherrer
Global moderator

Posts: 6308
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 90 times

Next