Compression of DASD BLKSIZE



Support for NetApp SyncSort for z/OS, Visual SyncSort, SYNCINIT, SYNCLIST and SYNCTOOL

Compression of DASD BLKSIZE

Postby The Mean Farmer » Wed Oct 10, 2012 1:13 am

In doing some research I stumbled over the obvious fact that many of my SyncSort jobs are creating their output with really bad BLKSIZE factors.

From SYSOUT of a Sync Sort Step:
WER108I SORTIN : RECFM=VB ; LRECL= 3854; BLKSIZE= 32760
WER110I SORTOUT : RECFM=VB ; LRECL= 3854; BLKSIZE= 32760

BLKSIZE is NOT coded anywhere in the JCL. The SMS Compression utility that we use is forcing this blocking factor on the file. This has lead to some odd results:

This particular file has a 90% compression ratio - so it is compressing the file quite well. 41,000 cylinders down to 3450.
If I force an optimal BLKSIZE on the Output of this SORT step the file utilizes 800 less cylinders and DASD and 1/3 less SORTWK areas.
If I force an optimal BLKSIZE on the COBOL Application that creates the Input to the SORT above is doesn't use any less DASD.

I forced the Input and Output to an optimal BLKSIZE in production last night and despite the optimum BLKSIZE on the file it consumed slightly less DASD and didn't help runtimes at all.

So, does Compression show bad blocking factors but work its magic behind the scenes? Is this something I should pursue as a cycle efficiency? Is this something that is expected? Should a file that size be utilizing the MAXSORT option?
The Mean Farmer
 
Posts: 16
Joined: Fri Nov 05, 2010 12:48 am
Has thanked: 0 time
Been thanked: 0 time

Re: Compression of DASD BLKSIZE

Postby dick scherrer » Wed Oct 10, 2012 1:51 am

Hello,

I see nothing in your post that mentions how the compression ratio was determined.

Typically, Syncsort will use the same blksize for the output as the input (unless told otherwise).

What created the SORTIN file? Why is this not at the better blksize? How many records/bytes are being sorted?
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6268
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 93 times

Re: Compression of DASD BLKSIZE

Postby steve-myers » Wed Oct 10, 2012 7:00 am

BLKSIZE=32760 is not very good on DASD, but it's fine on tape. On disk it's using roughly 60% of a 3390 track. That''s just a quick mental calculation, so it could be considerably off.

There can be problems with VB if the LRECL is large compared with BLKSIZE and many real records are close to LRECL since the actual data blocks can potentially be much less than the maximum BLKSIZE. I don't think this is too much of an issue with your data, but it's worth analyzing. Most print type data in programs I write is directed to VBA data sets, so I can't say I've actually observed this problem with my programs where LRECL is typically 125 or 137 and BLKSIZE is system determined BLKSIZE, but you can potentially have this problem.

I don't know how or why SORTIN or SORTOUT BLKSIZE would impact SORTWK usage; it might be a topic worth pursuing with Syncsort support. Perhaps if the moderators would move this topic to the Syncsort section, the Syncsort tech that sporadically audits the forum might pick it up and comment.

When I want to force system determined "optimal" BLKSIZE in JCL, I always specify DSORG=PS in the JCL so the BLKSIZE is calculated when the data set is allocated, rather than in OPEN, when the value is sometimes distorted.

These users thanked the author steve-myers for the post:
Akatsukami (Wed Oct 10, 2012 9:06 am)
steve-myers
Global moderator
 
Posts: 2105
Joined: Thu Jun 03, 2010 6:21 pm
Has thanked: 4 times
Been thanked: 243 times

Re: Compression of DASD BLKSIZE

Postby The Mean Farmer » Thu Oct 11, 2012 2:24 am

The FD statement that the COBOL application uses to create this file is coded as "BLOCK CONTAINS 0 RECORDS". If I run this program writing output to a non-compressed DASD dataset the Block Size comes out optimal, but if I write to a dataset that is utilizing Compressed DASD the Block Size is 32760.
Syncsort is using the Input BLKSIZE as the Output BLKSIZE. Unless I force it with a JCL BLKSIZE= parameter.

WER246I FILESIZE 90,996,724,689 BYTES
WER054I RCD IN 41914659, OUT 41914659

It looks like the compression utility is using tape block sizing for all files, but that doesn't make sense to me.

I will nag my systems people to see if they can get an answer. Thanks for the input!!!
The Mean Farmer
 
Posts: 16
Joined: Fri Nov 05, 2010 12:48 am
Has thanked: 0 time
Been thanked: 0 time

Re: Compression of DASD BLKSIZE

Postby dick scherrer » Thu Oct 11, 2012 7:53 pm

Hello,

I will nag my systems people to see if they can get an answer.
Yup :)
Suggest you speak with your storage management people and learn how your environment is set up. There are multiple ways to define "compression" and they should be able to clarify what is going on with your data.
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6268
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 93 times

Re: Compression of DASD BLKSIZE

Postby Yifat Oren » Wed Oct 17, 2012 7:09 pm

From z/OS DFSMS "Using Data Sets":

In a compressed format data set, the BLKSIZE value has no relationship with the actual size of blocks on disk. The BLKSIZE
value specifies the maximum length of uncompressed blocks.

So, SDB for compressed-format data sets is always half a track. This is the best BLKSIZE for I/O optimization (and it has no effect on dasd utilization) for compressed-format data sets.

Hope this helps,
Yifat
Yifat Oren
 
Posts: 6
Joined: Tue Jul 20, 2010 4:14 pm
Has thanked: 0 time
Been thanked: 0 time


Return to Syncsort/Synctool