Page 2 of 3

Re: VSAM fiel copy with different key lengths

PostPosted: Sun Apr 29, 2012 4:33 am
by Nara_513
Sorry ...the file shave diff record count:

Input sort file: 26652426
output file after sort: 26651978

Re: VSAM fiel copy with different key lengths

PostPosted: Sun Apr 29, 2012 7:46 am
by dick scherrer
Hello,

How many duplicates were eliminated? Is this the same number as the difference in the record counts? It should be.

If one file is the original and the other has the duplicates removed, there Should be a difference.

What should happen to the values that are not the key for the duplicate records?

Do you now have the result you want/need or is there still something that needs to be changed?

Re: VSAM fiel copy with different key lengths

PostPosted: Sun Apr 29, 2012 12:09 pm
by BillyBoyo
OK, so the confusion grows.

If your original source was a VSAM KSDS, which must have unique keys, so therefore unique "records", how come

SUM FIELDS=NONE


would lead to the deletion of duplicate records?

Can you provided us with a LISTCAT of the original KSDS, and let us know how you copied it to the flat file in the first place, please?

Re: VSAM fiel copy with different key lengths

PostPosted: Sun Apr 29, 2012 2:02 pm
by NicC
What is your 'normal sort step' to copy a VSAM to PS? What is the RECFM of the PS file? If it is VB then your SORT control cards are wrong - they should be sorting from position 5 as the first 4 bytes are the RDW which is NOT part of your data.

Re: VSAM fiel copy with different key lengths

PostPosted: Sun Apr 29, 2012 2:49 pm
by BillyBoyo
That's what I was thinking. Either the KSDS must contain records of different lengths, or the copy is a VB through accidental commision, but for whatever reason the PS is VB.

Then, the Sort starts on the record-length of the RDW. Thus the last two bytes of the actual key are ignored, so keys are "duplicate" where the first 13 digits are equal, and the records in question are an equal length.

So, can you provided us with a LISTCAT of the original KSDS, and let us know how you copied it to the flat file in the first place, please? Show the LISTCAT and the JCL in the Code tags for readability.

Re: VSAM fiel copy with different key lengths

PostPosted: Mon Apr 30, 2012 6:07 am
by Nara_513
Hi All,

Please find the following:

MY Original KSDS file info:
Data Component Information:                   
 Device type:             3390                 
 Organization:            KSDS  EXT-ADDR COMP 
 KSDS key length:           23                 
 KSDS key location:          0                 
 Average record size:       73                 
 Maximum record size:       73                 
Allocated Space:    Unit    Primary  Secondary
 Data:         CYLINDERS        796        199
 Index:           TRACKS         67         17


Sort step used for copying the VSAM to flat file:
//STEP01  EXEC  SORTD5
//SORTIN   DD  DSN=XXXXX.VSAM.MINDEX,
//             DISP=SHR
//SORTOUT  DD  DSN=XXXX.FLAT.MINDEX.BKP,
//             DISP=(NEW,CATLG,DELETE),
//             UNIT=SYSDA,SPACE=(CYL,(800,200),RLSE),
//             DCB=(*.SORTIN)
//SYSIN  DD  *
 SORT FIELDS=COPY
/*


Flat file properties:

General Data                           Current Allocation
 Management class . . : MCTSTSTD        Allocated cylinders : 2,320
 Storage class  . . . : SCBASE          Allocated extents . : 9
  Volume serial . . . : TSA009
  Device type . . . . : 3390
 Data class . . . . . : DCBASE
  Organization  . . . : PS             Current Utilization
  Record format . . . : FB              Used cylinders  . . : 2,320
  Record length . . . : 73              Used extents  . . . : 9
  Block size  . . . . : 27959
  1st extent cylinders: 800
  Secondary cylinders : 200            Dates
  Data set name type  :                 Creation date . . . : 2012/04/27
  SMS Compressible. . : NO              Referenced date . . : 2012/04/27
                                        Expiration date . . : ***None***


I used the below sort card to get the count of the files:
//SYSIN    DD *
 SORT FIELDS=COPY
 OUTFIL NODETAIL,REMOVECC,
    TRAILER1=(COUNT=(M10,LENGTH=15))
/*

I have the answer for the queries u have posted:

My VSAM file can have duplicates values for the column 1 to 15, but the original VSAM file with key length 23 doesnt have any duplicates cos the next coulmn values from 16 to 23 makes the record unique.

Since i defined the VSAM with 15 as the Key, when i tried to copy it with the flat file(unsorted) it failed.
So after sorting and by removing the duplicates(first column of lenght 15), the REPRO command worked as the duplicates were now removed.

I used the belwo JCL to get the dup count:
//STEP040 EXEC PGM=ICETOOL,REGION=128M
//INPUT   DD  DSN=XXXX.FLAT.MINDEX.BKP,DISP=SHR
//UNIQUE  DD  DSN=XXXXXX.MINDEX.UNIQUE,
//            DISP=(NEW,CATLG,DELETE),
//            UNIT=SYSDA,
//            SPACE=(CYL,(500,200),RLSE),
//            DCB=(*.INPUT)
//OUTPUT  DD  DSN=XXXXX.MINDEX.DUP,
//            DISP=(NEW,CATLG,DELETE),
//            UNIT=SYSDA,
//            SPACE=(CYL,(500,200),RLSE),
//            DCB=(*.INPUT)
//TOOLIN   DD *
      SELECT FROM(INPUT) TO(OUTPUT) -
      ON(01,15,CH) -
      ALLDUPS DISCARD(UNIQUE)
//SYSOUT DD SYSOUT=*
//SSMSG  DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//TOOLMSG DD SYSOUT=*


I have checked the record count of the dup file and it is 896, on making it half we get 448 which is the difference between the original file count and the sorted file count:

At last i had the result that i need , the duplicate records that are removed ...though are unique , they are not in my processing list of records :-)

Re: VSAM fiel copy with different key lengths

PostPosted: Mon Apr 30, 2012 11:53 am
by BillyBoyo
You gave us some bad information earlier in relation to Robert's question about whether the second part of the key was to ensure that the first part was unique.

Instead of sorting your data to get rid of the duplicates, why don't you use SELECT, ensuring that it does not "sort"? You should be able to include all the unique records and the first duplicate at one go, without sorting your file (as it is already in sorted order, as it is in key order).

Re: VSAM fiel copy with different key lengths

PostPosted: Mon Apr 30, 2012 8:03 pm
by halfteck
I might suggest another option. If you still require the 23 byte key on the KSDS, and the 15 buye key as a second verson. You could maybe build an alternate index to the cluster, where duplicate keys are allowed.

Re: VSAM fiel copy with different key lengths

PostPosted: Mon Apr 30, 2012 8:10 pm
by Nara_513
BillyBoyo wrote:You gave us some bad information earlier in relation to Robert's question about whether the second part of the key was to ensure that the first part was unique.

Instead of sorting your data to get rid of the duplicates, why don't you use SELECT, ensuring that it does not "sort"? You should be able to include all the unique records and the first duplicate at one go, without sorting your file (as it is already in sorted order, as it is in key order).



I apoligize for that..its my mistake..

Re: VSAM fiel copy with different key lengths

PostPosted: Mon Apr 30, 2012 10:16 pm
by BillyBoyo
OK. Just take care. It makes your own job easier, and easier for us if you come here.

I used to outline everything in pencil on an old listing, Gives you a good grasp of everything, and how to follow through with keys and counts, etc. I think pencil and paper would still work these days :-)