New Matching of files



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

New Matching of files

Postby pulcinella » Sun May 02, 2010 10:56 pm

Hello frank,

sorry to open this new post. I konw that This problem I've posted on April 20 but does not work

I want maching two files. This maching executed all days and compare the changes produced at two days (after day, before day)... Two files have been ordered and have the same key. The files have one only record by key

After day (16 position)

fieldAfterA have 4 position (the same value. Example: xxxx
fieldAfterB have 8 position (the key. No duplicates)
fieldAfterC have 4 position (may have duplicates)

Before day (16 position)

fieldBeforeA have 4 position (the same value. Example: xxxx)
fieldBeforeB have 8 position (the key. No duplicates)
fieldBeforeC have 4 position (may have duplicates)

I want obtain a third file (OUTPUT) with 17 position

fieldoutputA have 4 position (the same value. Example: xxxx)
fieldoutputB have 8 position
fieldoutputC have 4 position
fieldoutputD have 1 position (values: A (new record), b (old record), m (modification record)

If fieldAfterB equal to fieldBeforeB, I must be compare fieldAfterC with fieldBeforeC. If not the same, I must write OUTPUT file with the after day record and the value 'M' at field fieldoutputD

If fieldAfterB greather than fieldBeforeB, I write OUTPUT file with the after day record and the value 'A' at field fieldoutputD

If fieldAfterB less than fieldBeforeB, I write OUTPUT file with the before day record and the value 'B' at field fieldoutputD

RECFM = FB and LRECL = 16 to the input files
OUTPUT file is FB and LRECL = 17 (hte last position is 'A', 'B', 'M')

Very kindly skolosu gave me this example


//STEP0100 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD *
//SORTOUT DD DSN=&&HDR,DISP=(,PASS),SPACE=(TRK,(1,0),RLSE)
//SYSIN DD *
SORT FIELDS=COPY
OUTFIL REMOVECC,NODETAIL,BUILD=(16X),HEADER1=(C'$$$')
//*
//STEP0200 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=&&HDR,DISP=SHR,VOL=REF=*.STEP0100.SORTOUT
// DD DSN=your Afterday file,DISP=SHR
// DD DSN=&&HDR,DISP=SHR,VOL=REF=*.STEP0100.SORTOUT
// DD DSN=your Beforeday file,DISP=SHR
//SORTOUT DD SYSOUT=*
//SYSIN DD *
INREC IFTHEN=(WHEN=INIT,OVERLAY=(17:C'1')),
IFTHEN=(WHEN=GROUP,BEGIN=(1,3,CH,EQ,C'$$$'),PUSH=(18:ID=1))
SORT FIELDS=(5,12,CH,A),EQUALS
SUM FIELDS=(17,1,ZD)

OUTREC IFTHEN=(WHEN=INIT,OVERLAY=(20:SEQNUM,8,ZD,RESTART=(5,8))),
IFTHEN=(WHEN=GROUP,BEGIN=(20,8,ZD,EQ,1),PUSH=(19:18,1,28:13,4)),
IFTHEN=(WHEN=(18,2,ZD,EQ,11),OVERLAY=(32:C'A')),
IFTHEN=(WHEN=(18,2,ZD,EQ,12),OVERLAY=(32:C'M')),
IFTHEN=(WHEN=(18,2,ZD,EQ,21),OVERLAY=(13:28,4,32:C'M')),
IFTHEN=(WHEN=(18,2,ZD,EQ,22),OVERLAY=(32:C'B'))

OUTFIL OMIT=(17,1,ZD,EQ,2),BUILD=(17X),
REMOVECC,NODETAIL,SECTIONS=(5,12,TRAILER3=(1,16,32,1))
//*

When I be tested I find that has been records duplicated and I don't know why...

INPUT FILE1

XXXX001644700077
XXXX007429990005
XXXX027446520154
XXXX056949160128
XXXX099726340116
XXXX134319250167
XXXX147229920154
XXXX302214770144
XXXX313517600076
XXXX347038520171
XXXX358358390070
XXXX364342970019
XXXX365592750052
XXXX365608310272
XXXX366075660167
XXXX366210830003
XXXX366213370003
XXXX366267940224
XXXX367596540339
XXXX368957540056
XXXX368975720000
XXXX368975850183

INPUT FILE2

XXXX001644700139
XXXX007429990079
XXXX027446520052
XXXX056949160140
XXXX065226210041
XXXX099726340056
XXXX134319250117
XXXX141101800443
XXXX147229920025
XXXX302214770443
XXXX313517600122
XXXX347038520114
XXXX358358390443
XXXX364342970116
XXXX365592750125
XXXX365608310043
XXXX366075660003
XXXX366210830004
XXXX366213370004
XXXX366267940054
XXXX367596540174
XXXX368957540105
XXXX368975720444
XXXX368975850453


OUTPUT FILE

XXXX001644700077M
XXXX007429990005M
XXXX027446520052B
XXXX027446520154M
XXXX056949160128M
XXXX065226210041B
XXXX099726340056B
XXXX099726340116M
XXXX134319250117B
XXXX134319250167M
XXXX141101800443B
XXXX147229920025B
XXXX147229920154M
XXXX302214770144M
XXXX313517600076M
XXXX347038520114B
XXXX347038520171M
XXXX358358390070M
XXXX364342970019M
XXXX365592750052M
XXXX365608310043B
XXXX365608310272M
XXXX366075660003B
XXXX366075660167M
XXXX366210830003M
XXXX366213370003M
XXXX366267940054B
XXXX366267940224M
XXXX367596540174B
XXXX367596540339M
XXXX368957540056M
XXXX368975720000M
XXXX368975850183M
XXXX358358390070M
XXXX364342970019M
XXXX365592750052M
XXXX365608310043B
XXXX365608310272M
XXXX366075660003B
XXXX366075660167M
XXXX366210830003M
XXXX366213370003M
XXXX366267940054B
XXXX366267940224M
XXXX367596540174B
XXXX367596540339M
XXXX368957540056M
XXXX368975720000M
XXXX368975850183M

You can see that XXXX02744652, XXXX09972634, XXXX13431925, XXXX14722992 has a record with 'B' and record with 'M'... I only want one unique record. If the record is new have and the last position an 'A'; If the record has modified have a 'M' and the las position and if the record not exist have a 'B'... I don't have two records ('B' and 'M').

At sysout I see this:

ICE143I 0 BLOCKSET COPY TECHNIQUE SELECTED
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AN
ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R5 - 18:27 ON SUN MAY
SORT FIELDS=COPY
OUTFIL REMOVECC,NODETAIL,BUILD=(16X),HEADER1=(C'$$$')
ICE201I F RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE751I 0 C5-K90013 C6-K90013 C7-K90000 C8-K42135 E9-K90013 C9-BASE E5-K44563
ICE193I 0 ICEAM1 ENVIRONMENT IN EFFECT - ICEAM1 INSTALLATION MODULE SELECTED
ICE088I 0 PEAISORT.STEP0100. , INPUT LRECL = 80, BLKSIZE = 80, TYPE = FB
ICE093I 0 MAIN STORAGE = (MAX,6291456,6262784)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (6200910,6200910)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,
ICE128I 0 OPTIONS: SIZE=6291456,MAXLIM=2097152,MINLIM=450560,EQUALS=N,LIST=Y,ERE
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=
ICE130I 0 OPTIONS: RESALL=8192,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=
ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITC
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=2097152,SOLRF=Y,VLLONG=N
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE084I 0 BSAM ACCESS METHOD USED FOR SORTIN
ICE231I 0 STORAGE USED FOR OUTFIL : BELOW 16M = 22528, ABOVE 16M = 2124800
ICE210I 0 SORTOUT : EXCP USED, LRECL = 16, BLKSIZE = 27984, TYPE = FB (SDB)
ICE751I 1 EF-K10929 CB-K90013 F0-K49037 E8-K44563
ICE055I 0 INSERT 0, DELETE 0
ICE054I 0 RECORDS - IN: 0, OUT: 0
ICE227I 0 SORTOUT : DELETED = 0, REPORT = 1, DATA = 0
ICE228I 0 SORTOUT : TOTAL IN = 0, TOTAL OUT = 1
ICE174I 0 NO DATA RECORDS FOR AN OUTFIL DATA SET - RC=0
ICE052I 0 END OF DFSORT




ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AN
ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R5 - 18:27 ON SUN MAY
INREC IFTHEN=(WHEN=INIT,OVERLAY=(17:C'1')),
IFTHEN=(WHEN=GROUP,BEGIN=(1,3,CH,EQ,C'$$$'),PUSH=(18:ID=1))
SORT FIELDS=(5,12,CH,A),EQUALS
SUM FIELDS=(17,1,ZD)
OUTREC IFTHEN=(WHEN=INIT,OVERLAY=(20:SEQNUM,8,ZD,RESTART=(5,8))),
IFTHEN=(WHEN=GROUP,BEGIN=(20,8,ZD,EQ,1),PUSH=(19:18,1,28:13,4)),
IFTHEN=(WHEN=(18,2,ZD,EQ,11),OVERLAY=(32:C'A')),
IFTHEN=(WHEN=(18,2,ZD,EQ,12),OVERLAY=(32:C'M')),
IFTHEN=(WHEN=(18,2,ZD,EQ,21),OVERLAY=(13:28,4,32:C'M')),
IFTHEN=(WHEN=(18,2,ZD,EQ,22),OVERLAY=(32:C'B'))
OUTFIL OMIT=(17,1,ZD,EQ,2),BUILD=(17X),
REMOVECC,NODETAIL,SECTIONS=(5,12,TRAILER3=(1,16,32,1))
ICE201I F RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE751I 0 C5-K90013 C6-K90013 C7-K90000 C8-K42135 E9-K90013 C9-BASE E5-K44563
ICE193I 0 ICEAM1 ENVIRONMENT IN EFFECT - ICEAM1 INSTALLATION MODULE SELECTED
ICE088I 1 JOBSORT.STEP0200. , INPUT LRECL = 16, BLKSIZE = 27984, TYPE =
ICE093I 0 MAIN STORAGE = (MAX,6291456,6291456)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (6234096,6234096)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,
ICE128I 0 OPTIONS: SIZE=6291456,MAXLIM=2097152,MINLIM=450560,EQUALS=Y,LIST=Y,ERE
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=
ICE130I 0 OPTIONS: RESALL=8192,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=
ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITC
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=2097152,SOLRF=Y,VLLONG=N
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE084I 0 EXCP ACCESS METHOD USED FOR SORTIN
ICE750I 0 DC 111936 TC 0 CS DSVNN KSZ 16 VSZ 16
ICE752I 0 FSZ=6996 RC IGN=0 E AVG=24 0 WSP=218 C DYN=0 0
ICE231I 0 STORAGE USED FOR OUTFIL : BELOW 16M = 47104, ABOVE 16M = 84992
ICE210I 0 SORTOUT : BSAM USED, LRECL = 17, BLKSIZE = 17, TYPE = FB
ICE751I 1 DE-K24705 D5-K24705 D9-K24705 CB-K90013 E8-K44563
ICE080I 0 IN MAIN STORAGE SORT
ICE055I 0 INSERT 0, DELETE 1
ICE054I 0 RECORDS - IN: 48, OUT: 47
ICE227I 0 SORTOUT : DELETED = 47, REPORT = 33, DATA = 0
ICE228I 0 SORTOUT : TOTAL IN = 47, TOTAL OUT = 33
ICE174I 0 NO DATA RECORDS FOR AN OUTFIL DATA SET - RC=0
ICE134I 0 NUMBER OF BYTES SORTED: 768
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 0 , TRACKS USED: 0
ICE199I 0 MEMORY OBJECT STORAGE USED = 0M BYTES
ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES
ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
ICE052I 0 END OF DFSORT

Could you help me. Thank you very much.
pulcinella
 
Posts: 114
Joined: Mon Dec 10, 2007 10:18 pm
Has thanked: 0 time
Been thanked: 0 time

Re: New Matching of files

 

Re: New Maching of files

Postby Frank Yaeger » Tue May 04, 2010 2:21 am

I see the following in the last line of Kolusu's job:

REMOVECC,NODETAIL,SECTIONS=(5,8,TRAILER3=(1,16,32,1))

But I see the following in the last line of the job you say Kolusu posted:

REMOVECC,NODETAIL,SECTIONS=(5,12,TRAILER3=(1,16,32,1))

Examine these lines closely. He has 5,8 - you have 5,12. If I run with KOLUSU'S JOB, I do NOT get the duplicate records you're complaining about. If I run with YOUR JOB, I do.

After Kolusu told you his job worked, don't you think it would have made sense to do a compare of your job and his job to see if something was different?
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1080
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 14 times

Re: New Maching of files

Postby dick scherrer » Tue May 04, 2010 3:09 am

Hi Frank,

You and Kolusu must not have received the memo. . .

If the IBM provided solution is not used and the replacement code does not work, IBM will still provide resources to resolve this. . . 8-)

:twisted:

d
User avatar
dick scherrer
Global moderator
 
Posts: 6304
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 91 times

Re: New Maching of files

Postby Frank Yaeger » Tue May 04, 2010 3:13 am

Well, we'll at least try to point out that the original code we supplied does what we think was asked for (unless proven otherwise). :roll:
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1080
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 14 times

Re: New Maching of files

Postby pulcinella » Tue May 04, 2010 3:14 pm

Hello Frank (& Kolusu)

The first code I tried did not work. Were produced duplicates. Kolosu gave me another solution where only thought I had added a new line to the conditions. I did not realize he had also changed the last line.

I've done several tests and seems to work. Thank you both and apologies for any inconvenience.
pulcinella
 
Posts: 114
Joined: Mon Dec 10, 2007 10:18 pm
Has thanked: 0 time
Been thanked: 0 time

Re: New Maching of files

Postby skolusu » Tue May 04, 2010 9:17 pm

pulcinella wrote:Hello Frank (& Kolusu)

The first code I tried did not work. Were produced duplicates. Kolosu gave me another solution where only thought I had added a new line to the conditions. I did not realize he had also changed the last line.

I've done several tests and seems to work. Thank you both and apologies for any inconvenience.



pulcinella,

What part of my post did you NOT understand where I specifically pointed it out to "Pay attention to the control cards I posted especially the SECTIONS statement where I am picking only 1 key based on the key" in this post?

dfsort-icetool-icegener/topic3317-10.html#p13215

Last but not least , I would really appreciate if you can at least spell my name correctly.
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
skolusu
 
Posts: 586
Joined: Wed Apr 02, 2008 10:38 pm
Has thanked: 0 time
Been thanked: 39 times

Re: New Matching of files

Postby pulcinella » Tue May 04, 2010 9:55 pm

Ok Skolusu

Thanks
pulcinella
 
Posts: 114
Joined: Mon Dec 10, 2007 10:18 pm
Has thanked: 0 time
Been thanked: 0 time


Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post