How to split a file into two file and remove dups?



Support for NetApp SyncSort for z/OS, Visual SyncSort, SYNCINIT, SYNCLIST and SYNCTOOL

How to split a file into two file and remove dups?

Postby mainframegeek » Thu Oct 18, 2012 12:31 am

for Example

File 1
01 FILE-REC
05 LIC-NMBR PIC 9(2)
05 MIN-NMBR PIC 9(03)
05 MAX-NMBR PIC 9(03)

DATA IN LIC-FILE
11001004
11002005
12000006
12001007
13000001

OUTPUT FILE

11001005
12000007
13000001

We have to arrange the minimum value of MIN-NMBR field and maximum value of MAX-NMBR field for a particular LIC-NMBR.

Values for MIN-NMBR for LIC-NMBR 11 :
001
002
out of these two oo1 is minimum, so it has to be selected.

values for MAX-NMBR for LIC-NMBR 11:
004
005
so out of these 005 is maximum, so it has to selected.Output for LIC-NMBR 11 should be 11001005.


I need to do it using SYNCSORT, I tried doing it using ICETOOL. However i am unable to apply sort on MIN-NMBR and MAX-NMBR field.
Can anyone help me in this?
mainframegeek
 
Posts: 35
Joined: Sun Sep 16, 2012 1:57 pm
Has thanked: 3 times
Been thanked: 0 time

Re: How to split a file into two file and remove dups?

Postby dick scherrer » Thu Oct 18, 2012 1:38 am

Hello,

Post the job you submitted (using the Code tag), and the output you got (both the data and the informational messages) also with the code tag).

Hopefully, you tested with the same data you posted.
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6268
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 93 times

Re: How to split a file into two file and remove dups?

Postby BillyBoyo » Thu Oct 18, 2012 3:20 am

And what is wrong with what you were given for the previous time you asked with this title? Just look down a little, and there it is.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: How to split a file into two file and remove dups?

Postby mainframegeek » Thu Oct 18, 2012 12:39 pm

I didn't try with section and trailer3 option.

Below mentioned is the JCL i used.







/***********************************************************
//STEP0001 EXEC PGM=ICETOOL
//***********************************************************
//*$ COPY DATASET FROM TAPE TO DISK
//***********************************************************
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN01 DD DSN=RT.DTP0107.DISK.RANGE.SORTED.TEST1,DISP=SHR
//IN02 DD DSN=RT.DTP0107.DISK.RANGE.SORTED.TEST1,DISP=SHR
//T1 DD DSN=RT.DTP0107.DISK.RANGE.SORTED.TEST4,
// SPACE=(CYL,(300,300)),UNIT=(SYSDA,15),
// DISP=(MOD,CATLG,CATLG)
//T2 DD DSN=RT.DTP0107.DISK.RANGE.SORTED.TEST5,
// SPACE=(CYL,(300,300)),UNIT=(SYSDA,15),
// DISP=(MOD,CATLG,CATLG)
//T3 DD DSN=RT.DTP0107.DISK.RANGE.SORTED.TEST6,
// SPACE=(CYL,(300,300)),UNIT=(SYSDA,15),
// DISP=(MOD,CATLG,CATLG)
//OUT1 DD DSN=RT.DTP0107.DISK.RANGE.SORTED.TEST7,
// DCB=(RECFM=FB,LRECL=8,BLKSIZE=0),
// SPACE=(CYL,(300,300)),UNIT=(SYSDA,15),
// DISP=(NEW,CATLG,CATLG)
//TOOLIN DD DSN=RT.DTP0107.TEST1(TOOLIN),DISP=SHR
//CTL1CNTL DD DSN=RT.DTP0107.TEST1(CTL1CNTL),DISP=SHR
//CTL2CNTL DD DSN=RT.DTP0107.TEST1(CTL2CNTL),DISP=SHR
//CTL3CNTL DD DSN=RT.DTP0107.TEST1(CTL3CNTL),DISP=SHR
//CTL4CNTL DD DSN=RT.DTP0107.TEST1(CTL4CNTL),DISP=SHR

CTL1CNTL

INREC FIELDS=(01,02,03,03)
SORT FIELDS=(03,03,ZD,A)
OUTREC FIELDS=(001:001,002,
003:003,003,
009:C'11')

CTL2CNTL

INREC FIELDS=(01,02,06,03)
SORT FIELDS=(03,03,ZD,D)
OUTREC FIELDS=(001:001,002,
006:003,003,
009:C'22')

CTL3CNTL

SORT FIELDS=(01,02,ZD,A)
SUM FIELDS=NONE

CTL4CNTL

OUTFIL FNAMES=OUT1,INCLUDE=(09,2,CH,EQ,C'11',OR,
09,2,CH,EQ,C'22',OR,
09,2,CH,EQ,C'12')

TOOLIN

COPY FROM(IN01) TO(T1) USING(CTL1)
COPY FROM(IN02) TO(T2) USING(CTL2)
COPY FROM(T1) TO(T3) USING(CTL3)
COPY FROM(T2) TO(T3) USING(CTL3)
SPLICE FROM(T3) TO(OUT1) ON(1,2,ZD) -
WITH(10,1) WITH(06,3) USING(CTL4) KEEPNODUPS


Input File(IN01 & IN02) (RT.DTP0107.DISK.RANGE.SORTED.TEST1)

11002005
11001004
11003006
12003004
12001002
13004005

T1(RT.DTP0107.DISK.RANGE.SORTED.TEST4)
11002 11
11001 11
11003 11
12003 11
12001 11
13004 11

Value 11 at 9th column is used for indicating that this data comes from IN01.

T2 (RT.DTP0107.DISK.RANGE.SORTED.TEST5)
11 00522
11 00422
11 00622
12 00422
12 00222
13 00522

Value 22 at 9th column is used for indicating that this data comes from IN02.



T3(RT.DTP0107.DISK.RANGE.SORTED.TEST6)
11002 11
11001 11
11003 11
12003 11
12001 11
13004 11
11 00522
11 00422
11 00622
12 00422
12 00222
13 00522


Output OUT1(RT.DTP0107.DISK.RANGE.SORTED.TEST7)

11002006
12003002
13004005

Desired output
11001006
12001004
13004005
mainframegeek
 
Posts: 35
Joined: Sun Sep 16, 2012 1:57 pm
Has thanked: 3 times
Been thanked: 0 time

Re: How to split a file into two file and remove dups?

Postby enrico-sorichetti » Thu Oct 18, 2012 1:01 pm

Post the job you submitted (using the Code tag),


using the code tags is a due courtesy to people spending their time to help You

without
123456789012345678901234567890
ABCDEFGHIJKLMNOPQRSTUVWXYZ
abcdefghijklmnopqrst

with
123456789012345678901234567890
ABCDEFGHIJKLMNOPQRSTUVWXYZ
abcdefghijklmnopqrst


the info surrounded by the code tags is displayed with a fixed pitch font
making easy for everybody to REread code and jcl
cheers
enrico
When I tell somebody to RTFM or STFW I usually have the page open in another tab/window of my browser,
so that I am sure that the information requested can be reached with a very small effort
enrico-sorichetti
Global moderator
 
Posts: 2994
Joined: Fri Apr 18, 2008 11:25 pm
Has thanked: 0 time
Been thanked: 164 times

Re: How to split a file into two file and remove dups?

Postby BillyBoyo » Thu Oct 18, 2012 1:42 pm

If you cannot be bothered to even try the very simple solution already provided to you, then I don't see why anyone should spend time looking at your highly-convoluted approach. I'm not even going to "code" your post, as I don't believe anyone should be interested in it.

This is the most succint solution provided to the previous run through the same task. http://www.ibmmainframeforum.com/syncsort-synctool/topic8160.html#p38527

Note that the subject given to the post is misleading.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: How to split a file into two file and remove dups?

Postby mainframegeek » Thu Oct 18, 2012 3:07 pm

with
//***********************************************************
//STEP0001 EXEC PGM=ICETOOL                                 
//***********************************************************
//*$            COPY DATASET FROM TAPE TO DISK               
//***********************************************************
//TOOLMSG  DD SYSOUT=*                                       
//DFSMSG   DD SYSOUT=*                                       
//IN01     DD DSN=RT.DTP0107.DISK.RANGE.SORTED.TEST1,DISP=SHR
//IN02     DD DSN=RT.DTP0107.DISK.RANGE.SORTED.TEST1,DISP=SHR
//T1       DD DSN=RT.DTP0107.DISK.RANGE.SORTED.TEST4,       
//             SPACE=(CYL,(300,300)),UNIT=(SYSDA,15),       
//             DISP=(MOD,CATLG,CATLG)                       
//T2       DD DSN=RT.DTP0107.DISK.RANGE.SORTED.TEST5,       
//             SPACE=(CYL,(300,300)),UNIT=(SYSDA,15),       
//             DISP=(MOD,CATLG,CATLG)                       
//T3       DD DSN=RT.DTP0107.DISK.RANGE.SORTED.TEST6,       
//             SPACE=(CYL,(300,300)),UNIT=(SYSDA,15),       
//             DISP=(MOD,CATLG,CATLG)                       
//OUT1     DD DSN=RT.DTP0107.DISK.RANGE.SORTED.TEST7,       
//             DCB=(RECFM=FB,LRECL=8,BLKSIZE=0),             
//             SPACE=(CYL,(300,300)),UNIT=(SYSDA,15),       
//             DISP=(NEW,CATLG,CATLG)                       
//TOOLIN   DD DSN=RT.DTP0107.TEST1(TOOLIN),DISP=SHR         
//CTL1CNTL DD DSN=RT.DTP0107.TEST1(CTL1CNTL),DISP=SHR       
//CTL2CNTL DD DSN=RT.DTP0107.TEST1(CTL2CNTL),DISP=SHR       
//CTL3CNTL DD DSN=RT.DTP0107.TEST1(CTL3CNTL),DISP=SHR
//CTL4CNTL DD DSN=RT.DTP0107.TEST1(CTL4CNTL),DISP=SHR

CTL1CNTL

INREC FIELDS=(01,02,03,03)
SORT FIELDS=(03,03,ZD,A)   
OUTREC FIELDS=(001:001,002,
               003:003,003,
               009:C'11') 

CTL2CNTL

INREC FIELDS=(01,02,06,03)
SORT FIELDS=(03,03,ZD,D)   
OUTREC FIELDS=(001:001,002,
               006:003,003,
               009:C'22') 

CTL3CNTL

SORT FIELDS=(01,02,ZD,A)
SUM FIELDS=NONE

CTL4CNTL

OUTFIL FNAMES=OUT1,INCLUDE=(09,2,CH,EQ,C'11',OR,
                            09,2,CH,EQ,C'22',OR,
                            09,2,CH,EQ,C'12')   

TOOLIN

COPY FROM(IN01) TO(T1) USING(CTL1)             
COPY FROM(IN02) TO(T2) USING(CTL2)             
COPY FROM(T1)   TO(T3) USING(CTL3)             
COPY FROM(T2)   TO(T3) USING(CTL3)             
SPLICE FROM(T3) TO(OUT1) ON(1,2,ZD) -           
   WITH(10,1) WITH(06,3) USING(CTL4) KEEPNODUPS


Input File(IN01 & IN02) (RT.DTP0107.DISK.RANGE.SORTED.TEST1)

11002005
11001004
11003006
12003004
12001002
13004005

T1(RT.DTP0107.DISK.RANGE.SORTED.TEST4)
11002   11
11001   11
11003   11
12003   11
12001   11
13004   11

Value 11 at 9th column is used for indicating that this data comes from IN01.

T2 (RT.DTP0107.DISK.RANGE.SORTED.TEST5)
11   00522
11   00422
11   00622
12   00422
12   00222
13   00522

Value 22 at 9th column is used for indicating that this data comes from IN02.



T3(RT.DTP0107.DISK.RANGE.SORTED.TEST6)
11002   11
11001   11
11003   11
12003   11
12001   11
13004   11
11   00522
11   00422
11   00622
12   00422
12   00222
13   00522


Output OUT1(RT.DTP0107.DISK.RANGE.SORTED.TEST7)

11002006
12003002
13004005

Desired output
11001006
12001004
13004005
mainframegeek
 
Posts: 35
Joined: Sun Sep 16, 2012 1:57 pm
Has thanked: 3 times
Been thanked: 0 time

Re: How to split a file into two file and remove dups?

Postby BillyBoyo » Thu Oct 18, 2012 3:10 pm

And what is the output when you attempt the suggested solution instead of your overly complex approach biased by your misunderstanding of what is necessary?
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: How to split a file into two file and remove dups?

Postby mainframegeek » Thu Oct 18, 2012 9:03 pm

@Billy
I just wanna do it in a single step.
I tried to achieve it using the merge, which was suggested in old post.
But it need two steps of sort, in which merge will be applied with DUPKEYS MAX & DUPLEYS MIN on the fields MAX-NMBR and MIN-NMBR.
Third step will use ICETOOL and give the desired output.
I just wanna reduce the number of steps, can it be acheived with one step.

I tried using DUPKEYS in ICETOOL, but it does not support DUPKEYS. Do we have any keyword in ICETOOL with same kind of functionality as DUPKEYS?
mainframegeek
 
Posts: 35
Joined: Sun Sep 16, 2012 1:57 pm
Has thanked: 3 times
Been thanked: 0 time

Re: How to split a file into two file and remove dups?

Postby BillyBoyo » Thu Oct 18, 2012 9:12 pm

It does not need more than one step. The SORT or the MERGE (with SORTIN01 instead of SORTIN) that was in your previous topic will get you a single record per key, with the minimum value of the first field specified and the maximum value of the second.

However, that will not work if you now want it on DFSORT.

For DFSORT you were shown an example using OUTFIL and SECTIONS. One step. COPY operation.

If you want to insist that you have to split the records up, toss them about a bit, then stick them together, and put it all in one ICETOOL step just because you cannot see what you are really asking for, then you'll have to get it working on your own.

And please, don't now go off and ask in multiple other places.

Is this still just for "research" rather than for a client?
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Next

Return to Syncsort/Synctool

 


  • Related topics
    Replies
    Views
    Last post