How to split a file into two file and remove dups?



Support for NetApp SyncSort for z/OS, Visual SyncSort, SYNCINIT, SYNCLIST and SYNCTOOL

Re: How to split a file into two file and remove dups?

Postby mainframegeek » Thu Oct 18, 2012 9:32 pm

I didn't get you.If possible, can u explain it.
There is no need of splitting. I split the records because at a time i can arrange either MAX-NMBR in descending order or MIN-NMBR in ascending order. Thats why i used two steps of sort.
I tried using OUTFIL option so that single step will achieve two things, but OUTFIL does not support DUPKEYS keyword.
In that old post, 3nadh suggested the merge solution, but with that at time only one field will be arranged in either max or min order.

Exactly its not a client requirement, I am doing this to reduce the CPU time used by a JOB.
mainframegeek
 
Posts: 35
Joined: Sun Sep 16, 2012 1:57 pm
Has thanked: 3 times
Been thanked: 0 time

Re: How to split a file into two file and remove dups?

Postby BillyBoyo » Thu Oct 18, 2012 9:43 pm

3nadh suggested a SORT solution. I wondered about the possibility of MERGE, as you file was in order already.

Are you using SyncSort now or DFSORT?

If SyncSort, try specifying both MIN and MAX on different fields of the DUPKEYS.

If DFSORT, forget DUPKEYS, and use OUTFIL with SECTIONS which have their own MIN and MAX. You were given that code twice elsewhere.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: How to split a file into two file and remove dups?

Postby mainframegeek » Thu Oct 18, 2012 10:01 pm

I m using SYNCSORT. The file could be in any order.
I tried the below mentioned code and got the error message. Thats why i used two sort steps.
OPTION EQUALS                     
    MERGE FIELDS=(1,2,ZD,A)           
    DUPKEYS MAX=(06,03,ZD)
    DUPKEYS MIN=(03,03,ZD)
mainframegeek
 
Posts: 35
Joined: Sun Sep 16, 2012 1:57 pm
Has thanked: 3 times
Been thanked: 0 time

Re: How to split a file into two file and remove dups?

Postby BillyBoyo » Thu Oct 18, 2012 10:17 pm

Can you try DUPKEYS MIN=(03,03,ZD),MAX=(06,03,ZD)

If you get an error message, post the sysout from the step.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: How to split a file into two file and remove dups?

Postby mainframegeek » Mon Oct 22, 2012 12:55 am

Ya it worked.
Thanks Billy.
Don't need to put comma. Correct code is mentioned below.
OPTION EQUALS                     
    MERGE FIELDS=(1,2,ZD,A)           
    DUPKEYS MAX=(06,03,ZD)MIN=(03,03,ZD)


Now one more thing i need to do.

There is one other file which has the layout mentioned below.

01 FILE-REC
    05 LIC-NMBR PIC 9(2)
    05 nmbr PIC 9(03)


The file which we created from the above mentioned merge step has three things LIC-NMBR, MIN-NMBR and MAX-NMBR.
These two files has to be compared.


From the Merge step we got the output(mentioned below).
File-1
11001005
12000007
13000001

The second file has the data
File-2
11003
11002
11004
12001
12002
13001

it could be in ascending or descending order.
We have to compare these two files on the basis of Field NMBR, this field should be in between MIN-NMBR & MAX-NMBR of file 1 for the particular LIC.If it doennot belong to then it has to be written in output file.
For example
11001005
In this NMBR is 005.
In file-2
11003
11002
11004
For the LIC (11) Min-NMBR is 001 and MAX-NMBR IS 005. So above given number (003,002,004) are in between 001 & 005.They will not be written to the file. But if there would have been one more record like 11007 then it has to be written to the error file as 11007007.
How could it be achieved??
mainframegeek
 
Posts: 35
Joined: Sun Sep 16, 2012 1:57 pm
Has thanked: 3 times
Been thanked: 0 time

Re: How to split a file into two file and remove dups?

Postby BillyBoyo » Mon Oct 22, 2012 4:10 am

If you use JOINKEYS on the LIC, then for matched records:

  REFORMAT FIELDS=(F1:1,8,F2:1,5)


You will have each of multiple key on File 2 matched to the single record on File 1.

You can then use OUTFIL OMIT/INCLUDE to output 9,5 from the REFORMAT record, if number 11,3 is less than 3,3 or greater than 6,3.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: How to split a file into two file and remove dups?

Postby mainframegeek » Thu Oct 25, 2012 10:52 pm

Getting some error in joinkey statement....
mainframegeek
 
Posts: 35
Joined: Sun Sep 16, 2012 1:57 pm
Has thanked: 3 times
Been thanked: 0 time

Re: How to split a file into two file and remove dups?

Postby dick scherrer » Thu Oct 25, 2012 11:15 pm

Hello,

And you intend to keep this all to yourself . . . :?
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6268
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 93 times

Re: How to split a file into two file and remove dups?

Postby mainframegeek » Fri Oct 26, 2012 8:30 pm

I was little bit busy.
There is the complete code.Its working now.
JOINKEYS FILE=F1,FIELDS=(1,2,A),SORTED           
JOINKEYS FILE=F2,FIELDS=(1,2,A),SORTED           
REFORMAT FIELDS=(F1:1,2,F2:3,3,F2:6,3,F1:3,3)
SORT FIELDS=COPY                                 
OUTFIL FNAMES=OUT,BUILD=(1,11)                   


Now i need one include statement. So that i can compare the number with maximum and minimum number.
I will update soon.
mainframegeek
 
Posts: 35
Joined: Sun Sep 16, 2012 1:57 pm
Has thanked: 3 times
Been thanked: 0 time

Re: How to split a file into two file and remove dups?

Postby mainframegeek » Fri Oct 26, 2012 9:55 pm

I included omit condition instead of include.
JOINKEYS FILE=F1,FIELDS=(1,2,A),SORTED           
JOINKEYS FILE=F2,FIELDS=(1,2,A),SORTED           
REFORMAT FIELDS=(F1:1,2,F2:3,3,F2:6,3,F1:3,3)
SORT FIELDS=COPY                                 
OUTFIL FNAMES=OUT,OMIT=(9,3,ZD,LT,06,03,ZD,AND,
                           9,3,ZD,GT,3,3,ZD),
                                                 
                  OUTREC=(1:1,2,                 
                          03:09,03,             
                          06:09,03,             
                          9:X)                 



I want to reduce the length of Output dataset to 8, now its 11.
How could it be achieved? Any Suggestions?
mainframegeek
 
Posts: 35
Joined: Sun Sep 16, 2012 1:57 pm
Has thanked: 3 times
Been thanked: 0 time

PreviousNext

Return to Syncsort/Synctool

 


  • Related topics
    Replies
    Views
    Last post