discarding duplicates using condition in SORT



Support for NetApp SyncSort for z/OS, Visual SyncSort, SYNCINIT, SYNCLIST and SYNCTOOL

discarding duplicates using condition in SORT

Postby parthiban_82 » Thu Feb 22, 2018 3:02 pm

I have 2 files which I need to join and get the union of both after removing duplicates. But I need to remove the duplicates based on a 3rd file. If the duplicate record is present in 3rd file then I need to pick the duplicate record from 1st file else I need to pick the record from 2nd file.

Sample

1st file :
account1record from file1
account2record from file1
account3record from file1

2nd file :
account1record from file2
account2record from file2
account4record from file2

3rd file
account1record from file3

Expected result
account1record from file1 ------ since this is present in 3rd file
account2record from file2 ------ since this is not present in 3rd file
account3record from file1
account4record from file2
parthiban_82
 
Posts: 6
Joined: Thu Feb 22, 2018 2:12 pm
Has thanked: 0 time
Been thanked: 0 time

Re: discarding duplicates using condition in SORT

Postby NicC » Thu Feb 22, 2018 3:50 pm

They are not 'files' but data sets. You would need two steps - step one matches data set one and data set 3 dropping the duplicates and creataing a 4th data set. Tis would then be merged with data set 2 to create data set 5.
Or you could write a program to do a 3 data set match.
The problem I have is that people can explain things quickly but I can only comprehend slowly.
Regards
Nic
NicC
Global moderator
 
Posts: 3025
Joined: Sun Jul 04, 2010 12:13 am
Location: Pushing up the daisies (almost)
Has thanked: 4 times
Been thanked: 136 times

Re: discarding duplicates using condition in SORT

Postby parthiban_82 » Thu Feb 22, 2018 6:54 pm

Hi Nic .. Thanks for the reply .. But if my 3rd dataset has other accounts it should not be part of my output. The 3rd dataset is just a reference to check whether the account in the 1st dataset is present or not. Please correct me if I am missing something.


Sample

1st file :
account1record from file1
account2record from file1
account3record from file1

2nd file :
account1record from file2
account2record from file2
account4record from file2

3rd file
account1record from file3
account5record from file3

Expected result
account1record from file1 ------ since this is present in 3rd file
account2record from file2 ------ since this is not present in 3rd file
account3record from file1
account4record from file2
parthiban_82
 
Posts: 6
Joined: Thu Feb 22, 2018 2:12 pm
Has thanked: 0 time
Been thanked: 0 time


Return to Syncsort/Synctool

 


  • Related topics
    Replies
    Views
    Last post