Remove set of duplicates from input file



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

Remove set of duplicates from input file

Postby radhika_engimuri » Thu Jan 08, 2009 6:36 pm

Hi All,

I want to remove set of duplicates(not similar rows) from my input file .
Ex:
1st postion : header or trailer or remark
2 to 6 : primary key field

1st set of records

012345AAA --- > The record with '0' at 1st position is HEADER
112345AAA ---- > All the records with '1' AT 1ST position are DETAILS
112345AAA
212345AAA --- > All the records with '2' at 1st are REMARKS

2nd set of records

023456BBB
223456BBB
223456BBB
123456BBB
223456BBB

3rd set of records

012345CCC
212345CCC
112345CCC
212345CCC
212345CCC


Remarks can be present for header or detail or for both.

Now, I want to delete either 1st or 3rd set of records irrespective of how many detail or remark records present.

Required Output

012345AAA
112345AAA
112345AAA
212345AAA


023456BBB
223456BBB
223456BBB
123456BBB
223456BBB

OR
023456BBB
223456BBB
223456BBB
123456BBB
223456BBB

012345CCC
212345CCC
112345CCC
212345CCC
212345CCC

I can write program to acheive this but I want to kown easier way to do the same using DFSORT.
radhika_engimuri
 
Posts: 7
Joined: Thu Jan 08, 2009 6:06 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Remove set of duplicates from input file

Postby Frank Yaeger » Thu Jan 08, 2009 10:37 pm

You can use a DFSORT job like the following to remove a given group of records (e.g. third group). You'll need z/OS DFSORT PTF UK90013 (July, 2008) to use DFSORT's new WHEN=GROUP function. If you don't have this PTF, ask your System Programmer to install it.

I assumed your input file has RECFM=FB and LRECL=80, but the job can be changed appropriately for other attributes. I used 81,8,ZD,EQ,3 to remove the third group. If you want to remove another group, change 3 to the correct number (e.g. 1 for the first group).

//S1    EXEC  PGM=ICEMAN
//SYSOUT    DD  SYSOUT=*
//SORTIN DD *
012345AAA
112345AAA
112345AAA
212345AAA
023456BBB
223456BBB
223456BBB
123456BBB
223456BBB
012345CCC
212345CCC
112345CCC
212345CCC
212345CCC
/*
//SORTOUT DD SYSOUT=*
//SYSIN    DD    *
  OPTION COPY
  INREC IFTHEN=(WHEN=GROUP,BEGIN=(1,1,CH,EQ,C'0'),
    PUSH=(81:ID=8))
  OUTFIL OMIT=(81,8,ZD,EQ,3),BUILD=(1,80)
/*


For complete details on DFSORT's new WHEN=GROUP function and the other new functions available with PTF UK90013, see:

http://www.ibm.com/systems/support/stor ... /mvs/ugpf/
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1079
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 15 times


Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post