Need to remove duplicates



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

Need to remove duplicates

Postby Farhaan4mf » Tue Jul 08, 2014 6:34 pm

Hello,
My input file is UNSORTED (FB, 150)
23.59.41 JOB10161 ---- WEDNESDAY, 02 JUL 2014 ----
00.00.10 JOB10161 ---- THURSDAY,  03 JUL 2014 ----
21.58.36 JOB11825 ---- SATURDAY,  28 JUN 2014 ----
00.46.39 JOB11825 ---- SUNDAY,    29 JUN 2014 ----
23.59.42 JOB23922 ---- MONDAY,    30 JUN 2014 ----
00.00.10 JOB23922 ---- TUESDAY,   01 JUL 2014 ----
03.55.08 JOB25798 ---- TUESDAY,   01 JUL 2014 ----
03.55.08 JOB25798 ---- TUESDAY,   01 JUL 2014 ----
22.16.05 JOB23401 ---- MONDAY,    30 JUN 2014 ----
22.16.05 JOB23401 ---- MONDAY,    30 JUN 2014 ----

I want my output file as below (also in UNSORTED order as in Input file)
23.59.41 JOB10161 ---- WEDNESDAY, 02 JUL 2014 ----
21.58.36 JOB11825 ---- SATURDAY,  28 JUN 2014 ----
23.59.42 JOB23922 ---- MONDAY,    30 JUN 2014 ----
03.55.08 JOB25798 ---- TUESDAY,   01 JUL 2014 ----
03.55.08 JOB25798 ---- TUESDAY,   01 JUL 2014 ----
22.16.05 JOB23401 ---- MONDAY,    30 JUN 2014 ----
22.16.05 JOB23401 ---- MONDAY,    30 JUN 2014 ----

Background for the requirement. I have Job spools in PDS. I need to build a sort JCL that will scan the PDS and collects the batch info. I am able to get the START DATE, START tIME, END TIME, CPU time etc. uniquely. But 'start Day' gives me duplicate entry when a job starts at midnight and runs to next day. I need only one entry for those.
JOB25798 & JOB23401 has duplicate entry but those are not the same job that runs from midnight to next day. So I do not want to remove the duplicates for such scenario. job id starts at col 21 and time starts at col 12. Any help please. I already tried
//TOOLIN    DD *                                         
  SELECT FROM(IN) TO(OUT1) ON(21,8,CH) FIRST USING(CTL1) 
  SELECT FROM(IN) TO(OUT2) ON(21,8,CH) ALLDUPS USING(CTL1)
//CTL1CNTL DD *                                           
  SORT FIELDS=COPY                                       
/*                                                       
Farhaan4mf
 
Posts: 28
Joined: Mon May 13, 2013 4:53 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Need to remove duplicates

 

Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post