Page 1 of 2

Eleminating Duplicates using SELECT and LASTDUP.

PostPosted: Wed May 14, 2014 2:20 am
by Viswanathchandru
Hi All,

I have to remove duplicates from a flat file. I tried using SELECT (DFSORT) along with LASTDUP it removed the duplicates as expected. The sort card was like this.

SELECT FROM(IN) TO(OUT) ON(32,8,CH) LASTDUP


But, the problem is, there were few records like this which had duplicates.

 
    MAILABQ
    MAILTRN
    MAILLOM
    MAIEDD
    MAILWER


For the above record, it has removed all the data and it has given only MAILWER. Is there a way I can get the proper record.

My record length is 800.

Thanks,
Viswa..

Re: Eleminating Duplicates using SELECT and LASTDUP.

PostPosted: Wed May 14, 2014 2:31 am
by enrico-sorichetti
hard to believe, that any SORT flavor would drop on its own part of a record :shock:

Re: Eleminating Duplicates using SELECT and LASTDUP.

PostPosted: Wed May 14, 2014 2:53 am
by dick scherrer
Hello,

What do the informational messages show?

It may help if you post the complete JCL and control statements.

Re: Eleminating Duplicates using SELECT and LASTDUP.

PostPosted: Wed May 14, 2014 2:58 am
by Viswanathchandru
Hello Mr. Scherrer,

I'm not getting any error messages and the JCL goes with RC 0. Here is the JCL..

//S1 EXEC PGM=ICETOOL                       
//TOOLMSG DD SYSOUT=*                       
//DFSMSG DD SYSOUT=*                         
//IN DD DSN=VENUS3V.IN.TEXT2,DISP=SHR     
//OUT DD DSN=VENUS3V.IN.TEXT2,DISP=SHR   
//TOOLIN DD *                               
 SELECT FROM(IN) TO(OUT) ON(32,8,CH) LASTDUP


Hello Mr. Enrico,

I don't think sort is doing it wrong ! I'm 90% sure I'm going somewhere wrong ! so, thought of checking with experts..

Thanks,
Viswa..

Re: Eleminating Duplicates using SELECT and LASTDUP.

PostPosted: Wed May 14, 2014 3:01 am
by NicC
definitely the control statements and the sysout. Sample input/output would be good.

What the heck is
SELECT(DFSORT)
?

Re: Eleminating Duplicates using SELECT and LASTDUP.

PostPosted: Wed May 14, 2014 3:02 am
by dick scherrer
Hello,

"Some" informational messages Are generated, unless they have been discarded somehow.

Re: Eleminating Duplicates using SELECT and LASTDUP.

PostPosted: Wed May 14, 2014 3:03 am
by dick scherrer
Hi Nic,

TS is just saying SELECT was used, I believe.

Re: Eleminating Duplicates using SELECT and LASTDUP.

PostPosted: Wed May 14, 2014 3:09 am
by NicC
Then why mention DFSORT when we are in the Syncsort section of the forum? The (conveniently forgotten/not found) messages would have told us.

Re: Eleminating Duplicates using SELECT and LASTDUP.

PostPosted: Wed May 14, 2014 3:19 am
by Viswanathchandru
Hi ,

My apologies, I didn't notice a seperate section for DFSORT. If someone could please move this topic to DFSORT it will be helpful !

Thanks,
Viswa..

Re: Eleminating Duplicates using SELECT and LASTDUP.

PostPosted: Wed May 14, 2014 3:20 am
by enrico-sorichetti
//IN DD DSN=VENUS3V.IN.TEXT2,DISP=SHR     
//OUT DD DSN=VENUS3V.IN.TEXT2,DISP=SHR   


using as output the same dataset used for input is looking for troubles
usually unpredictable behavior

try again using the proper approach ( a different output dataset )