Need to add new character in O/P when input key gets change



Support for NetApp SyncSort for z/OS, Visual SyncSort, SYNCINIT, SYNCLIST and SYNCTOOL

Need to add new character in O/P when input key gets change

Postby Hariprasad K » Tue Sep 30, 2014 1:17 am

Hi,

I would like to add new character (any alphabet or Symbol) in output file when ever my input key is getting change, to identify that new record is coming for processing.

Input file layout as below: (first three bytes (01 - 03) sequence number, 4th position is space and next two bytes has Line number).
111 01
111 01
111 02
111 02
111 02
222 01
222 02
222 03
222 03

Output file should like below:
111 01
111 01 #
111 02
111 02
111 02 #
222 01 #
222 02 #
222 03
222 03 #

I want to use 'Sequence number & Line number' as a key and need to add special character at the end of the input record which has the same key.

Please drop me a note, if you are not able to understand my requirement. Thanks in Advance!

Regards,
Hariprasad.K
Hariprasad K
 
Posts: 19
Joined: Sat Feb 25, 2012 5:17 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Need to add new character in O/P when input key gets change

 

Re: Need to add new character in O/P when input key gets cha

Postby BillyBoyo » Wed Oct 01, 2014 5:56 pm

If you have a look at this one, and follow the links, http://ibmmainframes.com/about63205.html you should get to an example you can use.

They are "complex" solutions, because it is not an easy thing to do in SORT.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 264 times

Re: Need to add new character in O/P when input key gets cha

Postby Hariprasad K » Wed Oct 01, 2014 8:50 pm

Thanks Again Billy,
I tried like below and got required output:

//ICETOOL1 EXEC PGM=ICETOOL                                         
//IN       DD DSN=XXXX.XXXXXX.INPUT.FILE,DISP=SHR                     
//*                                                                 
//TOOLMSG  DD SYSOUT=*                                               
//DFSMSG   DD SYSOUT=*                                               
//SYSUDUMP DD SYSOUT=*                                               
//*                                                                 
//OUT      DD DSN=XXXX.XXXXXX.OUTPUT.FILE,                           
//            DISP=(NEW,CATLG,DELETE),                               
//            UNIT=DISK,SPACE=(CYL,(20,10),RLSE),                   
//            DCB=(*.IN)                                             
//*                                                                 
//SORTXSUM DD DSN=XXXX.XXXXXX.OUTPUT.DUPL,                           
//            DISP=(NEW,CATLG,DELETE),                               
//            UNIT=DISK,SPACE=(CYL,(20,10),RLSE),                   
//            DCB=(*.IN)                                             
//*                                                                 
//OUTFMT   DD DSN=XXXX.XXXXXX.OUTPUT.OUTFMT,                         
//            DISP=(NEW,CATLG,DELETE),                               
//            UNIT=DISK,SPACE=(CYL,(20,10),RLSE),                   
//            DCB=(*.IN)                                             
//TOOLIN   DD *                                                     
    SELECT FROM(IN) TO(OUT) ON(01,03,CH) -                           
                              ON(05,02,CH) LAST DISCARD(SORTXSUM)   
    COPY FROM(OUT) TO(OUTFMT) USING(CTL1)                           
/*                                                                   
//CTL1CNTL DD *                                                     
   INREC OVERLAY(13:C'Y')                                           
/*                                                                   
//SORT     EXEC PGM=SORT                               
//*                                                     
//SORTIN   DD DSN=XXXX.XXXXXX.OUTPUT.DUPL,DISP=SHR     
//         DD DSN=XXXX.XXXXXX.OUTPUT.OUTFMT,DISP=SHR   
//*                                                     
//SORTOUT  DD DSN=XXXX.XXXXXX.OUTPUT.FINAL,             
//            DISP=(NEW,CATLG,DELETE),                 
//            UNIT=DISK,SPACE=(CYL,(20,10),RLSE),       
//            DCB=(*.SORTIN)                           
//*                                                     
//SYSPRINT DD SYSOUT=*                                 
//SYSOUT   DD SYSOUT=*                                 
//*                                                     
//SYSIN    DD *                                         
   SORT FIELDS=(01,03,CH,A,05,02,CH,A,13,01,CH,A)       
//*


Code'd
Hariprasad K
 
Posts: 19
Joined: Sat Feb 25, 2012 5:17 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Need to add new character in O/P when input key gets cha

Postby BillyBoyo » Wed Oct 01, 2014 9:15 pm

OK, you're getting solutions, which is good.

Once you do that, take a little break from that particular thing (do something else, have a cup of tea and a biscuit, whatever).

You'll then have created some distance from the solution, so you can take another look at it.

Here, you've got two steps, the first with two ICETOOL operators. Can it all be done in one step? Yes. Does it require all that processing of the data?

A good rule of thumb is that if you have to read the same data more than twice, three times at a stretch, then you'd be better off doing it in another language.

Most things can be done in one step, without that step using multiple ICETOOL operators.

What I'd suggest you look at is the example of JOINKEYS which has the same DSN specified for both inputs, and which sets a sequence number in the JNFnCNTL files, the second sequence number starting from zero. Then JOINKEYS uses the sequence numbers as the keys and you get "current and next" data on the REFORMAT record. Then it's a simple test to know when to set your value.

There are a couple of examples of that solution in the links (or the links from the links). If you can't identify it, let us know.

I'm sure you can get that working, and then compare to what you have so far.

The best way to learn all this is to do it. Getting solutions is the first thing. Improving is the next.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 264 times

Re: Need to add new character in O/P when input key gets cha

Postby Hariprasad K » Wed Oct 01, 2014 9:19 pm

Good to hear your words, will try definitely and let you know. Thanks !
Hariprasad K
 
Posts: 19
Joined: Sat Feb 25, 2012 5:17 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Need to add new character in O/P when input key gets cha

Postby Hariprasad K » Thu Oct 02, 2014 7:04 pm

Hi Billy,

I have tried as you suggested in previous post and got expected results
Input data:
000001
000001
000002
000002
000002
000002
000020
000030
000030


I am expecting out like below:
000001   
000001   
000001   Y
000002   
000002   
000002   
000002   Y
000020   Y
000030   
000030   Y

(for my testing, I considered first 6 byes as my key)

Used below sort job to achieve my output:
//SORT001  EXEC PGM=SORT                                               
//*                                                                   
//SYSOUT   DD SYSOUT=*                                                 
//SYSPRINT DD SYSOUT=*                                                 
//*                                                                   
//T1       DD DSN=XXXXXX.XXXXXXX.XXXXX.INPUT,DISP=SHR                 
//*                                                                   
//T2       DD DSN=XXXXXX.XXXXXXX.XXXXX.INPUT,DISP=SHR                 
//*                                                                   
//SORTOUT  DD DSN=XXXXXX.XXXXXXX.XXXXX.OUTPUT,                         
//            DISP=(NEW,CATLG,DELETE),                                 
//            UNIT=DISK,SPACE=(CYL,(20,10),RLSE),                     
//            DCB=(*.T1)                                               
//*                                                                   
//SYSIN DD *                                                           
  JOINKEYS F1=T1,FIELDS=(8,6,A),SORTED,NOSEQCK                         
  JOINKEYS F2=T2,FIELDS=(8,6,A),SORTED,NOSEQCK                         
  JOIN UNPAIRED,F1,F2                                                 
  REFORMAT FIELDS=(F1:1,6,F2:1,6)                                     
  OPTION COPY                                                         
  OUTREC IFTHEN=(WHEN=((1,6,CH,NE,7,6,CH),AND,7,6,CH,NE,C' '),         
                 OVERLAY=(1:7,6,7:3C' ',10:C'Y',11:10C' ')),           
         IFTHEN=(WHEN=(1,6,CH,EQ,7,6,CH),                             
                 OVERLAY=(1:7,6,7:10C' '))                             
//JNF1CNTL DD *                                                       
  OPTION COPY                                                         
  INREC OVERLAY=(8:SEQNUM,6,ZD,START=0)                               
//JNF2CNTL DD *                                                       
  OPTION COPY                                                         
  INREC OVERLAY=(8:SEQNUM,6,ZD,START=1)


Please clarify : Using ICETOOL (my previous post), I processed my input file 3 times and used more than two output files to hold data to get desired output, but using Joinkeys I processed only one time without any temp datasets- so will it reduce CPU performance?

Code'd
Hariprasad K
 
Posts: 19
Joined: Sat Feb 25, 2012 5:17 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Need to add new character in O/P when input key gets cha

Postby Terry Heinze » Thu Oct 02, 2014 7:12 pm

Hariprasad K,
Please use the Code tags for better readability.
.... Terry
Terry Heinze
 
Posts: 193
Joined: Wed Dec 04, 2013 11:08 pm
Location: Richfield, MN, USA
Has thanked: 10 times
Been thanked: 11 times

Re: Need to add new character in O/P when input key gets cha

Postby BillyBoyo » Thu Oct 02, 2014 7:21 pm

With the JOINKEYS you read the same file twice, once on each JOINKEYS statement. In your ICETOOL version you also did a couple of SORTs (have a look at the sysouts).

You can look at the stats from both jobs. You'll have less IO, less CPU. How much you notice depends on the size of your file.

It is important you understand how it works. You'll realise that although there is no way provided to automatically identify the "last" of a key, it is quite simple once you know how.

To use the Code tags, highlight a block and click on Code button. User the Preview button to see what your post will look like.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 264 times

Re: Need to add new character in O/P when input key gets cha

Postby Hariprasad K » Thu Oct 02, 2014 7:23 pm

//SORT001  EXEC PGM=SORT                                               
//*                                                                     
//SYSOUT   DD SYSOUT=*                                                 
//SYSPRINT DD SYSOUT=*                                                 
//*                                                                     
//T1       DD DSN=XXXXXX.XXXXXXX.XXXXX.INPUT,DISP=SHR                   
//*                                                                     
//T2       DD DSN=XXXXXX.XXXXXXX.XXXXX.INPUT,DISP=SHR                   
//*                                                                     
//SORTOUT  DD DSN=XXXXXX.XXXXXXX.XXXXX.OUTPUT,                         
//            DISP=(NEW,CATLG,DELETE),                                 
//            UNIT=DISK,SPACE=(CYL,(20,10),RLSE),                       
//            DCB=(*.T1)                                               
//*                                                                     
//SYSIN DD *                                                           
  JOINKEYS F1=T1,FIELDS=(8,6,A),SORTED,NOSEQCK                         
  JOINKEYS F2=T2,FIELDS=(8,6,A),SORTED,NOSEQCK                         
  JOIN UNPAIRED,F1,F2                                                   
  REFORMAT FIELDS=(F1:1,6,F2:1,6)                                       
  OPTION COPY                                                           
  OUTREC IFTHEN=(WHEN=((1,6,CH,NE,7,6,CH),AND,7,6,CH,NE,C' '),         
                 OVERLAY=(1:7,6,7:3C' ',10:C'Y',11:10C' ')),           
         IFTHEN=(WHEN=(1,6,CH,EQ,7,6,CH),                               
                 OVERLAY=(1:7,6,7:10C' '))                             
//JNF1CNTL DD *                                                         
  OPTION COPY                                                           
  INREC OVERLAY=(8:SEQNUM,6,ZD,START=0)                                 
//JNF2CNTL DD *                                                         
  OPTION COPY                                                           
  INREC OVERLAY=(8:SEQNUM,6,ZD,START=1)                                 
Hariprasad K
 
Posts: 19
Joined: Sat Feb 25, 2012 5:17 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Need to add new character in O/P when input key gets cha

Postby Hariprasad K » Thu Oct 02, 2014 7:26 pm

Thanks Billy, I will use code tag here onwards. :)
Hariprasad K
 
Posts: 19
Joined: Sat Feb 25, 2012 5:17 pm
Has thanked: 0 time
Been thanked: 0 time

Next

Return to Syncsort/Synctool

 


  • Related topics
    Replies
    Views
    Last post