SORT VB records from SUPERC to remove headers & drop line#s



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

Re: SORT VB records from SUPERC to remove headers & drop lin

Postby Steve Coalbran » Fri May 09, 2014 3:38 pm

BillyBoyo wrote:...data.

Sorrry Billy - missed your post while I was reading my sudden influx of Notes tripe!
Steve
User avatar
Steve Coalbran
 
Posts: 102
Joined: Wed Apr 06, 2011 11:49 am
Location: Stockholm, Sweden
Has thanked: 5 times
Been thanked: 1 time

Re: Re: SORT VB records from SUPERC to remove headers & drop lin

 

Re: SORT VB records from SUPERC to remove headers & drop lin

Postby Steve Coalbran » Fri May 09, 2014 3:51 pm

Sorry Billy, that just gives me...
ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R12 - 12:12 ON FRI MA
            OPTION COPY                                                         
            OUTFIL FTOV,BUILD=(1,2,BI,TO=ZD,LENGTH=3,X,5,132)                   
ICE251A 9 MISSING RDW OR DATA FOR SORTOUT : REASON CODE 05, IFTHEN 0           

I'm not sure that this is going to work.
I don't want the RDW, record-length or anything else in the output but (in REXX) SUBSTR(recordwithoutRDW,12)
Steve
User avatar
Steve Coalbran
 
Posts: 102
Joined: Wed Apr 06, 2011 11:49 am
Location: Stockholm, Sweden
Has thanked: 5 times
Been thanked: 1 time

Re: SORT VB records from SUPERC to remove headers & drop lin

Postby BillyBoyo » Fri May 09, 2014 4:11 pm

My sorry. I misspelled VTOF... VTOF is Variable To Fixed. FTOV is Fixed To Variable. Since you are starting with variable, and I was trying to show fixed, it should have been VTOF.

I wasn't suggesting it as what you want for your solution, but just so you could see your data with the lenghts as human-readable, so that you know what the blank lines actually look like. I suspect a blank like will show 005 and then a load of blanks, but most of those blanks are SDSF padding, not on the data. Sorry to confuse.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 264 times

Re: SORT VB records from SUPERC to remove headers & drop lin

Postby Steve Coalbran » Fri May 09, 2014 5:58 pm

Thanks Billy,
Ha ha ha! Seems to be going backwards? :lol:
Although it didn't fail!? ;)
 SDSF OUTPUT DISPLAY K248610K JOB63408  DSID   105 LINE 0       C 
 COMMAND INPUT ===>                                            SC 
********************************* TOP OF DATA *******************
124 1  ISRSUPC   -   MVS/PDF FILE/LINE/WORD/BYTE/SFOR COMPARE UTI
081   LINE-Ä  SOURCE SECTION                    SRCH DSN: SYS4.EQQ
005                                                                             
099   467181   05/09 00.00.01 EQQE016I ETT HAS SUCCESSFULLY ADDED
059   467182   05/09 00.00.01 EQQE016I BS40P.OPC.START.MVS0                     
112   467183   05/09 00.00.01 EQQE040W JOB DSPS6571(JOB86033) CANN
112   467184   05/09 00.00.01 EQQE040W JOB DSPC4486(JOB86034) CANN
112   467185   05/09 00.00.01 EQQE040W JOB DSP06566(JOB86035) CANN
...

I was hoping my SYSOUT (for SORTOUT) looked more like...
 SDSF OUTPUT DISPLAY K248610K JOB63408  DSID   105 LINE 0       C
 COMMAND INPUT ===>                                            SC
********************************* TOP OF DATA *******************
05/09 00.00.01 EQQE016I ETT HAS SUCCESSFULLY ADDED ENDEVORPROD5 F
05/09 00.00.01 EQQE016I BS40P.OPC.START.MVS0                     
05/09 00.00.01 EQQE040W JOB DSPS6571(JOB86033) CANNOT BE HANDLED.
05/09 00.00.01 EQQE040W JOB DSPC4486(JOB86034) CANNOT BE HANDLED.
05/09 00.00.01 EQQE040W JOB DSP06566(JOB86035) CANNOT BE HANDLED.
...


DFSORT seems to be very hard at handling null VB records ?
REXX solution is looking good? ;)
Steve
User avatar
Steve Coalbran
 
Posts: 102
Joined: Wed Apr 06, 2011 11:49 am
Location: Stockholm, Sweden
Has thanked: 5 times
Been thanked: 1 time

Re: SORT VB records from SUPERC to remove headers & drop lin

Postby NicC » Fri May 09, 2014 6:19 pm

Hi Steve - this seems to be going nowhere! Can you post the latest SYSIN that you tried and the first few lines ofyour SORTIN in ISPF edit - not SDSF edit - with the third line HX'd (and do not allow edit to pad your short lines!). A 3.2 of the dataset may be useful for confirmation that we are all on the same song sheet!
The problem I have is that people can explain things quickly but I can only comprehend slowly.
Regards
Nic
NicC
Global moderator
 
Posts: 2690
Joined: Sun Jul 04, 2010 12:13 am
Location: Pushing up the daisys (almost)
Has thanked: 4 times
Been thanked: 105 times

Re: SORT VB records from SUPERC to remove headers & drop lin

Postby BillyBoyo » Fri May 09, 2014 6:50 pm

I think I'm just not being clear.

I suggested the short SORT so that we could see what the blank line looks like. And we can now confirm that it has a length of 005.

You seem to be favouring an F output, so we go back to Kolusu's suggestion:

  OPTION COPY,VLSCMP                       
                                           
  OMIT COND=(8,07,CH,EQ,C'ISRSUPC',OR,     
             7,06,CH,EQ,C'LINE-Ä',OR,       
             7,10,CH,EQ,C' ')               
                                           
  INREC OVERLAY=(134:X)                     
  OUTFIL VTOF,BUILD=(16,122)


Knowing that the blank lines are only five bytes long:

  OPTION COPY,VLSCMP                       
                                           
  OMIT COND=(1,2,BI,EQ,5,OR,
             8,07,CH,EQ,C'ISRSUPC',OR,     
             7,06,CH,EQ,C'LINE-Ä',OR,       
             7,10,CH,EQ,C' ')               
                                           
  INREC OVERLAY=(138:X)                     
  OUTFIL VTOF,BUILD=(16,122)


If the 7,10 is solely for the blank lines, that can now be removed, but I'll leave it.

You have 137 bytes, so an adjustment to the OVERLAY, it puts a space at position 138, outside your data, automatically space-padding to that point.

Then the BUILD on OUTFIL can find data, and it will be pre-space-padded rather than binary-zero padded.

The OUTFIL will ignore the RDW (four bytes) and the first 11 of each record, and give you data from position 16 for a length of 122 bytes.

Then there's the stuff at the bottom which we've not seen yet. Just bear in mind that what look like trailing spaces in SDSF will not be there.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 264 times

Re: SORT VB records from SUPERC to remove headers & drop lin

Postby skolusu » Fri May 09, 2014 9:48 pm

Steve Coalbran wrote:DFSORT seems to be very hard at handling null VB records ?


Steve,

Trust me DFSORT is more than capable of handling Null VB records. IMHO , I think we are just trying to tape the holes in the solution that you came up as we really don't know what the data looks like. If your intention is to simply pick the the data from byte 16 , then simply eliminate the records that have length less than that using OMIT condition.

It would be as simple as this
//SYSIN    DD *                                   
  OPTION COPY,VLSCMP                             
                                                 
  OMIT COND=(1,02,BI,LT,16,OR,                   
             8,07,CH,EQ,C'ISRSUPC',OR,           
             7,06,CH,EQ,C'LINE-Ä',OR,             
             7,10,CH,EQ,C' ')                     
                                                 
  INREC BUILD=(1,4,16)                           
//*


Steve Coalbran wrote:REXX solution is looking good? ;)


Try running the rexx exec against a file with more than million records and see how much time it takes.

Btw if you need further help on this issue, please feel free to contact me on sametime or email me directly(available in bluepages) or even use DFSORT hotline dfsort@us.ibm.com
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
skolusu
 
Posts: 586
Joined: Wed Apr 02, 2008 10:38 pm
Has thanked: 0 time
Been thanked: 39 times

Re: SORT VB records from SUPERC to remove headers & drop lin

Postby Steve Coalbran » Sat May 10, 2014 4:57 pm

Thanks,
The format of the data is standard for ISRSUPC except that they have put it to a VB 137 dataset!
An excerpt of the data is included in the JCLstream included below.
Assume NULLS ON meaning that any blank lines are length 0.
EDIT       K248610.USER.JCL(STEFANK) - 01.06               Columns 00001 00072
Command ===>                                                  Scroll ===> CSR 
****** ***************************** Top of Data ******************************
000001 //K248610K JOB (BSJ0,P,B),'STEFANK',CLASS=B,MSGCLASS=O,NOTIFY=&SYSUID   
000002 //*                                                                     
000003 //*--------------------------------------------------------------------
000004 //*        SORT EXTRACT FROM SUPERC                                     
000005 //*--------------------------------------------------------------------
000006 //SORT     EXEC PGM=SORT                                               
000007 //SYSOUT   DD SYSOUT=*                                                 
000008 //SORTIN   DD DISP=SHR,DSN=K248610.MLOG.ERROR.COPY                     
000009 //SORTOUT  DD SYSOUT=* ,OUTLIM=100                                     
000010 //INPUTDS  DD *                                                         
=COLS> ----+----1----+----2----+----3----+----4----+----5----+----6----+----7--
000011  BROWSE    SYS4.EQQ.OPCA.MLOG.ERROR.COPY.G9909V00                       
000012  COMMAND ===>                                                           
000013 ********************************* TOP OF DATA ********                 
000014 1  ISRSUPC   -   MVS/PDF FILE/LINE/WORD/BYTE/SFOR COMP                 
000015   LINE-Ä  SOURCE SECTION                    SRCH DSN:                   
000016                                                                         
=NOTE=            v- from here --->                                           
000017   340948   05/08 00.00.01 EQQM923I JOBLOG FOR BWFXRATE                 
000018   340949   05/08 00.00.01 EQQM923I FOR OPERATION: BWFX                 
000019   340950   05/08 00.00.02 EQQE016I ETT HAS SUCCESSFULL                 
====== -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 37746 Line                 
000020   502872   05/09 07.59.33 EQQE016I BJ80P.STARTA.PXSNIJ                 
000021   502873   05/09 07.59.49 EQQE016I ETT HAS SUCCESSFULL                 
000022   502874   05/09 07.59.49 EQQE016I PDGSX.XTCPNON.SRSTA                 
000023                                                                         
000024 1  ISRSUPC   -   MVS/PDF FILE/LINE/WORD/BYTE/SFOR COMP                 
000025       SEARCH-FOR SUMMARY SECTION            SRCH DSN:                   
000026                                                                         
000027  LINES-FOUND  LINES-PROC  DATASET-W/LNS  DATASET-WO/LN                 
000028      35694       562598            1              0                     
000029                                                                         
000030  PROCESS OPTIONS USED: ANYC MIXED                                       
000031                                                                         
000032  THE FOLLOWING PROCESS STATEMENTS (USING COLUMNS 1:72)                 
000033     SRCHFOR  '05/09'                                                   
000034  ISRS024W EXTRA DATA DETECTED AFTER NORMAL STATEMENT E                 
000035                                                                         
000036  ISRS004I LISTING LINES MAY BE TRUNCATED DUE TO LIMITI                 
000037 //*---- latest try -------------------------------------------------------
000038 //SYSIN    DD *                                       
000039   OPTION COPY                                         
000040   OUTFIL VTOF,BUILD=(1,2,BI,TO=ZD,LENGTH=3,X,5,132)   
000041 //*                                                   
****** **************************** Bottom of Data ****************************

I think it unlikely that this report will hit 1M lines. However (despite being a complite Rexxist) I still think a DFSORT solution would be much faster and more maintainable.
The dsired output is erely all real data lines from pos-16, no line-numbers, headers, trailers, sumaries,... ;) (see the inserted NOTEline?)
Steve
User avatar
Steve Coalbran
 
Posts: 102
Joined: Wed Apr 06, 2011 11:49 am
Location: Stockholm, Sweden
Has thanked: 5 times
Been thanked: 1 time

Re: SORT VB records from SUPERC to remove headers & drop lin

Postby BillyBoyo » Sat May 10, 2014 5:14 pm

Steve, your latest try is still my control cards which were just to show what the actual record-lengths were alongside the data that you see. I think you've again missed out on reading some posts.

I suggest you try some of the more recent examples, which were designed to get what you had thought you had coded for originally.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 264 times

Re: SORT VB records from SUPERC to remove headers & drop lin

Postby Steve Coalbran » Sat May 10, 2014 6:16 pm

OK I just wanted a working solution I guess. Like te REXX one I supplied.
None of the answers I have received have taken away the blank lines and I haven't even begun to address dropping the footers yet via DFSORT.
Steve
User avatar
Steve Coalbran
 
Posts: 102
Joined: Wed Apr 06, 2011 11:49 am
Location: Stockholm, Sweden
Has thanked: 5 times
Been thanked: 1 time

PreviousNext

Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post