How to add characters to a variable length record



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

Re: How to add characters to a variable length record

Postby fatmonster » Tue Mar 18, 2014 1:18 pm

I am out of office and cannot provide the details of sort step. I remember it is V1R12 and PTF UK90025 and UK90026 should be installed (I remember ICETOOL resize function works properly).

How many records? [no limit]
All text? [no, records might contain packed decimal]
Can a record ever be "full" with data - what to do then? [no]

Thanks for your help in advance. I am really want to know how to make it, either by new functions or other method. Or could you provide some hints and I try to work out the ans by myself first?
fatmonster
 
Posts: 9
Joined: Tue Mar 18, 2014 6:46 am
Has thanked: 2 times
Been thanked: 0 time

Re: How to add characters to a variable length record

Postby fatmonster » Tue Mar 18, 2014 1:33 pm

I found the new function in V2R1, thanks BillyBoyo!!

But how about other methods?

"Insert a string at the end of variable-length records
VLTRAIL=string is a new OUTFIL option that allows you to insert a character
string (C'string') or hexadecimal string (X'yy...yy') at the end of each variable-length
OUTFIL output record."
fatmonster
 
Posts: 9
Joined: Tue Mar 18, 2014 6:46 am
Has thanked: 2 times
Been thanked: 0 time

Re: How to add characters to a variable length record

Postby BillyBoyo » Tue Mar 18, 2014 3:06 pm

Good that you are willing. Best way to pick something up.

So, first look at the V2.1 manuals for DFSORT and find out how to do it with the new function (there is always a "what's new in this release" section in manuals, start there).

A fun one for small numbers of short records is to make the records fixed in length by inserting the X'FF' at the end of the record (256:X'FF' for instance). Then do JFY to the right for 5,255 and to the left for 5,256. Then the VLTRIM.

It seems though that you are looking for something general. What is your purpose?
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: How to add characters to a variable length record

Postby fatmonster » Tue Mar 18, 2014 3:25 pm

Thanks a lot. However, I do find this solution is not perfert if the last character of the record is space. Using VLTRIM will shorten the record in this case. Any idea?
fatmonster
 
Posts: 9
Joined: Tue Mar 18, 2014 6:46 am
Has thanked: 2 times
Been thanked: 0 time

Re: How to add characters to a variable length record

Postby BillyBoyo » Tue Mar 18, 2014 3:35 pm

Yes, certainly that will happen. Leading spaces in the record won't help when it jumps off to the left, either.

Without V2.1 you are dependant on finding a solution for the specifics of your data in each case. You seem to want a general solution. Usually textual data does not require the retention of trailing space on a record.

If you have fixed-length fields with binary and packed-decimal data, then it seems to be OCCURS you are trying to deal with. OK, so what is the point of the X'FF'? What are you trying to use that for? Since we will have to attempt to supply functionality by applying existing functionality, we need to know the limits of the data to know what is acceptable, and what is not.

Hit the manuals and try to find how to replace the "padding" character which is used. You can then pad to other than space. But, you need a value which cannot be the value of the last byte of the data (the space problem is just a specific example of this).
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: How to add characters to a variable length record

Postby fatmonster » Tue Mar 18, 2014 3:56 pm

I am thinking of developing a function to pad a line fee/carriage return at the end of each record and download to FTP server thru binary mode. Since this sort application will be used by different ppl and different purpose, limitation applying to the last char may not be appropriate. Today, I was thinking about using right justify with TRAIL="X", however, when i try to do this:

OUTREC FIELDS=(5,252,JFY=(SHIFT=LEFT,TRAIL=X'FF'))

It will prompt error as the record is too short. I tried to use VLSHRT option with no success :(
BTW, it is a good learning experience for that. Thanks for your help.
fatmonster
 
Posts: 9
Joined: Tue Mar 18, 2014 6:46 am
Has thanked: 2 times
Been thanked: 0 time

Re: How to add characters to a variable length record

Postby BillyBoyo » Tue Mar 18, 2014 4:44 pm

BUILD is the same as FIELDS. Identical. Just an Alias. FIELDS exists on INREC and OUTREC (and OUTREC exists on OUTFIL) for backwards-compatibility. Best to use BUILD on INREC/OUTREC/OUTFIL rather than FIELDS/FIELDS/OUTREC.

I know the examples in the manual are often with FIELDS, so I'm explaining.

I suspected it was going to be something like that. However...

If the receiver of the data is another Mainframe, no cuteness is needed.

If the receiver of the data is non-Mainframe, then by not giving everything in character format you are giving them extra work to do, and introducing a possible fault-point, that will get you big red crosses from any type of Audit. Why...?

If you send non-character data, you have to send as binary anyway (since the character translations are not at field-level, part of a binary/packed value can "look like" an EBCDIC character, so get translated to the equvalent ASCII character).

You receiver then has the task of taking the binary file (having avoided the above problem) and messing with the data so that it is in a format which their own systems can understand. You have "endianness". You have implications from NUMPROC compiler option. You have the fact that the languages they use may not understand "packed-decimal" (or worse, may understand it in a different way).

All of which may lead to un-audited screwing-about with your data.

On the other hand, if you give out solely character data, specific signs, actual decimals or scaling factor, then you can do all the necessary work in FTP/NDM/whatever and they just have to get on with verifying a complete file of the correct date and correct system where your stuff arrives.

It is an interesting exercise anyway.

You can't use a variable-length field for JFY (5,JFY...) and as you've discovered, you can't use bytes which don't have data in. You can fix the length of your data in two general ways: padding and PARSEing. Having fixed your data with regard to length, you need a sure way of returning it to variable, without loosing any data, so read the previous post.

With things like VLSHRT and VLSCMP, the documentation tells you exactly how and where your can use them. You can't use them in any other ways.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: How to add characters to a variable length record

Postby fatmonster » Tue Mar 18, 2014 5:08 pm

On the way home now and think about if the below solution is possible or not

1. Hex the vb record and convert it to a fixed len record
2. pad the end mark and squeeze the record
3. UNHEX and VTRIM (not sure if i can preserve the actual rec len in this step)
fatmonster
 
Posts: 9
Joined: Tue Mar 18, 2014 6:46 am
Has thanked: 2 times
Been thanked: 0 time

Re: How to add characters to a variable length record

Postby BillyBoyo » Tue Mar 18, 2014 9:08 pm

I think if you are desperate to have an end-of-record delimiter for a generalised variable-length-record file, an E35 exit (can be written in COBOL, very simple) is perhaps the way.

Your HEX idea will work, I think: On OUTFIL BUILD with 1,4,5,TRAN=HEX, OVERLAY the ((maximum length * 2) - 1) with two blanks, FINDREP to change two blanks to C'FF' (DO it only once) then BUILD as before but with UNHEX, and ensure you have VLTRIM.

However, you've just halved the maximum-length record it can deal with, you need two free bytes per LRECL, and I have no clue about how it will be perform with large-and-lots records.

Set against these things, is just to put everything into character format (including signs and decimal places (or indicate the factor for a field) and then just transfer, let the transfer handle EBCDIC to ASCII and the CR/CR=LF things.

These users thanked the author BillyBoyo for the post:
fatmonster (Tue Mar 18, 2014 10:24 pm)
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: How to add characters to a variable length record

Postby skolusu » Tue Mar 18, 2014 10:06 pm

fatmonster wrote:I found the new function in V2R1, thanks BillyBoyo!!

But how about other methods?

"Insert a string at the end of variable-length records
VLTRAIL=string is a new OUTFIL option that allows you to insert a character
string (C'string') or hexadecimal string (X'yy...yy') at the end of each variable-length
OUTFIL output record."



Fatmonster,

If you have V2R1 then it is a simple 1 keyword function. If you do not have DFSORT V2R1, then you can still do it but it requires as little bit of work depending on how your input data is.

you can use the following control cards

//SYSIN    DD *                                       
  OPTION COPY                                         
  INREC OVERLAY=(256:X)                               
  OUTFIL VLTRIM=C' ',                                 
  BUILD=(1,4,5,252,JFY=(SHIFT=LEFT,TRAIL=X'FF'))       
//*


Alternatively You can write an exit or generate dynamic sysin control cards checking the rdw for its length.

Here is a sample JCL to generate the sysin control cards.
//STEP0100 EXEC PGM=SORT                                           
//SYSOUT   DD SYSOUT=*                                             
//SORTIN   DD *                                                   
                                                                   
//SORTOUT  DD SYSOUT=*       
//SYSIN    DD *                                                   
  OPTION COPY                                                     
  OUTFIL REPEAT=252,IFOUTLEN=80,                                   
  IFTHEN=(WHEN=INIT,OVERLAY=(81:SEQNUM,3,ZD,START=4,               
                             85:SEQNUM,3,ZD,START=5)),             
  IFTHEN=(WHEN=INIT,                                               
  OVERLAY=(9:C'IFTHEN=(WHEN=(1,2,BI,EQ,',81,3,C'),OVERLAY=(',85,3,
           C':X''',C'FF''',C')),')),                               
  IFTHEN=(WHEN=(81,3,ZD,EQ,004),BUILD=(003:C'OPTION COPY')),       
  IFTHEN=(WHEN=(81,3,ZD,EQ,005),OVERLAY=(003:C'INREC')),           
  IFTHEN=(WHEN=(81,3,ZD,EQ,255),OVERLAY=(058:X))                   
//*
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

These users thanked the author skolusu for the post:
fatmonster (Tue Mar 18, 2014 10:24 pm)
skolusu
 
Posts: 586
Joined: Wed Apr 02, 2008 10:38 pm
Has thanked: 0 time
Been thanked: 39 times

Previous

Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post