Page 1 of 1

How to split and convert file to Hex?

PostPosted: Thu Jul 23, 2009 6:12 am
by vespa
Hello, I'm checking to see if Syncsort can handle this issue?
The data in the input file is wrapping (binary format) and has the double quote as CRLF. Is there an option/command in Syncsort to
split the data into single line when the double quote is encountered and convert the data to hex?

Input file: lrecl=112, FM=VB, binary data, record in file is wrapping and within double quote
Output file: lrecl=112, FM=VB, hex, one record per line

Thanks.

input file - in binary
1st record
000001                                    î                         Ã  î                         -` î   


input file - when set hex on
1st record:
000001                                    î                         Ã  î
       20333323333333333332002100020000020530000000010120021000200006405
       21002300905122215232DA200103000074264010000952012DA20020300005126

Re: How to split and convert file to Hex?

PostPosted: Thu Jul 23, 2009 7:45 am
by dick scherrer
Hello and welcome to the forum,

The data in the input file is wrapping
Your input has been "code"d for readability (using the Code tag). The hex on input shown is not the same as the "binary" input - not sure why this has been labeled binary. . . :?

Suggest you post more than one incomplete record to show what the input looks like. When posting the new data, use the Code tag.

You also need to explain what is meant by "hex" and "binary". These have no meaning in mainframe terminology. Every bit of data in every file is both hex and binary on the mainframe.

Where did the data originate? If the lrecl is always 112, why mention vb?

Re: How to split and convert file to Hex?

PostPosted: Thu Jul 23, 2009 8:13 am
by vespa
Thanks for responding, I usually do file attachment, can you me show me how to post the sample data using code tag?

The maximum record length in the file is 112 bytes, however, some of the record in the file has length < 112 bytes and was padded with spaces.
The file originated from the server, we used mainframe FTP to download it to mainframe environment for batch processing. As data in the file
is already binary/hex on the mainframe, can we use Syncsort to "unwrap" the file?

Re: How to split and convert file to Hex?

PostPosted: Thu Jul 23, 2009 8:30 am
by dick scherrer
Hello,

can you me show me how to post the sample data using code tag
Sure.

When you posted the first data, you used copy/paste from your mainframe session. To use the Code tag, do the same thing but after pasting the data, highlight the data in the post/reply and click on Code (Between Quote and List). To see how it will look to the forum, click Preview. When you are satisfied with the appearance, click Submit (make sure to Submit or the work will be lost - done that a few times. . .). You can add to your post and Preview several times as you go so that not too much effort is spend on something that is not as you want it. After a few uses, it becomes almost automatic :)

can we use Syncsort to "unwrap" the file
Maybe. We will be better able to tell when more complete data is available. I'm not sure we both have the same understanding for "wrapped", but more data should show this.

Re: How to split and convert file to Hex?

PostPosted: Thu Jul 23, 2009 9:23 am
by vespa
input file: sample of the first three record, I was able to cut and paste from 1 to 124, there are more data when you shift to the right ... also, typo error on record length, it's variable block of 512 bytes not 112 bytes.

000001                                    î                         Ã  î                         -` î                         -Ø 
       203333233333333333320021000200000205300000000101200210002000064053000000001012002100020000670530000000010120021000200006802
       21002300905122215232DA200103000074264010000952012DA200203000051264010000952022DA200303000009264010000952022DA20030300030018
----------------------------------------------------------------------------------------------------------------------------------
000002 à â ääáâ       Ã¢ à  âá  äá        ä    ä ãà  àáâ    ä           Ã¨           Â°       ÃˆÃ‹Ãˆ       /         Ã¡      á   Ã  â ääáâ
       434344443333333434334433443433333343344444433444333344333333200250000000000290000022777333200263333333344333333433343434444
       4127335235580002448325943591793329325113164654529308315642712DA2400000000002000100004342762DA210003608215182836501141273352


output file: need to have 1 record per line

000001                                                                                                                           
       003333233333333333344444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444
       110023009051222152300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
----------------------------------------------------------------------------------------------------------------------------------
000002             Ã®                                                                                                             
       100020000020530000000010144444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444
       001030000742640100009520100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
----------------------------------------------------------------------------------------------------------------------------------
000003          Á  î                                                                                                             
       100020000640530000000010144444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444
       002030000512640100009520200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000


also, I just checked the environment and I believe we use DFSORT, not Syncsort, sorry for the confusion, can you move this question under DFSORT forum? thanks....

Re: How to split and convert file to Hex?

PostPosted: Thu Jul 23, 2009 11:05 pm
by skolusu
vespa,

It looks like you want to split the records when you encounter CRLF (x'0D0A') . Do you know the max segments you have within each record? I guess you do realize that padding with spaces after the split would make them all 508 byte records thus making the VB file useless? It would better off to have a FB file when all the records are of the same length.

I think the whole problem could be avoided if you FTP data from the PC to mainframe in binary mode the CRLF will be removed automatically and your data will be split into different records once it encounters CRLF.

sample FTP job

//STEPFTP  EXEC PGM=FTP,PARM='/(EXIT',REGION=7000K
//INVB     DD DSN=<host file>,RECFM=VB,LRECL=512,BLKSIZE=27998
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
server name
user password
Binary
cd /filepath
get pc file //dd:invb
quit
/*


If you still insist on doing it via sort the following DFSORT JCL will give you the desired results. I assumed that you have a max of 10 segments within each record.

//STEP0100 EXEC PGM=SORT                                   
//SYSOUT   DD SYSOUT=*                                     
//SORTIN   DD DSN=Your input vb file,DISP=SHR
//SORTOUT  DD DSN=&&T1,DISP=(,PASS),SPACE=(CYL,(X,Y),RLSE) 
//SYSIN    DD *                                             
  SORT FIELDS=COPY                                         
  OUTFIL PARSE=(%01=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %02=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %03=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %04=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %05=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %06=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %07=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %08=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %09=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %10=(ENDBEFR=X'0D0A',FIXLEN=508)),         
  BUILD=(1,4,%01,/,1,4,%02,/,1,4,%03,/,1,4,%04,/,1,4,%05,/,
         1,4,%06,/,1,4,%07,/,1,4,%08,/,1,4,%09,/,1,4,%10)
//*   
//STEP0200 EXEC PGM=SORT                                   
//SYSOUT   DD SYSOUT=*                                     
//SORTIN   DD DSN=&&T1,DISP=SHR                             
//SORTOUT  DD SYSOUT=*                                     
//SYSIN    DD *                                             
  SORT FIELDS=COPY                                         
  OMIT COND=(5,256,CH,EQ,C' ',AND,257,252,CH,EQ,C' ')       
//*

Re: How to split and convert file to Hex?

PostPosted: Fri Jul 24, 2009 9:36 am
by vespa
Skolusu, thanks for your response. One quick question on mainframe FTP, does it allow you to wild card files? (ex: GET pcfile test_file*)

When I FTP the file using binary mode, the data is also wrapping. I tried nowrapping option and it still doesn't work.
if I use ASCII mode, the data splits into different records, however, looks like some of the data is converted to EBCDIC format??
so I'm assuming it works better if I FTP the file to mainframe using binary mode then reformat the record using DFSORT?

sample of input file after FTP in ASCII mode: record #1 thru 14
000001 " 0001 090531221523"                                                                   
000002 "    #     $ V4           "                                                           
000003 "    #    eA V4           "                                                           
000004 "    #    `y V4           "                                                           
000005 "    #    `  (            "                                                           
000006 "    eQy         ` 0#             "                                                   
000007 "    #    `        H      "                                                           
000008 "    %           6 P%  E        E "                                                   
000009 "     H  4Vx #Eg nnn    031243                             "                           
000010 "         E                 "                                                         
000011 "    #    `   T           "                                                           
000012 "S           "  W                                                                     
000013 "a00036082AE182836E011D1B7CCEB3558000B4D83BE94CE9A793329C25AACAFD65DEB9308CA564271"   
000014 "T          "        tst276"                                     


As each record is a long segment and its length is 512, and I do not need to padding it with the spaces after the split, is it correct if I change to this?

SORT FIELDS=COPY
OUTFIL PARSE=(%01=(ENDBEFR=X'0D0A',VLEN=508),

BUILD=(1,4,%01,/)

And do I need to change any thing in this step?

//STEP0200 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=&&T1,DISP=SHR
//SORTOUT DD SYSOUT=*
//SYSIN DD *
SORT FIELDS=COPY
OMIT COND=(5,256,CH,EQ,C' ',AND,257,252,CH,EQ,C' ')
//*

Thanks.

Re: How to split and convert file to Hex?

PostPosted: Fri Jul 24, 2009 11:08 pm
by skolusu
vespa,

vespa wrote:if I use ASCII mode, the data splits into different records, however, looks like some of the data is converted to EBCDIC format??


Everything on Mainframes is EBCDIC . What do you mean some of the data?

Skolusu, thanks for your response. One quick question on mainframe FTP, does it allow you to wild card files? (ex: GET pcfile test_file*)


I am not sure about wild cards but you can look it up in the FTP manual.

As each record is a long segment and its length is 512, and I do not need to padding it with the spaces after the split, is it correct if I change to this?


we need to add VLTRIM at the end of build parm. Change your step0100 cards to the following
//SYSIN    DD *                                             
  SORT FIELDS=COPY                                         
  OUTFIL PARSE=(%01=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %02=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %03=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %04=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %05=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %06=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %07=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %08=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %09=(ENDBEFR=X'0D0A',FIXLEN=508),           
                %10=(ENDBEFR=X'0D0A',FIXLEN=508)),         
  BUILD=(1,4,%01,/,1,4,%02,/,1,4,%03,/,1,4,%04,/,1,4,%05,/,
         1,4,%06,/,1,4,%07,/,1,4,%08,/,1,4,%09,/,1,4,%10), 
  VLTRIM=C' '                                               
//*


and step0200 control cards to the following

//SYSIN    DD *         
  SORT FIELDS=COPY       
  OMIT COND=(1,2,BI,EQ,5)
//*

Re: How to split and convert file to Hex?

PostPosted: Sat Jul 25, 2009 4:23 am
by vespa
skolusu, thanks for your helps ! :)