Page 1 of 2

Moving data from mainframe to *nix boxes

PostPosted: Sat Jul 14, 2012 12:09 am
by v1gnesh
Hello,

I need some help in moving data which contains packed decimal data(not the only content in it). I need to preserve the EBCDIC code so that the content doesn't go unreadable/lose alignment while receiving at the *nix end. I've tried using the iconv command
iconv -f ibm-1047 -t iso8859-1 filename > filename1
. The source is an MVS dataset that I copied to OMVS using OCOPY. Any help is highly appreciated. Thanks!

Thanks!
Vignesh

Re: Moving data from mainframe to *nix boxes

PostPosted: Sat Jul 14, 2012 12:23 am
by Robert Sample
There are two ways -- and ONLY two ways -- to handle packed decimal data on a *nix machine:

1) unpack the packed decimal fields before transferring the data set to the *nix box
2) transfer the data set as binary and work with it on the *nix box as EBCDIC data.

iconv translates characters. which is NOT what you want. The reason is that the value 97994 (for example) in packed decimal with a positive sign is X'97994C' in EBCDIC. Character-wise, this is a lower-case p, a lower-case r and the < symbol. Translate these characters to ASCII and you're looking at X'70723C' for these three characters. If you translate the data, either through iconv or during the FTP or in any other way, without first unpacking the packed decimal fields, the translate will completely change the packed decimal values.

Re: Moving data from mainframe to *nix boxes

PostPosted: Sat Jul 14, 2012 1:17 am
by dick scherrer
Hello,

The easiest way my teams have used is to create the file on the mainframe containing nothing but text. Numeric fields are punctuated so that precision is not lost (i.e. the decimal point and the +/- sign - commas not needed). The fields in the download file are tab-delimited (because the tab character (x'05') will not be in text data) and tab-delimited files are easily usable on both *nix and Win-based targets.

Re: Moving data from mainframe to *nix boxes

PostPosted: Sat Jul 14, 2012 1:40 am
by steve-myers
Mr. Scherrer is right on. About 10 years ago I was involved in analyzing SMF data. I had worked up a good mainframe report, but it appeared desirable to ship the stuff to Windoze to feed into Windoze spreadsheets for graphing. It turned out it needed sorting differently than the mainframe reports. After the sort I wrote the data into CSV format - note, no binary or packed decimal data, just text. The text CSV data was sent to Windoze, and loaded into a spreadsheet, and with minor work in the spreadsheet, produced excellent graphs. By using CSV or tab delimited data, the spreadsheet import realigns the data into spreadsheet cells. The spreadsheet import also converted date and time data into spreadsheet usable date & time data.

Re: Moving data from mainframe to *nix boxes

PostPosted: Sat Jul 14, 2012 3:24 am
by v1gnesh
Robert Sample wrote:2) transfer the data set as binary and work with it on the *nix box as EBCDIC data.


Would this require any software on the *nix box?

Re: Moving data from mainframe to *nix boxes

PostPosted: Sat Jul 14, 2012 3:26 am
by v1gnesh
dick scherrer wrote:Hello,

The easiest way my teams have used is to create the file on the mainframe containing nothing but text. Numeric fields are punctuated so that precision is not lost (i.e. the decimal point and the +/- sign - commas not needed). The fields in the download file are tab-delimited (because the tab character (x'05') will not be in text data) and tab-delimited files are easily usable on both *nix and Win-based targets.



I'm not quite sure I follow you about the 'nothing but text'.. Are you suggesting that I remove the packed decimal data before I send it over? If so, that's fine too because I just want to get the data(everything other than the packed decimal content) across.

Re: Moving data from mainframe to *nix boxes

PostPosted: Sat Jul 14, 2012 9:15 am
by dick scherrer
Hello,

Robert Sample wrote:
2) transfer the data set as binary and work with it on the *nix box as EBCDIC data.

Would this require any software on the *nix box?
It is a LOT of work on the *nix box. Not to be undertaken lightly. . . At least this was true a while back . . .

Are you suggesting that I remove the packed decimal data before I send it over?
Yes, in the process that creates the download file, generate edited numeric values for any packed-decimal or binary values.

Re: Moving data from mainframe to *nix boxes

PostPosted: Sat Jul 14, 2012 8:48 pm
by v1gnesh
dick scherrer wrote:It is a LOT of work on the *nix box. Not to be undertaken lightly. . . At least this was true a while back . . .


Can you please explain a little bit on this..?

dick scherrer wrote:Yes, in the process that creates the download file, generate edited numeric values for any packed-decimal or binary values.


How do I do this? I'm not the one creating the data - I'm asking this here on behalf of a client.

Thank you very much..

Re: Moving data from mainframe to *nix boxes

PostPosted: Sat Jul 14, 2012 10:53 pm
by Robert Sample
You have to write a program or use a utility (such as SORT) to convert the packed decimal data to zoned decimal data. It doesn't matter if your client does this or you do this -- but if you want the data to be valid and usable on the *nix machine, SOMEBODY has to convert the packed decimal data.

Re: Moving data from mainframe to *nix boxes

PostPosted: Sat Jul 14, 2012 11:53 pm
by dick scherrer
Hello,

Can you please explain a little bit on this..?
Not really a good topic for our type of forum. . .

If you do not already understand what this requires, it is probably not an alternative to consider. . .

What is needed is to download the native mainframe data to the target in binary (the easy part). Then the target has to have code to convert/translate the binary to something usable on the target. Which means (among other things) the target has to know the field type, position, and length, of every field in the download file. Then the target system code has to be able to properly convert each field from the binary file to whatever is needed. If there are hundreds or thousands of files, the time required to do this individually would be prohibitive. If you don't already have this on the target, the cost of writing generic code to do this would also be prohibitive.