cksum byte count different from OMVS and UNIX



Post anything related to mainframes (IBM & UNISYS) if not fit in any of the above categories

cksum byte count different from OMVS and UNIX

Postby bbonner » Sat Dec 20, 2008 5:49 am

I've been tasked with writing a JCL proc that creates a PS dataset containing the POSIX cksum of a passed dataset or member. Currently the proc is working fine. It ibgens the dataset over to OMVS, and then I use BPXBATCH to run cksum on the file. Now, when the same dataset is downloaded from the mainframe to a UNIX server via binary download the byte count (and obviously the CRC hash) are different. At this point I'm not sure how to pinpoint where the failure is. It could be the IEBGENER is monkeying around and not doing a faithful copy, or it could be that the FTP binary download is doing something (though to my understanding it shouldn't). To recap, I have a PS dataset that is copied to OMVS where a cksum is performed. Then, the cksum is put into a PS dataset, and then both the cksum dataset and the original PS dataset are FTP'd (client initiated). The number of bytes cksum reports on OMVS is different from the number of bytes reported on the UNIX server.

I would be inclined to think that the problem is with the FTP as I have tested IEBGENER by copying the data to OMVS, then copying that data into a new PS dataset, and then copying it back, and comparing the cksums each time, and they were identical. I would think is IEBGENER changed anything, it would do it both times. So I'm not sure what to try at this point. Any help would be appreciated.

Output from OMVS cksum: 568045815.25549481./tmp/input.ADW832
Output from UNIX cksum: 635370730.25544160.input.ADW832

Here is the JCL in the proc

//CKSUM    PROC DSOUT='*',DSIN='*',JNAME=''
//*******************************************************
//* Inputs  :  DSOUT,DSIN,JNAME                         *
//* &DSOUT  :  The dataset to send the cksums to        *
//* &DSIN   :  The Dataset you are getting a cksum of   *
//* &JNAME  :  The Name of the job invoking the proc    *
//*******************************************************
//*******************************************************
//*        Delete DSOUT File if it exists               *
//*******************************************************
//STEP1    EXEC PGM=IEFBR14
//SYSPRINT DD  SYSOUT=*
//LOOKHERE DD  DSN=&DSOUT,
//         DISP=(MOD,DELETE,DELETE),SPACE=(TRK,(1,1))
//*******************************************************
//*        Copy From DSIN to chksum_input.JNAME         *
//*******************************************************
//IEBGENER EXEC PGM=IEBGENER,REGION=5M,TIME=100
//SYSPRINT DD  SYSOUT=*
//SYSIN    DD  DUMMY
//SYSUT1   DD DSN=&DSIN,
//         DISP=(OLD,KEEP,KEEP)
//SYSUT2   DD PATH='/tmp/chksum_input.&JNAME',
//         PATHOPTS=(OWRONLY,OCREAT),PATHDISP=(KEEP,DELETE),
//         PATHMODE=(SIWUSR,SIXUSR,SIRUSR)
//*******************************************************
//*        Run BPXBATCH To cksum passed file            *
//*******************************************************
//BPXBATCH EXEC PGM=BPXBATCH,                                           
//         PARM='SH cksum /tmp/chksum_input.&JNAME'
//STDOUT   DD PATH='/tmp/cksmout.&JNAME',PATHOPTS=(OWRONLY,OCREAT),     
//         PATHDISP=(KEEP,DELETE),PATHMODE=(SIWUSR,SIXUSR,SIRUSR)       
//*******************************************************
//*        COPY FROM cksmout.&JNAME to DSOUT            *
//*******************************************************
//IEBGENER EXEC PGM=IEBGENER,REGION=5M,TIME=100
//SYSPRINT DD  SYSOUT=*
//SYSIN    DD  DUMMY
//SYSUT1   DD PATH='/tmp/cksmout.&JNAME',PATHOPTS=(ORDONLY),
//         PATHDISP=(DELETE,DELETE),FILEDATA=TEXT,LRECL=4096,RECFM=U,
//         BLKSIZE=4096
//SYSUT2   DD DSN=&DSOUT,DISP=(NEW,CATLG),
//         DCB=(DSORG=PS,RECFM=U,LRECL=80),
//         SPACE=(TRK,(1,1))
//*******************************************************
//*        Run BPXBATCH to delete all .&JNAME files     *
//*******************************************************
//BPXBATCH EXEC PGM=BPXBATCH,                                           
//         PARM='SH find /tmp/ -name "*.&JNAME" | xargs rm -f'
//STDOUT   DD PATH='/tmp/std.out',PATHOPTS=(OWRONLY,OCREAT),           
//         PATHDISP=(DELETE,DELETE),PATHMODE=(SIWUSR,SIXUSR,SIRUSR)
bbonner
 
Posts: 5
Joined: Sat Dec 20, 2008 4:25 am
Has thanked: 0 time
Been thanked: 0 time

Re: cksum byte count different from OMVS and UNIX

 

Re: cksum byte count different from OMVS and UNIX

Postby dick scherrer » Sat Dec 20, 2008 6:30 am

Hello and welcome to the forum,

FWIW - neither iebgener nor ftp will modify the contents of a file being processed unless specifically told to. . .

What is the content of the ps file? How is it created?
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6304
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 91 times

Re: cksum byte count different from OMVS and UNIX

Postby bbonner » Sun Dec 21, 2008 1:25 am

Thanks for the welcome. :)

The PS file is the output from another batch job and contains semicolon delimited data (text). It has about 250k records, record length of 516, block size of 27998, record format of VB, BIG data class, and allocated 32 cylinders.

Now, I just finished remoting into work, and tried the whole process with a different FTP client, and it worked this time. The only thing I can surmise is that the FTP session was interrupted before seeing as the number of bytes reported in UNIX is less than the number of bytes reported in OMVS, or the FTP client has a bug for large binary downloads. I'll test it a bunch more to be sure, because it failed on me at least 3 times before I was desperate enough to post in a forum, so its sudden cooperation concerns me a bit. :) Thanks for lending your brain.
bbonner
 
Posts: 5
Joined: Sat Dec 20, 2008 4:25 am
Has thanked: 0 time
Been thanked: 0 time

Re: cksum byte count different from OMVS and UNIX

Postby dick scherrer » Sun Dec 21, 2008 12:50 pm

Hello,


I noticed that recfm=u has been specified. If this:
record length of 516, block size of 27998, record format of VB,
is the actual attributes, i'd suggest using these rather than recfm=u.

When we transfer data from the mainframe to UNIX or Windows, we "always" use ascii. There have been a couple of exceptons where some code existed on a unix system that could deal with mianframe binary data. I've always transferred to/from the mainframe to the target without doing anything with the uss/omvs.

Good luck and someone will be here if there are questions/problems :)
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6304
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 91 times

Re: cksum byte count different from OMVS and UNIX

Postby bbonner » Tue Dec 23, 2008 4:03 am

The recfm=u that you see is for the new dataset that only contains the cksum of the passed one. The only thing that happens to the dataset with the data copied from the mainframe after getting its cksum is deleting it. :) Thanks again.
bbonner
 
Posts: 5
Joined: Sat Dec 20, 2008 4:25 am
Has thanked: 0 time
Been thanked: 0 time

Re: cksum byte count different from OMVS and UNIX

Postby AnandAK » Tue Apr 03, 2012 5:31 pm

Hi ,
I am a new member to the forum. My requirement is also same , want to get the cksum for a file calculated from MVS. I went throught this discussion and created jcl in the same way as given by bbonner. But when I run the job in MVS , the checksum value which I got is different from the value which I got when I ran cksum command on the same file in unix.

checksum calculated from Unix is :
=============================
2254304259 46375 MCCFXICT

checksum calculated from MVS is :
=============================
2610446388.46144./tmp/chksum_input.fxict

My JCL is given below:
**************************************************************************************************************
//CKSUM011 JOB (AAAA,TXX,000000),'XXXXXXXX',REGION=0K,
// MSGCLASS=X,
// CLASS=D,
// NOTIFY=&SYSUID
//*******************************************************
//* DELETE OUTPUT FILE IF IT EXISTS *
//*******************************************************
//STEP1 EXEC PGM=IEFBR14
//SYSPRINT DD SYSOUT=*
//LOOKHERE DD DSN=WPH.CHKSUM.OUT,
// DISP=(MOD,DELETE,DELETE),SPACE=(TRK,(1,1))
//*******************************************************
//* COPY FROM input TO CHKSUM_INPUT.FXICT *
//*******************************************************
//IEBGENER EXEC PGM=IEBGENER,REGION=5M,TIME=100
//SYSPRINT DD SYSOUT=*
//SYSIN DD DUMMY
//SYSUT1 DD DSN=WPH.MCCFXICT.CHKSUM,
// DISP=(OLD,KEEP,KEEP)
//SYSUT2 DD PATH='/tmp/chksum_input.fxict',
// PATHOPTS=(OWRONLY,OCREAT),PATHDISP=(KEEP,DELETE),
// PATHMODE=(SIWUSR,SIXUSR,SIRUSR)
//*******************************************************
//* RUN BPXBATCH TO CKSUM PASSED FILE *
//*******************************************************
//BPXBATCH EXEC PGM=BPXBATCH,
// PARM='SH cksum /tmp/chksum_input.fxict'
//STDOUT DD PATH='/tmp/cksmout.fxict',PATHOPTS=(OWRONLY,OCREAT),
// PATHDISP=(KEEP,DELETE),PATHMODE=(SIWUSR,SIXUSR,SIRUSR)
//*******************************************************
//* COPY FROM CKSMOUT.fxict TO output file *
//*******************************************************
//IEBGENER EXEC PGM=IEBGENER,REGION=5M,TIME=100
//SYSPRINT DD SYSOUT=*
//SYSIN DD DUMMY
//SYSUT1 DD PATH='/tmp/cksmout.fxict',PATHOPTS=(ORDONLY),
// PATHDISP=(DELETE,DELETE),FILEDATA=TEXT,LRECL=4096,RECFM=U,
// BLKSIZE=4096
//SYSUT2 DD DSN=WPH.CHKSUM.OUT,DISP=(NEW,CATLG),
// DCB=(DSORG=PS,RECFM=U,LRECL=80),
// SPACE=(TRK,(1,1))
//*******************************************************
//* RUN BPXBATCH TO DELETE ALL .fxict FILES *
//*******************************************************
//BPXBATCH EXEC PGM=BPXBATCH,
// PARM='SH find /tmp/ -name "*.fxict" | xargs rm -f'
//STDOUT DD PATH='/tmp/std.out',PATHOPTS=(OWRONLY,OCREAT),
// PATHDISP=(DELETE,DELETE),PATHMODE=(SIWUSR,SIXUSR,SIRUSR)
**************************************************************************************************************


Could someone help me out in finding out what is the issue
AnandAK
 
Posts: 6
Joined: Mon Mar 12, 2012 5:19 pm
Has thanked: 0 time
Been thanked: 0 time

Re: cksum byte count different from OMVS and UNIX

Postby Robert Sample » Tue Apr 03, 2012 7:28 pm

Are you aware that Unix System Services running on z/OS is using EBCDIC whereas almost all Unix implementations use ASCII? The hex representation of a capital A in ASCII will be x'41' while a mainframe EBCDIC captial A hex representation will be x'C1'. Hence you cannot compare a check sum generated on an Unix system with a check sum on a mainframe since the values are going to be different pretty much at all times.
Robert Sample
Global moderator
 
Posts: 3367
Joined: Sat Dec 19, 2009 8:32 pm
Location: Dubuque, Iowa, USA
Has thanked: 1 time
Been thanked: 222 times

Re: cksum byte count different from OMVS and UNIX

Postby steve-myers » Tue Apr 03, 2012 11:02 pm

Yes, indeed. I was about to write a post like Mr. Sample's post, but Mr. Sample hit the nail squarely on its head. I want to add -

A sequential "flat" file in OMVS contains EBCDIC characters, just like a sequential "flat" MVS dataset. An FTP transfer of text data translates the EBCDIC data as used in MVS to some form of ASCII before it leaves the MVS system.

In addition, you must realize "flat" text file in OMVS and Unix is structured differently than a "flat" dataset in MVS, or a "flat" text file in Windoze, and this structure affects the checksum. In OMVS and Unix, each text line is terminated with a "new line" control character. MVS does not use control characters: IBM made this mistake in some second generation systems and did not repeat it in System/360 data structures. For example, when you specify RECFM=FB, LRECL=80, each line contains 80 bytes, no more and no less, and the next line immediately follows the current line with no embedded control information like the "new line" control data used in Unix and OMVS or the "carriage return" and "line feed" characters used in Windoze and MS/DOS and, presumably, CP/M.
steve-myers
Global moderator
 
Posts: 1886
Joined: Thu Jun 03, 2010 6:21 pm
Has thanked: 4 times
Been thanked: 197 times

Re: cksum byte count different from OMVS and UNIX

Postby AnandAK » Wed Apr 04, 2012 12:21 pm

Thanks a lot for the answers... But the main use of cksum utiltily is to make sure that data transferred from one system to other has been recieved accurately without any data corruption. In that case I think there should be some way to tackle the issue. Is anybody aware how we can handle this.

Also in the first post in this topic Mr bbonner was tring to compare between the results form unix and mainframe. and finally he got the results correctly also.
So i think this can be handled, but not sure how.
AnandAK
 
Posts: 6
Joined: Mon Mar 12, 2012 5:19 pm
Has thanked: 0 time
Been thanked: 0 time

Re: cksum byte count different from OMVS and UNIX

Postby AnandAK » Wed Apr 04, 2012 5:03 pm

Hi all,

Can we use the chcp command to change code page before running the cksum in BPXBATCH?
AnandAK
 
Posts: 6
Joined: Mon Mar 12, 2012 5:19 pm
Has thanked: 0 time
Been thanked: 0 time

Next

Return to All other Mainframe Topics

 


  • Related topics
    Replies
    Views
    Last post