Unable to copy a flat file to VSAM KSDS file



Help for IBM's record-oriented filesystem VSAM, ESDS, KSDS, RRDS, LDS and Storage management Subsystems

Unable to copy a flat file to VSAM KSDS file

Postby Nara_513 » Wed Jul 24, 2013 9:58 pm

Hi ALL,

We have a production job which is failing while copying a flat file into a VSAM file with the below properties :

DEFINE CLUSTER                            -
(NAME(AAAAA.BB.CRD.XXXXX.TARG.ACCESS)  -
FREESPACE(20 20)                          -
INDEXED                                   -
KEYS(12 0)                                 -
SUBAL                                     -
SPEED                                     -
SHR (2 3)                                  -
RECORDSIZE(438 438)                        -
CYL(1500 1500))                                -
DATA                                       -
(NAME(AAAAA.BB.CRD.XXXXX.TARG.ACCESS) -
  CISZ(8192))                                 -
INDEX                                        -
(NAME(AAAAA.BB.CRD.XXXXX.TARG.ACCESS.INDEX) -
  CISZ(8192));



The flat file has :
WER246I  FILESIZE 2,510,714,960 BYTES
WER054I  RCD IN    5838872, OUT    5838872


Here the job is failing with the below error:


V370040 END OF VOLUME RECOVERY OPERATION SUCCESSFUL FOR
V370040 Job Name,STEP030,OUTDD,AAAAA.BB.CRD.XXXXX.TARG.ACCESS.DATA
V370040 NEW VOLUME SMS ADDED, NEW VOLUME COUNT 3
IEC070I 034(004)-220,Job name,STEP030,OUTDD,E619,VSS211,


When checked in Quick view we got the following description for the error :
IEC161I (return code 034) rc[(sfi)]-ccc,jjj,
        sss,ddname,dev,volser,xxx,dsname,cat

Routing Code: 11

Descriptor Code: 6

Explanation:  For general information about message IEC161I, see the
explanation for return code 001.

Specific information for this return code: The maximum relative byte
address (RBA) limit (4,294,967,295) has been reached.

The subfunction information field, if present, contains a decimal
problem-determination code (reason code) for use by the IBM Support Center
in case further problem diagnosis is necessary. The following table lists

+---------------+--------------------------------------------------------|
| 004           | End of volume - Non-extended addressable. The new      |
|                   | allocation amount would exceed 4 GB.                            |



The same is inferred from the IDCAMS Sysout log as well :

IDCAMS  SYSTEM SERVICES                                           

REPRO -
   INFILE(INDD) -
   OUTFILE(OUTDD)
IDC3302I  ACTION ERROR ON AAAAA.BB.CRD.XXXXX.TARG.ACCESS
IDC3351I ** VSAM I/O RETURN CODE IS 28  - RPLFDBWD = X'2908001C'
IDC31467I MAXIMUM ERROR LIMIT REACHED.
IDC0005I NUMBER OF RECORDS PROCESSED WAS 5836320
IDC3003I FUNCTION TERMINATED. CONDITION CODE IS 12

IDC0002I IDCAMS PROCESSING COMPLETE. MAXIMUM CONDITION CODE WAS 12

IDC3351I ** VSAM {OPEN|CLOSE|I/O} RETURN CODE IS return-code
     {RPLFDBWD=nnnnnnnn}

Explanation:  An error was encountered during virtual storage access
method (VSAM) open, close, or action request processing, as indicated in
the text of the message.

28   Data set cannot be extended because VSAM cannot allocate
   additional direct-access storage space. Either there is not
   enough space left to make the secondary allocation request,
   you attempted to increase the size of a data set while
   processing with SHROPT=4 and DISP=SHR, or the index CI is
   not large enough to hold the entire CA. This error could
   also be due to a data set trying to extend beyond 4GB on a
   system that does not support extended addressability.




So here my problem is , when i run the same job JCL (which got failed in production) using the personal datasets in spool , the job completed fine . I have used the same VSAM file definition to create my personal VSAM dataset and copied the same flat file to it using IDCAMS Repro command .

Now , in this situation, do i need to change the VSAM file properties to fix my issue or is it something issue with the production Volume Pools which is causing the job to fail ? since my personal job ran fine copying the flat file into VSAM .

Note :The flat file doesnt have any duplicates , did ran a sort and verified it .

I even checked with our IBM storage management guys , they have replied that the Volume pool is having enough spacefor the VSAm to extend . They also ranout of ideas regarding this .


Can any one suggest whether am missing any thing here in this , which actually solves my problem .

Thanks for your time
Nara
Nara_513
 
Posts: 44
Joined: Fri Dec 02, 2011 1:11 am
Has thanked: 0 time
Been thanked: 0 time

Re: Unable to copy a flat file to VSAM KSDS file

Postby Robert Sample » Wed Jul 24, 2013 10:40 pm

You are hitting the 4 GB limit for a VSAM file. You should change your DEFINE to indicate it is an extended format VSAM file.
Robert Sample
Global moderator
 
Posts: 3719
Joined: Sat Dec 19, 2009 8:32 pm
Location: Dubuque, Iowa, USA
Has thanked: 1 time
Been thanked: 279 times

Re: Unable to copy a flat file to VSAM KSDS file

Postby c62ap90 » Wed Jul 24, 2013 10:52 pm

Google helps for IEC070I 034(004)-220
http://www-01.ibm.com/support/docview.w ... sg1OA30026
Your IBM Storage management guys "should" be able to solve this quickly. SMS vs. non-SMS DASD.

Just curious if any records loaded to your VSAM file? If so, how many records? LISTCAT will tell you.
c62ap90
 
Posts: 125
Joined: Thu Oct 11, 2012 10:24 pm
Has thanked: 1 time
Been thanked: 7 times

Re: Unable to copy a flat file to VSAM KSDS file

Postby Nara_513 » Thu Jul 25, 2013 2:45 pm

Robert,

I do have a doubt, if you check the flat file size its is showing around 2.5 GB , so can i know whether the file size will be varying when we copy it into VSAm file ?

FILESIZE 2,510,714,960 BYTES
Nara_513
 
Posts: 44
Joined: Fri Dec 02, 2011 1:11 am
Has thanked: 0 time
Been thanked: 0 time

Re: Unable to copy a flat file to VSAM KSDS file

Postby Nara_513 » Thu Jul 25, 2013 2:47 pm

c62ap90 wrote:Google helps for IEC070I 034(004)-220
http://www-01.ibm.com/support/docview.w ... sg1OA30026
Your IBM Storage management guys "should" be able to solve this quickly. SMS vs. non-SMS DASD.

Just curious if any records loaded to your VSAM file? If so, how many records? LISTCAT will tell you.


I checked , No records were loaded into the VSAM file
Nara_513
 
Posts: 44
Joined: Fri Dec 02, 2011 1:11 am
Has thanked: 0 time
Been thanked: 0 time

Re: Unable to copy a flat file to VSAM KSDS file

Postby Robert Sample » Thu Jul 25, 2013 3:11 pm

Because you told the system to allocate the VSAM file with 20% free space in both the CA and CI. My Excel space estimator indicates you need roughly 5900 cylinders for the number of records, which is WELL beyond the 4 GB limit.
Robert Sample
Global moderator
 
Posts: 3719
Joined: Sat Dec 19, 2009 8:32 pm
Location: Dubuque, Iowa, USA
Has thanked: 1 time
Been thanked: 279 times

Re: Unable to copy a flat file to VSAM KSDS file

Postby Nara_513 » Thu Jul 25, 2013 4:39 pm

Hi Robert,

Yep I do agree with you , but we ran the job in Test , using the below VSAM properties and copied the same flat file into the VSAM file .


DEFINE CLUSTER                            -
(NAME(AAAAA.BB.CRD.XXXXX.TARG.ACCESS)  -
FREESPACE(20 20)                          -
INDEXED                                   -
KEYS(12 0)                                 -
SUBAL                                     -
SPEED                                     -
SHR (2 3)                                  -
RECORDSIZE(438 438)                        -
CYL(20 90))                                -
DATA                                       -
(NAME(AAAAA.BB.CRD.XXXXX.TARG.ACCESS.DATA) -
  CISZ(8192))                                 -
INDEX                                        -
(NAME(AAAAA.BB.CRD.XXXXX.TARG.ACCESS.INDEX) -
  CISZ(4096));




We have found the soultion :D to it by working with our IBM partners , but the solution is a work around but we are not able to figure out the original root cause :

As per IBM
" I update the VAM code to send the files to the test pool. This is where XXXXX was able to successfully create a test version of these files. The job was restarted from the top so it will be 10 hours before it gets to the failing step. "

So , in the sense , the production job was recovered by running a script which enabled the production job to use the test pool where it has created the VSAM file . We verified that the production VSAM file created is in same test pool .

But what wonders me is why the job failed when running in production and couldnt able to use the production pools , where as the same completed fine when used thetest pools . we verifeid with the Storage management team and confirmed that both the pools are having same properties . So where does the root cause lie ? :?:

Thanks Nara
Nara_513
 
Posts: 44
Joined: Fri Dec 02, 2011 1:11 am
Has thanked: 0 time
Been thanked: 0 time

Re: Unable to copy a flat file to VSAM KSDS file

Postby Robert Sample » Thu Jul 25, 2013 5:07 pm

The third or fourth volume allocated is where your problem will be (most likely). The reason it ran in test is that you did not allocate 1500 cylinders in test. 1500 cylinders of data with 8192 bytes per block will run about 1.1 GB per primary space allocation; the fourth one allocated will exceed the 4 GB limit. This could be on a single volume, or two volumes, but most likely SMS put each primary on a different volume.

The analysis that should have been done before the VSAM allocation: using half-track 3390 blocking, the sequential file places 1890 438-byte records per cylinder or about 3090 cylinders for the 5,838,872 records. For VSAM, you get 90 CI per CA with 8192 CISIZE. 20% of those are free, so 72 usable CI per cylinder. Taking out the overhead, and with 20% free space per CI, there are 6545 bytes of usable space per CI, which allows 14 records per CI. 14 times 72 gives a total of 1008 records per cylinder and hence requires 5793 cylinders to hold 5,838,872 records. Using 1450 cylinders instead of 1500 would give 4 primary allocations about the right amount of space and probably keep your file size under 4 GB.

Bonus observation: your index CI is WAY overallocated. The rule of thumb I've used for years is 4 bytes + half the key length gives the average compressed key length stored in the index CI. 4 + (12 / 2) is 10 bytes; 10 bytes times 90 CI / CA indicates your index CI size should be at least 900; you could use either 1024 or 1536 and be safe. Using 8192 (or even 4096) means 7000 (or 3000) bytes per index CI is not going to be used -- ever.
Robert Sample
Global moderator
 
Posts: 3719
Joined: Sat Dec 19, 2009 8:32 pm
Location: Dubuque, Iowa, USA
Has thanked: 1 time
Been thanked: 279 times

Re: Unable to copy a flat file to VSAM KSDS file

Postby Nara_513 » Fri Jul 26, 2013 12:49 pm

Robert,

Thanks for the detailed explanation, will consider chaging the VSAM file definition in our job .

Nara
Nara_513
 
Posts: 44
Joined: Fri Dec 02, 2011 1:11 am
Has thanked: 0 time
Been thanked: 0 time

Re: Unable to copy a flat file to VSAM KSDS file

Postby Shikha » Sat Dec 13, 2014 1:21 pm

A similar porblem to the above ->

Unable to copy a flat file to VSAM KSDS file -

following is the definition and it is hitting 4GB limit. What can be done to ensure the copy is complete. Can you please help?

OWNER(CA7CSF) -
KEY(29 0) -
RECSZ(126 5344) -
FREESPACE(0 0) -
RECOVERY -
NOERASE -
INDEXED -
NOWRITECHECK -
NOIMBED -
NOREPLICATE -
UNORDERED -
REUSE -
CYL(1500 1500) -
NONSPANNED -
SHR(2 3) -
VOLUME(****))
Shikha
 
Posts: 5
Joined: Sat Dec 13, 2014 1:13 pm
Has thanked: 0 time
Been thanked: 0 time

Next

Return to VSAM/SMS

 


  • Related topics
    Replies
    Views
    Last post