CICS transaction failures in production



Support for CICS/ESA, CICS/TS & Transaction Gateway, CICS Configuration Manager and CICS Performance Analyzer

CICS transaction failures in production

Postby Nara_513 » Mon Nov 11, 2013 10:40 am

Hi All,

Last week we had a production issue in mainframes CICS, where in we had lot of transactions failures(different transactions) continoulsy in one particular region alone. for recovery we had recycled the region and the transactions failures stopped.

For root cause, We have checked the transaction failures and found few things from the Abend-aid.

Initally a transaction got failed with the ASRA abend code.

An ASRA abend occurred in program XXXXXX.  The abending transaction was
QSDS running at terminal for user ID YYYYY.

Analysis of error

The Segment-translation exception occurs when:

     -  A program attempted to reference storage that was paged out.

     -  A program attempted to reference storage that was not acquired.

     -  A program attempted to reference storage that was not available to
        the subspace that it was running in.
To correct the error either page-fix the storage, allocate the storage or
make sure that the program can reference the storage in the subspace.


So, i have checked the working storage and linkage section varaibles values and found that the linkage section variables were not intialaised :


Analysis of error

004776              MOVE TABLE-MAINT-INTFC-KEY
004777                TO PHYSICAL-DB-RECORD.


Current values of fields on this statement:

Level/Field Name                      Picture/Type        Value
03 TABLE-MAINT-INTFC-KEY              X(21)               *NOT INITIALIZED*
02 PHYSICAL-DB-RECORD                 X(5000)             SPACES



Now, i have checked the remaining transactions failures which got failed (after the first transaction failure) in abend aid and all the transactions failed with the same reason ASRA while performing the initailisation section where in we are setting the linkage pointer to the working storage variable. Linkage pointer in abend aid is showing as "Not Initialised"

So i have checked the trace of the first transaction failure and found the below:


DLD  ENTRY ACQUIRE_PROGRAM       1C5C3AC0,YES
MLM  ENTRY LOCK                  1893CCB0,SHARED
MLM  EXIT  LOCK/OK
DLD1 EVENT PRE-SVC-CALL          LLACOPY
DLD1 *EXC* SVC-EXCEPTION         LLACOPY,NOT-FOUND
MLM  ENTRY UNLOCK                1893CCB0,SHARED
MLM  EXIT  UNLOCK/OK
DLD  EXIT  ACQUIRE_PROGRAM/EXCEPTION PROGRAM_NOT_FOUN
GLD  EXIT  LOAD_EXEC/EXCEPTION   PROGRAM_NOT_LOADABLE



For remaining transactions, the trace logs showing the below:


EVENT LINK-TO-USER-EXIT-PROGRAM GLUEEXIT AT EXI
EVENT RETURN-FROM-USER-EXIT-PROGRAM GLUEEXIT WI
EXIT  GETMAIN               OK
*EXC* Program-Check         START_PROGRAM,QAOSW
EVENT CALL-TO-LE/370        Program_Check_Recov
EVENT RETURN-FROM-LE/370    Program_Check_Recov
*EXC* PROGRAM_CHECK
ENTRY INQUIRE_TASK
EXIT  INQUIRE_TASK/OK       ESSENTIAL_YES
ENTRY INQUIRE_CURRENT_PROGRAM
EXIT  INQUIRE_CURRENT_PROGRAM/OK CA48,1D55F7D0,
*EXC* ABEND_ASRA            QAXXXXX,00002242,U




We are checking with our IBM partners regarding this, before that i want to make sure that i have enough information regarding this. So I have few doubts here:

1. I have assumed that the first transaction failure might have caused the remaining transactions to fail. If so, how i can prove this ? What pramaters or things i need to check in the abend aid to prove that the first transaction caused this.

Here the calling transaction XXXX is same for all of the failing transactions(We have some 15 different transactions which are called by calling program XXXX ).

2. Is their any chance that this might have happened because of the Storage issue ? Because the transactions failed untill we recycled the region.

3. Can any one explain what the statment means : DLD1 *EXC* SVC-EXCEPTION LLACOPY,NOT-FOUND
I did a search and found the below statements:


The Loader domain will do a MVS BLDL SVC or LLACOPY PC Call,
and a MVS Direct LOAD request for every new program LOAD, LINK, and
XCTL, after a Cold Start or a NEWCOPY, not just for LOADs.




Thansk All for your time
Nara
Nara_513
 
Posts: 44
Joined: Fri Dec 02, 2011 1:11 am
Has thanked: 0 time
Been thanked: 0 time

Re: CICS transaction failures in production

 

Re: CICS transaction failures in production

Postby Robert Sample » Mon Nov 11, 2013 5:59 pm

1. It is a reasonable assumption that the abends cascaded from the first one.
2. What storage issue? Unitialized LINKAGE-SECTION variables are NOT a "storage issue" -- depending upon how the programs are coded, this could be a normal and expected event.
3. It sounds like your site has set LLACOPY=YES for this region; whether or not that is the correct value to use will depend upon site factors.

Unless and until you discuss this issue with your site support group, there is no way you are going to resolve it. The people responding on this forum do not work at your site, and hence we do not have the information nor the tools to be able to completely diagnose the problem. One possibility (among many): someone has put a LINKLIST library in the DFHRPL concatenation, the program in question was changed and the CICS NEWCOPY was not coordinated with the console LLA refresh command.
Robert Sample
Global moderator
 
Posts: 3126
Joined: Sat Dec 19, 2009 8:32 pm
Location: Bellevue, Iowa
Has thanked: 1 time
Been thanked: 185 times


Return to CICS

 


  • Related topics
    Replies
    Views
    Last post