mainframe GDG overwrite issue

Post anything related to mainframes (IBM & UNISYS) if not fit in any of the above categories

mainframe GDG overwrite issue

Postby prog1234 » Sat Mar 12, 2011 3:51 am

We run a transaction processing system that uploads files from a unix server to a mainframe using NDM (Network DataMover).

The file uploads to a GDG base using a (+1), then it kicks off a job that copies it to another GDG base and then removes the initial one that was uploaded.

We are having a problem where we have two files get uploaded at the exact same second - in this case it appears that one file overwrites the other file.

One would think that the mainframe OS would see that one file was writing to the GDG base and hold the other one out or kick back a message that the GDG base is in use, and that does appear to be happening in some cases. In others it appears that it doesn't know and overwrites one file with the other.

Any suggestions on how to handle this. We have considered doing a round robin format and having more than one GDG base to upload to, but are looking for something less invasive as that would be a large coding effort.

Posts: 1
Joined: Sat Mar 12, 2011 3:29 am
Has thanked: 0 time
Been thanked: 0 time

Re: mainframe GDG overwrite issue


Re: mainframe GDG overwrite issue

Postby NicC » Sat Mar 12, 2011 12:58 pm

The bases do not get held - the datsets themselves should be locked if they have DISP=(NEW,CATLG,DELETE). If you are deleting the upload after copying why are you using a GDG?
If your JCL is really setup correctly then the simplest solution is to have each upload use its own file. This need not be deleted after copying as the next upload will simply overwrite the contents.
The problem I have is that people can explain things quickly but I can only comprehend slowly.
Global moderator
Posts: 2511
Joined: Sun Jul 04, 2010 12:13 am
Location: Down on the pig farm
Has thanked: 4 times
Been thanked: 86 times

Re: mainframe GDG overwrite issue

Postby steve-myers » Sat Mar 12, 2011 11:43 pm

GDGs and long running servers like an FTP server, and I presume an NDM server, that use dynamic allocation do not mix very well.

What happens is the first time a dynamic allocation that references a GDG the connection between the relative generation and the absolute generation is locked in for the duration of the long running function.

In other words, if you have


when you start


will translate to


If another GDG(+1) request comes in 30 minutes later (or 24 hours later, for that matter) it will still be GDG.G0238V00, not GDG.G0239V00 if the same FTP server (or NDM server) are still running.
Global moderator
Posts: 1772
Joined: Thu Jun 03, 2010 6:21 pm
Has thanked: 4 times
Been thanked: 168 times

Re: mainframe GDG overwrite issue

Postby MrSpock » Mon Mar 14, 2011 9:13 am

To the best of my knowledge, though I'm not an expert on this subject, I believe that NDM places an enqueue on the GDG base when the process starts, using the same method that JES does, and keeps that enqueue until the dataset is closed. Logically, I would think that any subsequent processes using that same GDG base would be placed in the hold queue until the enqueue was released, so I find it hard to understand how this situation could happen.

But, I do agree with Nic. If you aren't using the GDG's for their real purpose as data repositories, then why use a GDG? You might as well use just regular, uniquely-named datasets. Maybe use a template where the dataset names are the same, except add a node of the process number (that's unique) to make them different enough.
User avatar
Global moderator
Posts: 811
Joined: Wed Jun 06, 2007 9:37 pm
Location: Raleigh NC USA
Has thanked: 0 time
Been thanked: 2 times

Re: mainframe GDG overwrite issue

Postby MrH » Mon Mar 14, 2011 6:09 pm

I don't think that's entirely correct, MrSpock. There shouldn't be an enqueue against the GDG base, because a GDG base is nothing more than a catalog entry.

I think what's happening here is that one request is being sent to the FTP server, which dynamically allocates a new generation. Before dynamic deallocation is complete, the FTP server receives an identical request, and it interprets that request as being for the same generation it's already using.

OR... and this is what I think is more likely... the new generation isn't cataloged until the dataset is deallocated. Therefore, if you have two different threads running, the first one is dynamically allocated as GDG.G0238V00, and the next request hits the catalog, and since GDG.G0238V00 hasn't been cataloged yet, that's the DSN the catalog returns for the next generation. If that's the case, you might have to run VTOC queries to see where the uncataloged version is, because chances are it didn't get overwritten, it's just orphaned somewhere.

I don't make any claims to expertise in this area, either. But I have had some experiences with this sort of thing.

I do agree that the best solution would be to key off of some other unique property and abandon the use of GDGs for this process. Date and time stamps are usually a good place to start.

Re: mainframe GDG overwrite issue

Postby Robert Sample » Mon Mar 14, 2011 6:14 pm

From the JCL Reference manual, section 12.19 (empahsis added):
| DISP and ENQ for generation data sets: The way the initator issues an ENQ
| to control generation data sets can be different than with other data
| sets. The initiator only issues the ENQ for the GDG base name for a
| generation data set that is referenced by either:

| Their relative GDG names (for example, DSN=TEST.GDG.DATASET(0)).

| As GDG ALLs (for example, DSN=TEST.GDG.DATASET)

| For example, the initiator issues the ENQ for the GDG base name,
| TEST.GDG.DATASET for the generation data sets shown in the list above.
| Generation data sets referenced by either their relative GDG names or as
| GDG ALLs are processed this way because the initiator does not know which
| specific absolute generation names will be required. This is because the
| conversion from relative generation name to absolute generation name is
| done during the allocation for the step referencing the data set. The ENQ
| for the specific, absolute, generation (G0000V00) data set name or names
| is issued at the start of the step requesting the relative GDG or GDG ALL.

| Note that the initiator does not issue an ENQ for the GDG base name for a
| generation data set that is referenced by its absolute GDG name. Instead
| it issues an ENQ for that specific G0000V00 data set name.

Robert Sample
Global moderator
Posts: 3099
Joined: Sat Dec 19, 2009 8:32 pm
Location: Bellevue, Iowa
Has thanked: 1 time
Been thanked: 176 times

Re: mainframe GDG overwrite issue

Postby steve-myers » Mon Mar 14, 2011 6:23 pm

MrH wrote:... OR... and this is what I think is more likely... the new generation isn't cataloged until the dataset is deallocated.

Dynamic allocation and SMS allocation update the catalog at the time of allocation.
Global moderator
Posts: 1772
Joined: Thu Jun 03, 2010 6:21 pm
Has thanked: 4 times
Been thanked: 168 times

Return to All other Mainframe Topics


  • Related topics
    Last post