Creation of Dynamic Copy Job



JES, JES2, JCL utilities, IDCAMS, Compile & Run JCLs, PROCs etc...

Creation of Dynamic Copy Job

Postby pinakimishra » Thu Mar 07, 2019 7:23 am

I have a requirement where I have to copy around 10000 files daily. These files are created during different times of the day. And after the files are created, I have to create the copy. I can't modify the existing jobs which create the file to include a new copy step as I have to modify 10000 jobs. I don't want to create 10000 new jobs as successor jobs as these will be new scheduling effort as well as creation of jobs. I was considering the below 2 options but couple of issues I am not able to solve to reach the Solution.

1. Schedule a job (Job A )every 15 minutes which will read a file that has the 10000 filenames and write a Cobol program with INTRDR option which will create another job (Job B) with the copy steps on the fly and submit the job B. But I need another file to be input for my Cobol program which will tell me the jobs completed in the 15 mins so that I can include the files created in those jobs in my Job B. We have CA7 OPC Scheduler and I am not sure how to identify the jobs that completed in last 15 mins in batch. I can go to the scheduler through TSO Menu but not sure how I can get it realtime in batch.

2. Schedule a REXX job every 15 minutes which will read the files and get the creation date. If the creation date is equal to today's date then copy the file. It will work for GDG but I will face the issue when the file is still being written and I will try to copy it. Also for flat files that get overwriiten everyday it will have the original creation date but not today's date.

Any suggestions or other options will help.
pinakimishra
 
Posts: 7
Joined: Thu Mar 07, 2019 6:40 am
Has thanked: 0 time
Been thanked: 0 time

Re: Creation of Dynamic Copy Job

Postby expat » Thu Mar 07, 2019 12:45 pm

Have you spoken with your storage management team to see if these datasets are included in the daily backup process.

Have you considered using a DFdss incremnental process which uses and resets the DS1CHA bit of the VTOC - however DO NOT attempt this without first consulting with your storage management people as this may well bugger up the process indicated as the first option. Using DFdss will have its own inherent problems with regard to knowing exactly which dump has which backup on it.

Of course the knob heads that won't allow you to change the jobs and use the easy option - perhaps ask them for the solution :mrgreen:
expat
 
Posts: 459
Joined: Sat Jun 09, 2007 3:21 pm
Has thanked: 0 time
Been thanked: 8 times

Re: Creation of Dynamic Copy Job

Postby pinakimishra » Thu Mar 07, 2019 7:48 pm

I have talked to the storage management team for the copy and they can do it. But I have another step where I need to after the copy I have to count the number of records and ftp it to the distributed system for each file. Hence I need to do it through a job and the storage management copy will not help.
pinakimishra
 
Posts: 7
Joined: Thu Mar 07, 2019 6:40 am
Has thanked: 0 time
Been thanked: 0 time

Re: Creation of Dynamic Copy Job

Postby enrico-sorichetti » Thu Mar 07, 2019 9:25 pm

the most idiotic requirement I ever heard about :shock:
cheers
enrico
When I tell somebody to RTFM or STFW I usually have the page open in another tab/window of my browser,
so that I am sure that the information requested can be reached with a very small effort
enrico-sorichetti
Global moderator
 
Posts: 2994
Joined: Fri Apr 18, 2008 11:25 pm
Has thanked: 0 time
Been thanked: 164 times

Re: Creation of Dynamic Copy Job

Postby Robert Sample » Thu Mar 07, 2019 10:08 pm

CA-7 supports data set triggers. Setting them up for 10,000 data sets might be something of a pain but it would only need to be done one time.
Robert Sample
Global moderator
 
Posts: 3719
Joined: Sat Dec 19, 2009 8:32 pm
Location: Dubuque, Iowa, USA
Has thanked: 1 time
Been thanked: 279 times

Re: Creation of Dynamic Copy Job

Postby pinakimishra » Thu Mar 07, 2019 10:19 pm

Thanks Robert. Dataset trigger will be same like creating a new job Dependency. Also may be I can look that as an option but I will like to have one common job as a dataset trigger and perform the common option but there is no way I can get that as an i/p to my job.
pinakimishra
 
Posts: 7
Joined: Thu Mar 07, 2019 6:40 am
Has thanked: 0 time
Been thanked: 0 time

Re: Creation of Dynamic Copy Job

Postby Blackthorn » Tue Mar 12, 2019 3:33 pm

You said you have "CA7 OPC Scheduler ". Which is it? CA7 or OPC? They are two different products written by two different companies.

Whichever it is you could certainly write a batch utility to enquire what has completed in the last 15 minutes.

If it's OPC that you have, then I'm pretty sure (I haven't used it for a while) that you can specify the dataset triggering name using generic characters, that is, any dataset that matches a particular pattern will trigger the job. Similarly the actual triggering name can be passed in to the JCL as a variable. So you would only actually need one trigger definition and one job to process all 10,000 datasets.
Blackthorn
 
Posts: 130
Joined: Tue Feb 01, 2011 7:12 pm
Has thanked: 1 time
Been thanked: 9 times


Return to JCL

 


  • Related topics
    Replies
    Views
    Last post