Reduce the consummation of CPU of the job



JES, JES2, JCL utilities, IDCAMS, Compile & Run JCLs, PROCs etc...

Reduce the consummation of CPU of the job

Postby samb01 » Mon Jan 13, 2014 3:24 pm

Hello,

i am going to try to explain a difficult problem with my bad english.

After the end of the TP (07h00) pm, the batch begin. About 100 jobs à the same time. My job is to find a solution to reduce the consommation CPU of these jobs to prevent the capping of the machine.

I am thinking about ths solution :

To begin 20 at 07h00 pm, and 20 jobs at 08h00 and 20 jobs at 09h00 pm etc... Do you think it's a good solution ? Or would you have ideas to help me. I am just in the department Production, so i can't do every things.

An other important thing : We have two partition :

one where the jobs can run during the day (during the TP) but this parition is not very powerfull.
and we have an other partition where the jobs can only run after the TP (because this parition is reserved to the TP during the day). But after 07h00 pm, we can use this partition which is very powerfull (5 à 7 more then the other) to run the job until 06h00 am the next day.

I don't know if i successed to make me andestand but i tried.

Thank's for your help.
samb01
 
Posts: 427
Joined: Mon Nov 16, 2009 7:24 pm
Has thanked: 1 time
Been thanked: 0 time

Re: Reduce the consommation of CPU of the job

Postby BillyBoyo » Mon Jan 13, 2014 3:42 pm

When you say Partition, do you mean LPAR or Initiator?
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: Reduce the consommation of CPU of the job

Postby steve-myers » Mon Jan 13, 2014 4:20 pm

  1. Go through the jobs. Find the job with the highest CPU usage.
  2. For the job with the highest CPU usage, analyze the steps. Select the step with the highest CPU usage.
  3. What does this step do? Is it really necessary? Does anyone actually use the reports from the step? Does the step produce data required for further processing? What I'm getting at, what will happen if the step is no longer run? Can this step be eliminated?
  4. If the step must be run, what does the step do? Steps that run your sort product are frequently the higher CPU users. There are often things you can do to reduce CPU utilization in sort. Do them.
  5. If the step must be run, if the program is a user supplied program, analyze the program to determine where it is using CPU. You will need products like Strobe - granted a very expensive product - to do this analysis. If possible, repair the program.
  6. Continue this analysis until you have achieved your goal.
No one can give you a magic bullet to achieve this goal. It will take a lot of work.
steve-myers
Global moderator
 
Posts: 2105
Joined: Thu Jun 03, 2010 6:21 pm
Has thanked: 4 times
Been thanked: 243 times

Re: Reduce the consommation of CPU of the job

Postby Ed Goodman » Mon Jan 13, 2014 8:17 pm

Another thing I notice is that you've already decided the problem is CPU usage. Are you CERTAIN that is what you need to fix? Is you mainframe REALLY getting all of it's CPU taken up, or are the jobs just running a long time?

Also, and this crucial, are you sure that it's YOUR jobs causing the issue? We had a similar request come down from some meeting and it turned out to be the new DB2/SQL server that had been moved to the new partition. We could use a product called Omegamon to actually point to it and say 'Look, that one task is using 85% of the CPU power.' Once they tuned that, the problem went away.

After that, Steve's recommendations are really it. Just make sure that you are truly concentrating on the jobs that actually use a lot of CPU, not the jobs that have a high CPU percentage. I've seen people work for weeks on reducing a 1 minute job to 30 seconds. Meanwhile a 3 hour job goes unchanged because it had a lower CPU percentage.
Ed Goodman
 
Posts: 341
Joined: Thu Feb 24, 2011 12:05 am
Has thanked: 3 times
Been thanked: 17 times

Re: Reduce the consummation of CPU of the job

Postby samb01 » Tue Jan 14, 2014 2:12 pm

Hello and thank's for your help.
I am so pleased because i successed to make me anderstand :D .
You anderstood my job. I know it's really difficult to solve a problem like it.
Steve-myers, i agree with you. it's a very had and a long time work. I have already study un job which use a lot of CPU. And i suggest solutions about this job to reduce the use of CPU. And this took me a very long time only for a job... So i know i have to do it whith the other jobs whith highest CPU usage.

Now, i am thinking about a solution with the automate TWS. Actually, the jobs run at the same time. (07h00 pm). I think it's not very good for the CPU. So i want to run job ten by ten (for example, or 20 by twenty) every hour. Mays be it's better for the CPU and the capping of the machine ?

On other idea si to take the ten jobs with the highest CPU usage and run it at different time (not at the same time like actually).

Ed Goodman, thank's for your usefull advices. Your experience is really important for me. I loved your example with your minute job ;) (it's so true)

A think i didn't anderstood : What is the difference between

"Jobs using a lot CPU"

and

"Job with high CPU percentage" ?

Tank's for your help.
samb01
 
Posts: 427
Joined: Mon Nov 16, 2009 7:24 pm
Has thanked: 1 time
Been thanked: 0 time

Re: Reduce the consummation of CPU of the job

Postby steve-myers » Tue Jan 14, 2014 4:49 pm

  • "Jobs using a lot of CPU" is a raw measure. See the IEF376I message near the end of the JESYSMSG data set. You can get step CPU usage in the IEF374I message in the JESYSMSG data set.
  • "Jobs with high CPU percentage" refers to CPU usage over a period of time. A job using 20 minutes of CPU time over 20 minutes elapsed time is using 100% of the CPU. This is sometimes important, though it may be meaningless if the LPAR where the job is running is capped.
steve-myers
Global moderator
 
Posts: 2105
Joined: Thu Jun 03, 2010 6:21 pm
Has thanked: 4 times
Been thanked: 243 times

Re: Reduce the consummation of CPU of the job

Postby Robert Sample » Tue Jan 14, 2014 6:44 pm

Moving some of the jobs to start at different times may -- or may not -- have any impact; the answer depends upon many factors AT YOUR SITE. The only way to know for sure is to make the change and see what happens.

If a job has 100% CPU usage, but only runs for 1 minute -- you are not going to care about this job even though it has high CPU percentage. A job that has 5% CPU usage but runs for three hours will use over 500 seconds of CPU time (compared to the 60 seconds of CPU time for the other job); you may want to look at this job due to its high CPU use even though the CPU percentage is low.
Robert Sample
Global moderator
 
Posts: 3719
Joined: Sat Dec 19, 2009 8:32 pm
Location: Dubuque, Iowa, USA
Has thanked: 1 time
Been thanked: 279 times

Re: Reduce the consummation of CPU of the job

Postby samb01 » Tue Jan 14, 2014 7:46 pm

Ok, thank you. It's really interesting.

I have the statistique of using CPU Batch since the beginning of the month. I can go farther (december 2013; november 2013). But i want to see this month (January).

Each day we h ave have a statistique about the sue of CPU by jobs of our production.

I don't want to put these documents on the forum because it is quite offcial document.Is there a way to send you an example of our statistics. You will have a better idea about the use of CPU by jobs.

thank's for your help.
samb01
 
Posts: 427
Joined: Mon Nov 16, 2009 7:24 pm
Has thanked: 1 time
Been thanked: 0 time

Re: Reduce the consummation of CPU of the job

Postby dick scherrer » Tue Jan 14, 2014 8:04 pm

Hello,

Is there a way to send you an example of our statistics. You will have a better idea about the use of CPU by jobs.
Sure, just copy/paste the captured info and change the jobname to something generic rather than something that should not be shared.

Then we can see some of your specifics.
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6268
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 93 times

Re: Reduce the consummation of CPU of the job

Postby Ed Goodman » Tue Jan 14, 2014 8:38 pm

Do you have access to any system-level measuring tools? Again, you may be fixing the 'wrong' problem if you are not able to see the entire machine's CPU usage.
Ed Goodman
 
Posts: 341
Joined: Thu Feb 24, 2011 12:05 am
Has thanked: 3 times
Been thanked: 17 times

Next

Return to JCL