Job Performance



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

Job Performance

Postby ajuatsgp » Fri Jun 08, 2012 10:57 pm

Hi,
I Have a job where I am doing JOIN in Sort.I have 8 JOIN steps.Is my job performance will decrease and run time will increase if I use many Joins in Single job?

Thanks
ajuatsgp
 
Posts: 82
Joined: Thu May 20, 2010 6:50 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Job Performance

Postby BillyBoyo » Fri Jun 08, 2012 11:55 pm

If you were to start out with one step in a job, then add one step at a time, each time the job would run for longer.

I guess you're not meaning that.

Is one step going to be affected by the fact that there are seven others in the job? No.

If all the data for all the steps were on the same volume, maybe letting them rip all at once in seperate jobs would introduce some delay, with read-heads flying about everywhere. Maybe.

You may want to split your job for scheduling purposes. Some other job may be able to kick-off when step 2 finishes, but has to wait until the end of the job, for instance.

OK, what is the real question?
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: Job Performance

Postby ajuatsgp » Sat Jun 09, 2012 4:27 am

To build my final output I nred all these Joins.
First two Join steps have 1file common.
3rd and 4th Join step have 1 file common.
.
.
7th and 8 Join step have 1 file common.
Is the performance of my.job will be bad amd will it take lot of time for
the job to get completed?
ajuatsgp
 
Posts: 82
Joined: Thu May 20, 2010 6:50 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Job Performance

Postby dick scherrer » Sat Jun 09, 2012 8:54 am

Hello,

Have you read about MERGE?

The resources used by this will depend on how much data has to be processed and how many times it must be passed.

How many records of what length are in the input files? How much output is expected?
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6268
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 93 times

Re: Job Performance

Postby NicC » Sat Jun 09, 2012 1:36 pm

How long a job takes depends on may factors - some of which are outside of your control e.g. total system workload. You say =you are running 8 SORT steps (DFSort or SYNCSORT?) - if you can merge these into one step you may save about 1 second due to step start/termination tasks. If you can reduce the number of actions to be taken by the sort product then your elapsed time will decrease accordingly. If you have not already done it well then doing it well will also save some time. But, with all the information that you have given, you might as well ask 'how long is a piece of string?'
The problem I have is that people can explain things quickly but I can only comprehend slowly.
Regards
Nic
NicC
Global moderator
 
Posts: 3025
Joined: Sun Jul 04, 2010 12:13 am
Location: Pushing up the daisies (almost)
Has thanked: 4 times
Been thanked: 136 times

Re: Job Performance

Postby ajuatsgp » Sat Jun 09, 2012 2:05 pm

We have both available,I am using DFSort.
Secondly as I said in all the 8 steps I am using JOIN(each time atleast one of the Joined file is different some times bothe the files are different from the previous step),so is it possible to merge them?
LRECL of my join output files are in the range of 90-150,my incoming files have bigger LRECL.
ajuatsgp
 
Posts: 82
Joined: Thu May 20, 2010 6:50 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Job Performance

Postby dick scherrer » Sat Jun 09, 2012 8:12 pm

Hello,

To get more useful replies, you need to answer all of the questons asked. . .
How may input records in all 8 files?
How much output - in how many files - intermediate files count also.

You need to post some sample input records and the desired output from these sample input records. You don't need to post whole records, just the fields relevant to your process.

Use the Code tag to preserve alignment and improve readability.
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6268
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 93 times


Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post