Various record counts from the same dataset



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

Various record counts from the same dataset

Postby mfrookie » Fri Jan 27, 2012 7:34 pm

Hi,

I have a requirement where in I have to obtain number of record counts from the same dataset based on different conditions. I would like this to be done in a single pass.

e.g.

HI-1
HI-2
HI-3
HI-1
HI-2
HI-5
HI-3
HI-1
HI-2
HI-3
HI-8
HI-8
HI-1
HI-2
HI-3
HI-3
HI-1
HI-2
HI-3
HI-1
HI-2
HI-3
HI-1
HI-2
HI-3


I need the total number of record for below conditions but in the same output dataset.

1) Total number of records (INCLUDE=ALL)
2) Total number of records with '1' in position 4 (INCLUDE=(4,1,CH,EQ,C'1')
2) Total number of records with '2' in position 4 (INCLUDE=(4,1,CH,EQ,C'2')
2) Total number of records with '3' in position 4 (INCLUDE=(4,1,CH,EQ,C'3')

Expected output
RECORD COUNT  : 0000000000000025
RECORD COUNT1 : 000000000000007
RECORD COUNT2 : 000000000000007
RECORD COUNT3 : 000000000000008


I tried it with COPY operator and 4 different OUTFIL datasets with output dataset as MOD, but I am just getting the last count which is for condition INCLUDE=(4,1,CH,EQ,C'3'). SORT does not allow same DD statement to be used for OUTFIL (which is quite logical).

OUTFIL FNAMES=OUTPUTFI,REMOVECC,NODETAIL,                           
       INCLUDE=ALL,                                                 
       TRAILER1=(1C'RECORD COUNT1 : ',COUNT=(M11,LENGTH=20))       
OUTFIL FNAMES=OUTPUTF1,REMOVECC,NODETAIL,                           
       INCLUDE=(4,1,CH,EQ,C'1'),                                   
       TRAILER1=(1C'RECORD COUNT2 : ',COUNT=(M11,LENGTH=20))       
OUTFIL FNAMES=OUTPUTF2,REMOVECC,NODETAIL,                           
       INCLUDE=(4,1,CH,EQ,C'2'),                                   
       TRAILER1=(1C'RECORD COUNT3 : ',COUNT=(M11,LENGTH=20))       
OUTFIL FNAMES=OUTPUTF3,REMOVECC,NODETAIL,                           
       INCLUDE=(4,1,CH,EQ,C'3'),                                   
       TRAILER1=(1C'RECORD COUNT4 : ',COUNT=(M11,LENGTH=20))       


What I am getting in the output dataset -

RECORD COUNT4 :      000000000000008       


Is there any way to achieve the results in a single pass, in the same dataset.

Thanks
mfrookie
 
Posts: 40
Joined: Mon Apr 25, 2011 8:46 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Various record counts from the same dataset

 

Re: Various record counts from the same dataset

Postby BillyBoyo » Fri Jan 27, 2012 8:31 pm

All your OUTFIL datasets will be open at the same time. I don't think the DISP=MOD will work like that.

If you can wait until tommorow/Monday I'm sure you'll get a good solution. In the meantime, ensure that you have provided all the details necessary for a solution, including RECFM/LRECL.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 264 times

Re: Various record counts from the same dataset

Postby mfrookie » Fri Jan 27, 2012 8:40 pm

Thanks Bill.

I will wait till tomorrow. Please assume that the input and output files are FB with LRECL as 80 bytes.

Meantime if anyone has any other solution, please do suggest.

Thanks.
mfrookie
 
Posts: 40
Joined: Mon Apr 25, 2011 8:46 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Various record counts from the same dataset

Postby BillyBoyo » Fri Jan 27, 2012 8:55 pm

If you are looking to crack on with it, good for you.

I've just searched the forum with REMOVECC and looked at some solutions frovided by Frank and Kolusu, picking likely ones from the topic subjects. I'd use one or more of those as a starting point.

Frank and Kolusu, DFSORT developers, visit this forum regularly in San Jose, Calif. working hours. One of them will give you an optimal solution, but I'd encourage you to work at it before then for what you'll learn, and then continue the learning from the solution provided.

It's how I do it :-)

My time is short today. When I do have a crack at these, they tend to be so far away from a "good" solution that it keeps me wondering :-)
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 264 times

Re: Various record counts from the same dataset

Postby mfrookie » Fri Jan 27, 2012 9:21 pm

I am going thru various threads to find out the answer. But I do not want to sort the file as this will add overhead to processing and some of the files we have are quite big. If we were to SORT the files, then probably I could have got the answer using OCCURS operator or with different options of report writing. But we will be running the count process on thousands of files to get record counts, based on different conditions. The conditions and the count processing can very well be added to the existing applications, but that will take time.

The objective is to have a quick process which will present record counts of different datasets, OR the same dataset with different conditions. So let's say if I have 4 types of counts to be obtained as shown in the exmaple above, I would like it to be obtained in a single pass, rather than going over the same again and again.

I did try COUNT operator, and it works well for the scenario where the count is to be obtained for all records or if there is only one condition. But I can not apply different conditions in the same pass to obtain the different counts. I will be able to apply a condition by specifying it in the CNTL card in the USING option of COUNT. But then what about the counts that are to be derived based on other conditions.

I could also add another COUNT or COPY operator and write the record counts to the same output dataset by using MOD and that works well (the usual technnique we follow when using SPLICE), but then that means the number of passes will be equal to the number of conditions I have.

Thanks for the information about the work timings of Frank and Kolusu. I do know that they are based on west coast, I will wait for them to reply.

Thanks.
mfrookie
 
Posts: 40
Joined: Mon Apr 25, 2011 8:46 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Various record counts from the same dataset

Postby skolusu » Fri Jan 27, 2012 10:08 pm

mfrookie,

Use the following DFSORT JCL which will give you the desired results. I assumed all the records have 'H' in the first position.
//STEP0100 EXEC PGM=SORT                                   
//SYSOUT   DD SYSOUT=*                                     
//SORTIN   DD *                                             
----+----1----+----2----+----3----+----4----+----5----+----6
HI-1                                                       
HI-2                                                       
HI-3                                                       
HI-1                                                       
HI-2                                                       
HI-5                                                       
HI-3                                                       
HI-1                                                       
HI-2                                                       
HI-3                                                       
HI-8                                                       
HI-8                                                       
HI-1                                                       
HI-2                                                       
HI-3                                                       
HI-3                                                       
HI-1                                                       
HI-2                                                       
HI-3                                                       
HI-1                                                       
HI-2                                                       
HI-3                                                       
HI-1                                                       
HI-2                                                       
HI-3                                                       
//SORTOUT  DD SYSOUT=*                                     
//SYSIN    DD *                                             
  INREC IFTHEN=(WHEN=INIT,OVERLAY=(10:7C'0',C'1',24C'0')), 
  IFTHEN=(WHEN=(4,1,CH,EQ,C'1'),OVERLAY=(25:C'1')),         
  IFTHEN=(WHEN=(4,1,CH,EQ,C'2'),OVERLAY=(33:C'1')),         
  IFTHEN=(WHEN=(4,1,CH,EQ,C'3'),OVERLAY=(41:C'1'))         

  SORT FIELDS=(1,1,CH,A)                                   
  SUM FIELDS=(10,8,18,8,26,8,34,8),FORMAT=ZD               

  OUTFIL BUILD=(C'RECORD COUNT  : ',10,8,/,           
                C'RECORD COUNT1 : ',18,8,/,           
                C'RECORD COUNT2 : ',26,8,/,           
                C'RECORD COUNT3 : ',34,8,80:X)       
//*   
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
skolusu
 
Posts: 586
Joined: Wed Apr 02, 2008 10:38 pm
Has thanked: 0 time
Been thanked: 39 times

Re: Various record counts from the same dataset

Postby mfrookie » Fri Jan 27, 2012 10:23 pm

Thanks Kolusu.

Just wanted to know if there is any other way to achieve the results without SORTing the file as that will add unnecessary overhead to the processing time. I will be running the same process over different files with different conditions. SO I was looking for an approach without explicitly SORTing the records. Once I have the approach , I should be able to customize it for different files.
mfrookie
 
Posts: 40
Joined: Mon Apr 25, 2011 8:46 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Various record counts from the same dataset

Postby skolusu » Fri Jan 27, 2012 10:44 pm

mfrookie,

Use the following control cards which use reporting features to get the desired results
//SYSIN    DD *                                               
  SORT FIELDS=COPY                                           
  INREC IFTHEN=(WHEN=INIT,OVERLAY=(10:3C'0')),               
  IFTHEN=(WHEN=(4,1,CH,EQ,C'1'),OVERLAY=(10:C'1')),           
  IFTHEN=(WHEN=(4,1,CH,EQ,C'2'),OVERLAY=(11:C'1')),           
  IFTHEN=(WHEN=(4,1,CH,EQ,C'3'),OVERLAY=(12:C'1'))           
                                                             
  OUTFIL REMOVECC,NODETAIL,                                   
  TRAILER1=('RECORD COUNT  : ',COUNT=(M11,LENGTH=8),/,       
            'RECORD COUNT1 : ',TOT=(10,1,ZD,M11,LENGTH=8),/, 
            'RECORD COUNT2 : ',TOT=(11,1,ZD,M11,LENGTH=8),/, 
            'RECORD COUNT3 : ',TOT=(12,1,ZD,M11,LENGTH=8))   
//*                                                           
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
skolusu
 
Posts: 586
Joined: Wed Apr 02, 2008 10:38 pm
Has thanked: 0 time
Been thanked: 39 times

Re: Various record counts from the same dataset

Postby mfrookie » Fri Jan 27, 2012 10:47 pm

Thanks a lot kolusu.

It satisfies what I wanted.

Thanks once again.
mfrookie
 
Posts: 40
Joined: Mon Apr 25, 2011 8:46 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Various record counts from the same dataset

Postby Frank Yaeger » Fri Jan 27, 2012 11:46 pm

Just wanted to know if there is any other way to achieve the results without SORTing the file as that will add unnecessary overhead to the processing time.


Note that you could also use Kolusu's first job with MERGE and SORTIN01 instead of SORT and SORTIN. MERGE will not add unnecessary overhead.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1080
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 14 times

Next

Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post