Include records with special characters



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

Include records with special characters

Postby xcspg3 » Fri Aug 09, 2013 3:30 pm

I would like to extract from a file only records in which there is at least one character that is not a letter or a number or character point.
Records that contain only letters, numbers, and the point should not be written in output
The data set is RECFM=FB and LRECL=790.

Input
****** ***************************** Top of Data ******************************
000001 AAAAAA
000002 1233455
000003 ABC1234
000004 A.B
000005 A\BC
000006 ;:è
****** **************************** Bottom of Data ****************************

Output
****** ***************************** Top of Data ******************************
000001 A\BC
000002 ;:è
****** **************************** Bottom of Data ****************************

Thanks

Max
xcspg3
 
Posts: 29
Joined: Tue Sep 25, 2007 6:34 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Include records with special characters

Postby BillyBoyo » Fri Aug 09, 2013 4:27 pm

If all the characters you need it identify are "displayable", then just get a list of the ones you want. If any are non-displayable, get their values in hexadecimal.

It should then be simple to arrange a "substring search" to identify the records. Have a look at "SS" in the manual, and post your characters/values and what you have tried and its results, if you get stuck. Someone will be here to help.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: Include records with special characters

Postby xcspg3 » Fri Aug 09, 2013 9:58 pm

This is not an easy way because in function SS I must list all the characters and these are all characters not equal letters, numbers and point.

Max
xcspg3
 
Posts: 29
Joined: Tue Sep 25, 2007 6:34 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Include records with special characters

Postby BillyBoyo » Sat Aug 10, 2013 12:01 am

You either have a list of characters/values you don't want to have in your data, so that you can exclude records with some "program", or you don't, and then you are stuck using the editor and doing it "by hand". There isn't a "middle way".
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: Include records with special characters

Postby dick scherrer » Sat Aug 10, 2013 12:04 am

Hello,

Well, you can list the values you want or the values you want to omit.
Suggest you only list the valid values (letters, numbers, and point).
There are fewer of them and it should basically require little thought.

Why do you believe there must be an easy way?
Sometimes we have to actually do a bit of work . . .
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6268
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 93 times

Re: Include records with special characters

Postby xcspg3 » Mon Aug 12, 2013 12:12 pm

The problem is being able to include only records that have at least one character that is not a letter, number or point.
It 'easy to include the records with at least one invalid character with "SS" but the list of possible characters is too long.
In my opinion I think the solution is to make a Cobol program that scans the entire record.

thanks

Max
xcspg3
 
Posts: 29
Joined: Tue Sep 25, 2007 6:34 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Include records with special characters

Postby NicC » Mon Aug 12, 2013 1:30 pm

I do not know why you think that. Your COBOL program is going to have to do exactly what sort will have to do and COBOL will do it more slowly. All you need are, if you can only check one character at a time, is 63 (a-z,A-Z,.,0-9) omit conditions doing SS on the entire record - or field if you are only concerned with one field. You only have to construct the control cards once and that is easily done with ISPF.
The problem I have is that people can explain things quickly but I can only comprehend slowly.
Regards
Nic
NicC
Global moderator
 
Posts: 3025
Joined: Sun Jul 04, 2010 12:13 am
Location: Pushing up the daisies (almost)
Has thanked: 4 times
Been thanked: 136 times

Re: Include records with special characters

Postby BillyBoyo » Mon Aug 12, 2013 2:44 pm

It's going to take much longer to write and test it in COBOL. However, until you are happy enough with Sort Control cards, that might be a better way, if acceptable to whoever gave you the task. If not acceptable, then they need to provide extra training/support to you so that you can understand the Sort Control cards required.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: Include records with special characters

Postby skolusu » Mon Aug 12, 2013 9:34 pm

xcspg3,

Come September when DFSORT z/OS V2R1 is available you can perform Alphanumeric tests. Till then you can try the following approach.

1. Using INREC IFTHEN=(WHEN=INIT, copy the entire record to the end of the record ie. position 791.

2. Using another IFTHEN=(WHEN=INIT, use FINDREP with STARTPOS=791 and replace A-Z and 0-9 and . to C''. By doing so you will only be left with special characters.

3. Use OUTFIL INCLUDE=(791,1,CH,GT,C' ') and build only the first 790 Characters which is your input.
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
skolusu
 
Posts: 586
Joined: Wed Apr 02, 2008 10:38 pm
Has thanked: 0 time
Been thanked: 39 times

Re: Include records with special characters

Postby BillyBoyo » Mon Aug 12, 2013 10:46 pm

I happened to have a FINDREP with a big number of hex values (256, or all of them). Slight variation on Kolusu's method, the intention being to strain the CPU less...

Firstly, remove the X'nn' for all the values which are valid for you (paying attention to whether space is OK, for instance).

The FINDREP will operate zero (all characters in every position valid) or once (one character, somewhere, not valid). If one invalid character is found, the data on the record will be from that point on will be shifted one character to the left. So, due to the padding character which will be inserted in the last position of the record being space, the record will not be OMITted. If no invalid character is found, the C'1' will still be where it was placed in the OVERLAY, and the record will be OMITted on OUTFIL.


  INREC INREC=(WHEN=INIT,OVERLAY=(1592:1,1591,C'1')),

        IFTHEN=(WHEN=INIT,FINDREP=(IN=(X'00',
        X'01',X'02',X'03',X'04',X'05',X'06',X'07',X'08',X'09',
  X'0A',X'0B',X'0C',X'0D',X'0E',X'0F',X'10',X'11',X'12',X'13',
  X'14',X'15',X'16',X'17',X'18',X'19',X'1A',X'1B',X'1C',X'1D',
  X'1E',X'1F',X'20',X'21',X'22',X'23',X'24',X'25',X'26',X'27',
  X'28',X'29',X'2A',X'2B',X'2C',X'2D',X'2E',X'2F',X'30',X'31',
  X'32',X'33',X'34',X'35',X'36',X'37',X'38',X'39',X'3A',X'3B',
  X'3C',X'3D',X'3E',X'3F',X'40',X'41',X'42',X'43',X'44',X'45',
  X'46',X'47',X'48',X'49',X'4A',X'4B',X'4C',X'4D',X'4E',X'4F',
  X'50',X'51',X'52',X'53',X'54',X'55',X'56',X'57',X'58',X'59',
  X'5A',X'5B',X'5C',X'5D',X'5E',X'5F',X'60',X'61',X'62',X'63',
  X'64',X'65',X'66',X'67',X'68',X'69',X'6A',X'6B',X'6C',X'6D',
  X'6E',X'6F',X'70',X'71',X'72',X'73',X'74',X'75',X'76',X'77',
  X'78',X'79',X'7A',X'7B',X'7C',X'7D',X'7E',X'7F',X'80',X'81',
  X'82',X'83',X'84',X'85',X'86',X'87',X'88',X'89',X'8A',X'8B',
  X'8C',X'8D',X'8E',X'8F',X'90',X'91',X'92',X'93',X'94',X'95',
  X'96',X'97',X'98',X'99',X'9A',X'9B',X'9C',X'9D',X'9E',X'9F',
  X'A0',X'A1',X'A2',X'A3',X'A4',X'A5',X'A6',X'A7',X'A8',X'A9',
  X'AA',X'AB',X'AC',X'AD',X'AE',X'AF',X'B0',X'B1',X'B2',X'B3',
  X'B4',X'B5',X'B6',X'B7',X'B8',X'B9',X'BA',X'BB',X'BC',X'BD',
  X'BE',X'BF',X'C0',X'C1',X'C2',X'C3',X'C4',X'C5',X'C6',X'C7',
  X'C8',X'C9',X'CA',X'CB',X'CC',X'CD',X'CE',X'CF',X'D0',X'D1',
  X'D2',X'D3',X'D4',X'D5',X'D6',X'D7',X'D8',X'D9',X'DA',X'DB',
  X'DC',X'DD',X'DE',X'DF',X'E0',X'E1',X'E2',X'E3',X'E4',X'E5',
  X'E6',X'E7',X'E8',X'E9',X'EA',X'EB',X'EC',X'ED',X'EE',X'EF',
  X'F0',X'F1',X'F2',X'F3',X'F4',X'F5',X'F6',X'F7',X'F8',X'F9',
  X'FA',X'FB',X'FC',X'FD',X'FE',
                    X'FF'),OUT=C'',DO=1,STARTPOS=792,ENDPOS=1584))

  OUTFIL OMIT=(1585,1,CH,EQ,C'1'),
         BUILD=(1,1591)
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times


Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post