IBM Mainframe Forum

Hello,



We have a VB input file (relevant attrib given below) which has over 60 mil records in xml layout (this dataset is a combined unsorted data pool of over 50 datasets from different sources). These records are a mix of variable and fixed format records, meaning - in case of fixed format, data is present at a specific location controlled by record layout copybooks, au-contraire in case of variable layout records, data is free-form. Variable record data is built by reference modification and padding of xml tags.



Both the aforementioned record types present in this dataset have an identifier (259,8 --> for fixed layout; preceding </idrequest> tag for variable layout). The aim is to extract unique identifiers records from the input file.

Current approach:
a. Since the input is variable length, hence extend the records temporarily towards left.
b. Dump the identifier in this extended field
c. Once INREC processing is completed, SORT the data and apply SUM FIELD=NONE on the extended key

Problem with this approach:
a. I feel that this code can be made better- much better; it is very clunky at the moment.
b. If the record count is increased by even a mil, the SORTing goes haywire
c. To fix the problem in point-b, a two step approach is taken, i.e. build the INREC data first; and then have another step which SORTs and SUM FIELDS the key position.

Any suggestions on trimming the above SORT card are much appreciated.

PS: The SKIPREC is to avoid a junk chunk of data, and can be ignored. The 'XXXX' and 'YYYY' are identifiers that segregate the fixed/variable records. Xs are variable records, Ys are fixed records.

Thank you.

OK, several things. Firstly, unless you are leaving "gaps" (which will be automatically filled with blanks) it is only a headache to include column-numbers in a BUILD. In all your uses, the data naturally occupies the next available position. For instance:



You have a PARSE for %02, but you don't use it. So not needed at all.

Unless using OUTFIL reporting features, you don't get given a CC, so no need for a REMOVECC. The default fill is blank, so no need for the VLFILL.

The SORT is not doing what you think. It does not matter the order you put the control cards in, SORT will process them exactly how they should be processed. In your case OMIT COND, INREC, SORT, SUM, OUTFIL. Which means your SORT key is wrong, you need it to be 5,8.

I assume, since you have no other treatment for otherwise, that XXXX and YYYY are the only things you can have. Which means you can use WHEN=NONE rather than the second condition (so like an ELSE/WHEN OTHER).

Are your keys contiguous on the input data? If so, you don't need the SORT and SUM, you can instead actually use the OUTFIL reporting features, with SECTIONS and TRAILER3.

Some representative sample data and expected output (shortened data) would help to see what you are expecting to happen.

BillyBoyo wrote:Which means your SORT key is wrong, you need it to be 5,8.

Thank you Billy; my bad for totally goofing up the SORT key, missed it in the whole mess, forgetting that OUTFIL is processed at end.
The PARSE for %02 is leftover code from a previous test run I'll clean it up.

VLFILL- yup, that should definitely save me a bunch, thank you.
A specific check for 'YYYY' was added to avoid junk data occurrence for fixed length records; which had conked up during the initial runs with WHEN=NONE - which was originally coded.

Have added the sample input/output records for variable type data below (the three lines in the text pasted are but one - a single continuous record spanned across 30k LRECL). The position/length of 'VALUE NEEDED AS SORT KEY' can change depending upon the type of data that is being pushed in. The length of the complete record will also vary depending upon the type of records in question.
In output, this key needs to come at the start of the record, so that SORTing can be carried out on this data.



Lastly, completely agree on 'it is only a headache to include column-numbers in a BUILD'; it has become a habit of sorts to add the column; will work on fixing it

Once again, thank you for the help.