RECFM=U|V



IBM OS/370, MVS, OS/390, Linux, TPF, VM/CMS, VM/ESA, VSE/ESA, z/VM, z/VSE, z/OS, z/OS.e etc...

RECFM=U|V

Postby trengri » Sat Nov 17, 2018 11:32 pm

I'm studying mainframe concepts and cannot understand why RECFM=U and RECFM=V both exist.
As far as I understand, "V" format is like "U", except that it stores the record length in the first few bytes of the record (please correct me if I'm wrong). But why would anyone want to store the record length in the first few bytes, if the record length could be determined without it (interblock gaps are always there)?
So it's unclear to me why both of these formats exist, one of them is redundant to my mind.
trengri
 
Posts: 9
Joined: Tue Sep 25, 2018 2:12 am
Has thanked: 8 times
Been thanked: 0 time

Re: RECFM=U|V

Postby steve-myers » Sun Nov 18, 2018 2:22 am

No, they are not redundant; they serve slightly different purposes.

The primary - nearly exclusive - use of RECFM U data sets is for data used for binary programs, where the entire data area is binary program text, and the length data in Block Descriptor Words and Record Descriptor would be a hindrance to loading binary program text.

A secondary use of RECFM U is to allow an entry point for data structures that are not really defined in the operating system so that the user program can still use the data, probably after the data has been massaged by the user program to bring it into conformance with "standard" record types.

System/360 marked the transition from 6-bit bytes to 8-bit bytes, and RECFM U was often used to bring 6 bit data into 8 bit data, often aided by 7 track tape drives to bring 6-bit bytes into the 8-bit byte world. The need for this capability faded out in the 1980s.
steve-myers
Global moderator
 
Posts: 2086
Joined: Thu Jun 03, 2010 6:21 pm
Has thanked: 4 times
Been thanked: 235 times

Re: RECFM=U|V

Postby trengri » Sun Nov 18, 2018 3:42 am

Thank you.
Could not even imagine it was used to handle 6-bit bytes. The usage of RECFM U for load modules is fully understandable.

And what RECFM V is used for?
To my mind, RDW is always a hindrance, regardless of what the program is doing. The program should already know the record length after it has read it, so why deal with these RDWs?
trengri
 
Posts: 9
Joined: Tue Sep 25, 2018 2:12 am
Has thanked: 8 times
Been thanked: 0 time

Re: RECFM=U|V

Postby Robert Sample » Sun Nov 18, 2018 5:35 am

And what RECFM V is used for?
To my mind, RDW is always a hindrance, regardless of what the program is doing. The program should already know the record length after it has read it, so why deal with these RDWs?
There are plenty of times when variable length records make sense -- I had a system once that contained about 950 bytes of fixed data, and then anywhere from 0 to 150 occurrences of a 200-byte area, so the minimum record length was 950 bytes and the maximum was over 30,000 bytes. If fixed length records had been used, thousands of bytes would have been wasted for most of the records (the average number of occurrences was down around 20 or 25).

You have a MAJOR misunderstanding -- a program knows the record length after reading the record by looking at the RDW and ONLY by looking at the RDW. Otherwise, since data sets are almost always blocked, how does the program know 370 bytes of the 27,998 byte block represent the first record? The RDW is the ONLY way for the program to know this.

These users thanked the author Robert Sample for the post:
trengri (Sun Nov 18, 2018 3:33 pm)
Robert Sample
Global moderator
 
Posts: 3657
Joined: Sat Dec 19, 2009 8:32 pm
Location: Dubuque, Iowa, USA
Has thanked: 1 time
Been thanked: 270 times

Re: RECFM=U|V

Postby steve-myers » Sun Nov 18, 2018 7:10 am

Robert Sample wrote:
And what RECFM V is used for?
To my mind, RDW is always a hindrance, regardless of what the program is doing. The program should already know the record length after it has read it, so why deal with these RDWs?
There are plenty of times when variable length records make sense -- I had a system once that contained about 950 bytes of fixed data, and then anywhere from 0 to 150 occurrences of a 200-byte area, so the minimum record length was 950 bytes and the maximum was over 30,000 bytes. If fixed length records had been used, thousands of bytes would have been wasted for most of the records (the average number of occurrences was down around 20 or 25).

You have a MAJOR misunderstanding -- a program knows the record length after reading the record by looking at the RDW and ONLY by looking at the RDW. Otherwise, since data sets are almost always blocked, how does the program know 370 bytes of the 27,998 byte block represent the first record? The RDW is the ONLY way for the program to know this.

For many years I followed Mr. Sample's advice when reading RECFM V records and used the RDW to obtain the record length. Then, replying to a post either here or on IBM mainframes followed by some testing I quickly determined that DCBLRECL after GET works just as well as the RDW.

Back in the 1970s I switched over to RECFM V for SYSPRINT type data after the common observation that most lines are smaller than 120 or 133 bytes. Much smaller. Writing blanks at the end of a line is a waste of media and I/O time. Granted reading RECFM V data is a bit of a pain in the neck compared to RECFM F data, but I've been doing this for so long switching my brain to reading RECFM F data causes a few cob webs to fly away.

These users thanked the author steve-myers for the post:
trengri (Sun Nov 18, 2018 3:32 pm)
steve-myers
Global moderator
 
Posts: 2086
Joined: Thu Jun 03, 2010 6:21 pm
Has thanked: 4 times
Been thanked: 235 times

Re: RECFM=U|V

Postby trengri » Sun Nov 18, 2018 3:38 pm

Robert Sample wrote:You have a MAJOR misunderstanding -- a program knows the record length after reading the record by looking at the RDW and ONLY by looking at the RDW. Otherwise, since data sets are almost always blocked, how does the program know 370 bytes of the 27,998 byte block represent the first record? The RDW is the ONLY way for the program to know this.

Thank you for pointing this out. Surely I have misunderstanding, I am just trying to grasp the concepts, I haven't written a single line of code for mainframes yet.
If the records are blocked (RECFM=VB), then RDW is necessary, no questions.
But I was asking about RECFM=V, which is not blocked, so the program should be able to determine the length without RDW, the same way like RECFM=U does (I assume that it uses interblock intervals for that).
trengri
 
Posts: 9
Joined: Tue Sep 25, 2018 2:12 am
Has thanked: 8 times
Been thanked: 0 time

Re: RECFM=U|V

Postby steve-myers » Sun Nov 18, 2018 6:46 pm

trengri wrote:
Robert Sample wrote:You have a MAJOR misunderstanding -- a program knows the record length after reading the record by looking at the RDW and ONLY by looking at the RDW. Otherwise, since data sets are almost always blocked, how does the program know 370 bytes of the 27,998 byte block represent the first record? The RDW is the ONLY way for the program to know this.

Thank you for pointing this out. Surely I have misunderstanding, I am just trying to grasp the concepts, I haven't written a single line of code for mainframes yet.
If the records are blocked (RECFM=VB), then RDW is necessary, no questions.
But I was asking about RECFM=V, which is not blocked, so the program should be able to determine the length without RDW, the same way like RECFM=U does (I assume that it uses interblock intervals for that).

Sorry. RECFM=V has 1 BDW and 1 RDW in each physical record. When I discovered this unexpected fact back in the 1960s (or was it the early 1970s?) I have to admit I was confused. Looking back on it I guess the intent was to make RECFM V readers consistent.

By the way, inter block intervals have nothing to do with length determination. On disk, block length is an attribute of the block. Only on magnetic tape do the inter block gaps affect reading, and, by implication data length determination.

As I said, on disk, block length is, in effect, an attribute of the block. The BDW is still there, but the block length definition specified by the disk hardware rules. Assembler programmers using EXCP can dig out the hardware length definition for a block, but there are few Assembler programmers that actually know EXCP and know how to dig out the record length.

 ** COUNT 00A2000001006D5E **                      
                                                                   
0000     0  6D5E0000 01560000  1E0E007E 57C80118  *_;.........=.H..*

The count area has the record length (6D5E) as well as other data. Technically, the count area is defined by the hardware, but the program creating the record also creates the associated count area. For most programs this is done under the covers as it were.

Jumping to the record itself we see the BDW (6D5E0000) immediately followed by the first RDW (01560000) in the physical record. The 0000 0 to the left of the data area are the hexadecimal and decimal offset of the data in the line; they are there for the convenience of the analyst examining the data.

As Mr. Sample says, most programs reading RECFM V data only see the BDW and the data that follows the BDW. In fact, there are obscure DCB definitions that can be used so the user program does not even see the BDW! But that's best left for another topic.
steve-myers
Global moderator
 
Posts: 2086
Joined: Thu Jun 03, 2010 6:21 pm
Has thanked: 4 times
Been thanked: 235 times

Re: RECFM=U|V

Postby Robert Sample » Sun Nov 18, 2018 7:08 pm

RECFM=U has a Block Descriptor Word (BDW) which contains the length of the block. RECFM=V also has a BDW, but it also has an RDW.
so the program should be able to determine the length without RDW, the same way like RECFM=U does (I assume that it uses interblock intervals for that).
You assume wrong. Interblock gaps have NOTHING to do with block lengths -- the block length for RECFM=U, RECFM=V, RECFM=VB data sets is determined by the BDW which is the first 4 bytes of the physical block. Technically the RDW in a RECFM=V block is not required, but if it is not present than the operating system routine that reads records must have different logic for RECFM=V and RECFM=VB data sets since VB data sets have to have the RDW. I would expect that IBM, not wanting to have to maintain two different sets of code for variable length records, decided that having an RDW for a RECFM=V data set simplifies their logic.
Robert Sample
Global moderator
 
Posts: 3657
Joined: Sat Dec 19, 2009 8:32 pm
Location: Dubuque, Iowa, USA
Has thanked: 1 time
Been thanked: 270 times

Re: RECFM=U|V

Postby steve-myers » Sun Nov 18, 2018 8:10 pm

Robert Sample wrote:RECFM=U has a Block Descriptor Word (BDW) which contains the length of the block. RECFM=V also has a BDW, but it also has an RDW.
so the program should be able to determine the length without RDW, the same way like RECFM=U does (I assume that it uses interblock intervals for that).
You assume wrong. Interblock gaps have NOTHING to do with block lengths -- the block length for RECFM=U, RECFM=V, RECFM=VB data sets is determined by the BDW which is the first 4 bytes of the physical block. Technically the RDW in a RECFM=V block is not required, but if it is not present than the operating system routine that reads records must have different logic for RECFM=V and RECFM=VB data sets since VB data sets have to have the RDW. I would expect that IBM, not wanting to have to maintain two different sets of code for variable length records, decided that having an RDW for a RECFM=V data set simplifies their logic.

No.

RECFM U has nothing. On disk, RECFM U has the record length on the count area, but the loader does not see it. There is nothing in the data record.
 ** COUNT 0000000A08000014  **                      
                                                                   
0000     0  01000001 00040000  06000000 40000E20  *............ ...*
0010    16  00010E20                              *....            *
                                                                   
  ** COUNT 0000000A09000E20 **                      
                                                                   
0000     0  47F0F01A 15D3D7F3  F7F0C660 F1F161F1  *.00..LP370F-11/1*
0010    16  F861F1F8 60F0F94B  F0F990EC D00C0DC0  *8/18-09.09......*
...
0E00  3584  0C0C0C0C 0C0C0C0C  0C0C0C0C 0C0C0C0C  *................*
0E10  3600  0C0C0C0C 0C0C0C0C  0C0C0C0C 00000000  *................*
                                                                   
  ** COUNT 0000000A0A00004C  **                      
                                                                   
0000     0  0E000000 0000003C  00000000 00000000  *................*
0010    16  00010001 090006B1  090006B5 090006B9  *................*
0020    32  090006BD 090006C1  090006C5 09000749  *.......A...E....*
0030    48  0900074D 090007A9  0D0007E8 090007F5  *...(...z...Y...5*
0040    64  0D0008AC 0D0008B0  0C0008B4           *............    *

You see the 0E20 in the count area, but there is no BDW in the data.

The first record there is called a "control" record, as indicated by the 01 in its first byte. Believe it or not, it's not actually used when fetching the load module, as the relative address of the first block and its length are in the directory entry as well as the control record.The 06000000 4000E20 in the control record also specifies the relative address and length of the record. Notice there is nothing that resembles a record length of the data block in the data block.

The last record has RLD data to direct program fetch to insert real addresses in address constants.

These users thanked the author steve-myers for the post:
trengri (Mon Nov 19, 2018 1:39 am)
steve-myers
Global moderator
 
Posts: 2086
Joined: Thu Jun 03, 2010 6:21 pm
Has thanked: 4 times
Been thanked: 235 times

Re: RECFM=U|V

Postby trengri » Mon Nov 19, 2018 1:45 am

Thank you!
Actually, this is what I wanted to figure out - how the data is layed out on the disk with different access methods.
How do you obtain these CKD dumps? Did you write your own channel program and use EXCP?
trengri
 
Posts: 9
Joined: Tue Sep 25, 2018 2:12 am
Has thanked: 8 times
Been thanked: 0 time

Next

Return to Operating Systems