Episode 3.  Addressing and it's limits Part One!

First, let me apologise for this post as it's going to be a large one.  Second, I don't talk about 64 bit modes here because I have never used them.  But the basics will not have altered.  IBM really does put a lot of effort into maintaining backwards compatibility.

Secondly, I don't actually know anything about the internals of FreePascal or any other compiler come to that,  some, or all, of the techniques discussed here, and in part 2, may be impractical or even impossible to implement.  It should be noted however that it is not an exhaustive list.

Thirdly, it should be noted here that if the intention is to provide support on Hercules based systems, that Hercules allows us to use the newer instructions introduced by the processor upgrades even though we are using processors that shouldn't, in theory, support them. This doesn't apply to providing 31 or 64 bit addressing however, as considerable operating system support is required to handle these modes.

Finally, I may include bits of 370 assembler in this post.  I don't see how I can avoid it.  I will try and keep them as brief and non-technical as possible, but if you feel your eyes glazing over, ask and I will try and explain another way.

                 ------------------------------------

So, does 370 architecture have a 4k limit on code and data?  Well, yes... and no... Sort of... maybe...  It depends...

Prior to the upgrades of the 390 processor there was only 1 addressing mode, Base / Displacement or effective addressing.  The newer processors introduced Program Counter (it's called the PSW on 390 systems) PC relative addressing but it only applies to code and, perhaps constants, and then only to some instructions; It doesn't apply to data, so the limits are still relevant.

Base / Displacement consists of a 16 bit value, the first 4 bits enumerate a register, and the other 12 bits hold a displacement from 0 to 4095. The actual or Effective address for each storage operand is calculated as the unsigned addition of the value held in the base register to the displacement from the instruction itself.

The effective address for each storage reference is real or virtual and 24 or 31 bit depending upon the mode the processor is in at the time. In our case it will, probably, always be a virtual address.

It should be noted that the base register may not be register 0. Register 0 has an implied value of 0 when used for addressing purposes.

It is plain that each instruction reference of the Base / Displacement form can only reference a range of 4k, hence the urban myth that that this is a limit on the size of a module.  This is where USING enters the fray.  USING is an instruction to the assembler.  It tells it that a particular register holds the address (24 or 31 bit) of the label mentioned.  It is still up to the programmer to load that address into the register, the assembler won't (actually can't) do this for us.

Throughout, I am assuming that we will be using what IBM defines as standard linkage conventions between modules.  Let's start with a basic bit of code that represents a function:
PROG     CSECT              defines the name of our function         £START             set up standard linkage           LR    R12,R15      R15 has the address of PROG, copy it to R12         USING PROG,R12     Tell the assembler to use R12 as base           <code goes here>
        £END               return to caller SAVEAREA DS    18F          save area for standard linkage                   <working storage goes here>
LITPOOL  LTORG
          <constants get defined here>
        END
£START and £END are macros to set up the standard linkage stuff, SAVEAREA is a required area.  The details don't really matter.  If the total size of the code, working storage and constants grows beyond 4k, we will get assembly errors.

However, we can use the USING instruction to help us out here.  Part of the standard linkage is that R13 has to point to an area of 18 fullwords (32 bits each).  By adding a USING SAVEAREA,R13 to our code; PROG     CSECT              defines the name of our function         £START             set up standard linkage         LR    R12,R15      R15 has the address of PROG, copy it to R12         USING PROG,R12     Tell the assembler to use R12 as base         USING SAVEAREA,R13 use R13 as base register for working storage           <code goes here>
        £END               return to caller SAVEAREA DS    18F          save area for standard linkage           <working storage goes here>
LITPOOL  LTORG
          <constants get defined here>
        END
Now we have defined 2 base registers.  We are not allocating an extra register, we have to use R13 as a save area pointer anyway.  We are using it to address storage after the save area.  Now our code can be up to 4k, and our working storage plus constants can be 4k;  8k in total but still limited.

One final example and we'll call it a day for part 1.  This post is long enough as it is.
PROG     CSECT              defines the name of our function         £START             set up standard linkage         LR    R12,R15      R15 has the address of PROG, copy it to R12         LR    R11,R12      we can set up a second base for the code         AH    R11,=H'4096' by pointing it 4k past the first one         USING PROG,R12,R11 Tell the assembler to use R12 and R11 as bases         LA    R10,LITPOOL  and we can address the literal pool separately.         USING LITPOOL,R10
        USING SAVEAREA,R13 use R13 as base register for working storage           <code goes here>
        £END               return to caller SAVEAREA DS    18F          save area for standard linkage           <working storage goes here>
LITPOOL  LTORG
          <constants get defined here>
        END
Here we have set up s second register, R11, to point 4k past R12 and we have use d this as a base.  The code segment can now be 8k.  We have also added a separate register R10, to handle the literals.  We now have 16k we can address. A further refinement we can pull is to address all the initialisation code with R12.  When we enter the main code, we reset R12 to the start of the main code. Similarly with the exit code.  This could give us upto 12k with one base register.

But there is a limit to the registers we toss around like this and anything more complicated than the above would, if were coding by hand, probably get split into two or more modules.

This reminds me of 8086 "real" mode, and 16 bit mode with (far) addresses splitted into segment:offset. Using an 16 bit compiler and syntax (near/far pointers and seg:off notation) would be easier to support a segmented-memory architecture. I also don't see right now how data structures > 4KB could be addressed, without subroutines to map virtual into physical (here seg:off) addresses.

When I had a first look at the 8086 CPU, decades ago, I didn't understand how to handle such a memory architecture in an compiler. Only when I came across the Microsoft compilers and their memory models, I saw how "hints" in the source code would allow an compiler to create efficient machine code. But all these syntax externsions have been stripped from nowadays compilers and languages, so that it will be very hard to support such a dinosaur architecture in an 32/64 bit compiler.

DoDi

steve smithers wrote

Episode 3. Addressing and it's limits Part One!

Steve, are you OK if I archive this into the Wiki so it's easily accessible in the future?

Does the list owner (Jonas?) have any complaints about this on account of copyright etc.?

The best hint I had was in an OS/2 v1 programming manual, where it described in detail what conventions the MS/IBM code generators used to support arrays > 64K.

The zSeries Linux is, of course, 32-bit flat. However it has definite limitations on what platform it will run on (an actual zSeries system rather than an S/390 if I recall correctly), although it is rather more relaxed about the platform that application software is coded for (an S/390 G5 IIRC).

I still think that the place to start is with Linux on account of the familiar character set and ready availability of systems, but I'm putting some time in trying to read around the various older operating systems available. Hopefully Paul, Steve, or anybody else quietly lurking will correct me on this, but my understanding is that there is a number of older IBM operating systems which are now considered to be /res/ /nullius/, but these are all limited to 24-bit addressing. However there is an interesting project http://mvs380.sourceforge.net/ which is not only gathering and bundling the available code, but is slowly moving forwards with patches that allow application software with memory demands >16Mb to run on these operating systems, hosted on a patched variant of Hercules. If they continue having success, this could provide an interesting set of platforms in the future.

The list administrator has no say regarding the copyright of anything that anyone may post here. The copyright remains with the person who posted the text. In theory, you are not allowed to duplicate anything anyone writes without their explicit permission (save for fair use and other exceptions). In practice, since this is a public mailing list with publicly accessible archives, it's extremely unlikely that anyone would object.

Jonas

Mark Morgan Lloyd wrote Tue, 07 Feb 2012 11:29:01 +0000 Steve, are you OK if I archive this into the Wiki so it's easily accessible in the future?

I have no issue with that at all.

Does the list owner (Jonas?) have any complaints about this on account of copyright etc.?

If there any copyright issues on my behalf, I am happy to grant free use to anyone.

What does /res/ /nullius/ mean, I skipped Latin!

Purely in case anybody wants to complain that it shouldn't be moved between different servers etc.

Nobody's property [double-checks using Wp]. I think that in the case of the IBM operating systems it's a fairer description than (the frequently-applied) public domain, since IBM never explicitly donated them to the public, their creation was not funded by the public purse, and their ownership status if heavily modified is unclear.

steve smithers wrote

Thirdly, it should be noted here that if the intention is to provide support on Hercules based systems, that Hercules allows us to use the newer instructions introduced by the processor upgrades even though we are using processors that shouldn't, in theory, support them.

If Hercules is configured as an S/390 to keep the guest operating system (e.g. Linux) happy, is it possible to validate that a binary is completely safe to run on an S/370, i.e. contains no 390-specific features?

This doesn't apply to providing 31 or 64 bit addressing however, as considerable operating system support is required to handle these modes.

Although the OS/380 project appears to have made major progress on this.

It's not either / or.  Just because someone is fluent in IBM Assembler, has no influence on the fluency in other assembly languages.  I've had a PC at home for 20 years, plenty of time to become familiar with x86 assembler;  I just haven't had the inclination since I discovered Turbo Pascal.  In fact I would suggest that knowing IBM Assembler would be a great help in learning x86 assembler, or any other assembler come to that.

I'm not exactly clear on what you are asking.  You could turn off the 390 features on Hercules and see if it crashes, or you could ensure you don't generate thme by using some compiler switch, or you could remove the macros required for the IBM Assembler to support the mods, they will cause assembly errors.  Or what ...?

OS/380 IS the operating system support.  But it only supports 31 bit addressing using some custom enhancements to both Hercules and MVS 3.8.

But assuming that an initial port is to Linux, you need Hercules to provide S/390 features or the operating system won't run. The way I read it, if you're also targeting classic operating systems such as VM/370 then the only way to prove that the code generation is 370-safe is to run the code on a (simulated) 370: there's no way of testing compliance at an earlier stage.

Mark Morgan Lloyd wrote on Fri Feb 10 21:20:15 CET 2012

Hercules provides the S/390 features for Linux, but we don't have to use them. S/370 is a subset of S/390, that will run quite happily on S/390 processors. The only requirement for S/390 extensions is gas.  But gas doesn't support USING and is a waste of space for our requirements.  (This may not apply to the version of gas supplied with OS/380)

There is nothing provided in the extensions that we cannot achieve without them, sometimes it may be more aggro, but we can still do it.  We could, if we were bored and had nothing better to do, write macros for the System/370 assembler to support the S/390 extensions on a base "real-iron" box.

Anyway, assuming that Linux/390 is the target.  There is no way to prove compliance of the initial port until we are ready to run on Linux/390.  At some stage, you have to bite the bullet.  We also have to modify the RTL, even for Linux/390.

I'm a bit worried that gas might not support all the arithmetic operations that appear to be required by your examples, but at the same time I still feel that Linux is the best initial target. Has anybody ported a fully-featured IBM macro assembler to Linux?

To take your points in reverse order:

Tachyon produce a free HLA compatible assembler.  IBM produce one that may or may not be free.  There may be others too.

Yes.  I agree, Linux/390 would seem a logical first step.

Gas is brain-dead, I wouldn't use it.  It WON'T work for my examples as it supports, as far I know, neither USING nor MACROs, let alone the doubts you have.

Finally, my examples are just that, examples.  It is not a specification for a final product, it is a wild theory based on zero knowlege of the internals of the compiler.  I can come up with a few others if you don't like that one.  In fact, episode 6 may just do that, but it will be a while before it appears.

The Tachyon one looks interesting, although I note that the free variant is for non-commercial use only- I hope that doesn't cause anybody any problems.

Yes. I agree, Linux/390 would seem a logical first step.

Reading around, I note that IBM started supporting IEEE maths with the zSeries, not (as I previously thought) with the 390. I think that implies that the best initial target would probably by zSeries running Linux, with migration to other operating systems etc. following.

Gas is brain-dead, I wouldn't use it. It WON'T work for my examples as it supports, as far I know, neither USING nor MACROs, let alone the doubts you have.

I notice people discussing health warnings associated with the IBM macro libraries- emphatically not in the public domain. I've definitely never seen these, so if necessary I could be part of a cleanroom reimplementation of what's needed.

I suppose there's a possibility that once the dust has settled it would be possible to implement a preprocessor to allow gas to serve as a backend.

Hopefully I'm not the only person reading your notes with interest.