longpelaexpertise.com.au/ezine/MainframeSecurityFromInside.php?ezinemode=printfriend

LongEx Mainframe Quarterly - May 2018

technical: Mainframe Security From the Inside

You hear a lot about IBM Z mainframes, and how they are the world's most secure platform. But why is this so? What makes IBM mainframes more secure than other computer systems - particularly as there are always new ones being created?

To me, the answer is simple: hardware-based privilege. Let me explain.

Privileged Instructions

If you're an assembler geek like me, you'll be browsing the IBM Principle of Operations (POPS) manual from time to time. This is a manual with information about all the machine instructions that can be issued for a mainframe. Unless you're an assembler programmer, you probably haven't worked with these directly.

If you look in chapter 10 of POPS, you'll see information on 'privileged instructions'. If you've never seen it, the chapter starts like this:

This is interesting: a whole bunch of commands that cannot be called by normal programs: a program must be in a privileged state called 'supervisor state'.

Let's look at one of these instructions: Set Storage Key Extended (SSKE). This instruction lets you change the storage key of a 4k block of memory. Storage keys are how mainframes protect 'sensitive' memory like that used by RACF. Sensitive storage is set a key that only privileged programs can access. This is a hardware feature, not a z/OS feature.

Chapter 14 of POPS also explains some other privileged commands: I/O commands. These are commands used to directly access disk and tape units. They're used by access methods like VSAM and QSAM. Normal programs are prevented from using these commands. This stops them bypassing dataset and other security checking.

The protection of the commands in Chapter 10 and 14 are hardware based: they are not related to z/OS, z/VM or other operating systems that can run on the mainframe. However, these operating systems do provide their own privileged APIs. For example, z/OS has a whole bunch of macros (think APIs) that can only be called in supervisor state. For example, the macro CHANGEKEY to change the key of a block of memory.

These privileged instructions are important: without them z/OS itself couldn't do many things it needs to. Database systems and access methods like VSAM use them. Security software like RACF uses them. So, we need these instructions for operating systems and supporting software to work. But we also need to protect them from 'normal' programs.

Storage Keys

In the section above, we talked about storage keys, and how mainframes use this to protect storage. The way this works is that the mainframe checks the storage key of a page of memory (a 4k block) with the storage key that the program is executing in. That's right, every program runs in a storage key: stored in the PSW.

If the storage keys match, all is OK. If not, you may be prevented from accessing the storage (depending on the storage subpool, and access required).

But if you're in storage Key 0 (zero), there are no restrictions. Programs running in storage key 0 are effectively privileged: they can access any storage they want. Many restricted z/OS macros can be executed by a program running in storage key 0.

Remember, storage keys are a hardware feature. They're not implemented in the operating system.

How to be Privileged

We talked before about how only programs in supervisor state can access some z/OS macros like CHANGEKEY. But programs running in storage key 0 can also do a lot more things than normal programs. You can think supervisor state and key 0 both being privileged states. But how do you write a program in supervisor state or key 0? There are a couple of ways:

  • Be a program in the z/OS nucleus
  • Be setup to run in supervisor state by another supervisor state program
  • Use a macro like the z/OS MODESET macro

Let's think about these for a second. It makes sense that z/OS itself needs to be privileged. So, the z/OS nucleus, should be privileged. The nucleus programs will probably need to call other programs to do privileged things. So, we need to allow privileged programs to set other programs as privileged. z/OS does this in many ways, including parmlib members such as IEFSSNxx (subsystems), IEASVCxx (SVCs), SCHEDxx (PPT entries) and PROGxx (exits).

Some programs that are called from normal user programs may need to be privileged. For example, if a user program needs to get storage, it will need to call a privileged program to do that. SVCs and PC instructions provide this service.

There will be other programs that need to be privileged for short periods. It's actually smart to only run in privileged mode if you need to. So, if a program does something wrong, there is less chance of catastrophe. In z/OS, the MODESET macro can be used to set a program from 'normal' state to supervisor state or key 0, and back again.

APF Authorization

As we've said before, the privilege and storage keys we've been talking about are IBM Z hardware features: applicable to z/OS, z/VM, z/VSE, z/TPF and any other operating system running on the mainframe.

z/OS builds on this privilege with APF: Authorized Program Facility. APF and privilege aren't the same thing: one is a z/OS feature, one is a hardware feature. Let's take a look at that MODESET macro in the z/OS MVS Programming: Authorized Assembler Services Reference. You can only use this macro if either:

  • You're APF authorized
  • You're already privileged

There are a lot of z/OS macros that can only be called by APF authorized or privileged programs.

How does z/OS know if a program is APF authorised? A field in a control block (the JFCB if you're interested) in protected memory.

The Mainframe Advantage

IBM mainframes make security a lot easier by implementing a line in hardware between two groups:

  • Privileged users that can do anything they want
  • The rest

To protect the mainframe, all we need to do is protect this line: limit who can create a privileged program. In z/OS, most think of limiting access to APF authorised load libraries. But this is just the beginning.


David Stephens