LongEx Mainframe Quarterly - May 2018
IBM Z mainframes are generally accepted as the most secure computing platform available. Many have used the Common Criteria EAL rating of mainframes as proof. But what is EAL? Is it a valid measurement of computer security, and did z/OS really achieve EAL5+? Spoiler alert: the answer to the last two questions is "no." Common Criteria
So, let's see how this works for z/OS. IBM want to show that z/OS is a secure operating system. Well, they can't. One of the basics of Common Criteria is that validation must be done by an accredited third party. In the US, this is the Common Criteria Testing Laboratory (CCTL). The z/OS 2.2 validation was done by the German IT Security Evaluation Facility (ITSEF). The first thing we need is a Security Target: what is being tested. This includes the Target Operating Environment (TOE): what product or configuration is to be tested. For z/OS 2.2, the TOE was a single instance of z/OS 2.2 running in PR/SM or as a z/VM guest. It had the usual features (JES2, OpenSSH, Communication Server) and RACF as the security product. A couple of hardware configurations (z114, EC12, z13 or z196) were covered - all with at least one Crypto Express card. Interestingly, some features were specifically excluded, including JES2 NJE over TCP/IP, Object Access Method (OAM) and RACF RRSF. The z/OS 2.2 Security Target also specifies a list of things that must work: Security Functional Requirements. For example, user identification, and file protection. The good news is that the Common Criteria has pre-packaged functional requirements for some products in 'Protection Profiles' (PP) and packages. The z/OS 2.2 certification used the Operating System Protection Profile (OSPP), plus two 'extension packages':
EALOK, so we know what we're testing. The German Federal Office for Information Security (BSI) has released a second document with the Certification results: how it all went. 
You can find the full certification document here. IBM also explains a little about the testing here. The first thing this document says is that the TOE conformed to the OSPP profile and the two extension packages: it does everything in these specifications. It also states that the TOE conformed to EAL4+. What does this mean? EAL, or Evaluation Assurance Level, is a measurement of assurance - how sure we can be that the TOE achieves the Security Target. It is a number from 1 to 7, 1 being the lowest, 7 being the highest. The entry level (EAL 1) provides basic testing and checking. An important part of any test is the Security Target. For EAL 1, this doesn't have to be fully specified. As the numbers go up, the amount of testing and verification increases. The developer helps by providing documentation, design specifications and more - how much depends on the EAL level. For example, for EAL 2, the Security Target must be fully specified. The developer also provides product manuals, basic design documentation, and some testing results that are independently confirmed. A vulnerability analysis is also performed. By EAL4, the developer provides full interface specifications and a modular design of the product for more formal verification. Or in other words, the higher the number, the more testing, analysis and checking was performed (and the higher the costs). The Common Criteria explains what must be done to achieve each EAL level. In our z/OS 2.2, the testing was EAL4+. What's the plus sign? It means there is a little assurance outside of the EAL4 specification. In the case of z/OS, IBM have promised to document and fix any securities issues found (meeting Common Criteria specification ALC_FLR.3). Let's think about this for a minute. A higher EAL rating doesn't necessarily mean that a product is more secure. It just means that more thorough testing and checking was done to ensure that the product meets the Security Target. To get a real idea of security, we need to combine the EAL with the Security Target: what functionality was checked, and how thoroughly. z/OS and EAL5+OK, so z/OS 2.2 got to EAL4+. But didn't z/OS get an EAL5+? No. z/OS 1.6 achieved EAL3+, subsequent releases up to z/OS 2.2 achieved EAL4+. But there have been tests not only for the base z/OS operating system, but other relevant software configurations. Here are some of them:
It seems like z/OS is running with the pack in the operating system race. Though as is so often the case, the devil is in the details. To properly compare each certificate, we'd need to closely analyse the Security Target of each test and compare. Let's skip that and accept that most operating systems (other than Windows) adhered to OSPP to EAL4+, some with additional packages. Operating systems often run under virtualization software. IBMs mainframe-based PR/SM and z/VM have their own certification. The non-mainframe VMWare was certified to EAL2+ in 2015, However any more recent virtualization certifications are hard to be find. So, it can be argued that the PR/SM virtualization (last certified in 2016) has been more rigorously tested and is more secure. z/OS also has a separate security certification for RACF, which none of the others can match. And this was very well analysed to EAL5+ standards. This Meansz/OS has been through more certifications than other operating systems. Every version after 1.6 (until 2.2) has met the OSPP with two extra packages to EAL4+. RACF and PR/SM both have exceeded this EAL level with their own separate Security Targets and certification. Although z/OS has never achieved an EAL5+ rating, EAL4+ isn't bad. What's more, an EAL rating alone isn't a good indicator of security. But a typical z/OS environment with RACF running under PR/SM has been more rigorously tested to Common Criteria standards than any other. This is a good indicator that this environment's security is hard to beat. |
