management: What is EAL, And Is It Any Use?

IBM Z mainframes are generally accepted as the most secure computing platform available. Many have used the Common Criteria EAL rating of mainframes as proof. But what is EAL? Is it a valid measurement of computer security, and did z/OS really achieve EAL5+?

Spoiler alert: the answer to the last two questions is "no."

Common Criteria

The Evaluation Assurance Level (EAL) that we're talking about comes from the ISO/IEC 15408 standard: The Common Criteria for Information Technology Security Evaluation. This standard is a basis for evaluating the security properties of an IT product.

So, let's see how this works for z/OS. IBM want to show that z/OS is a secure operating system. Well, they can't. One of the basics of Common Criteria is that validation must be done by an accredited third party. In the US, this is the Common Criteria Testing Laboratory (CCTL). The z/OS 2.2 validation was done by the German IT Security Evaluation Facility (ITSEF).

The first thing we need is a Security Target: what is being tested. This includes the Target Operating Environment (TOE): what product or configuration is to be tested. For z/OS 2.2, the TOE was a single instance of z/OS 2.2 running in PR/SM or as a z/VM guest. It had the usual features (JES2, OpenSSH, Communication Server) and RACF as the security product. A couple of hardware configurations (z114, EC12, z13 or z196) were covered - all with at least one Crypto Express card. Interestingly, some features were specifically excluded, including JES2 NJE over TCP/IP, Object Access Method (OAM) and RACF RRSF.

The z/OS 2.2 Security Target also specifies a list of things that must work: Security Functional Requirements. For example, user identification, and file protection. The good news is that the Common Criteria has pre-packaged functional requirements for some products in 'Protection Profiles' (PP) and packages. The z/OS 2.2 certification used the Operating System Protection Profile (OSPP), plus two 'extension packages':

• Extended Identification and Authentication (OSPP-EIA)
• Labelled Security (OSPP-LS)

EAL

OK, so we know what we're testing. The German Federal Office for Information Security (BSI) has released a second document with the Certification results: how it all went.

You can find the full certification document here. IBM also explains a little about the testing here.

The first thing this document says is that the TOE conformed to the OSPP profile and the two extension packages: it does everything in these specifications.

It also states that the TOE conformed to EAL4+. What does this mean?

EAL, or Evaluation Assurance Level, is a measurement of assurance - how sure we can be that the TOE achieves the Security Target. It is a number from 1 to 7, 1 being the lowest, 7 being the highest. The entry level (EAL 1) provides basic testing and checking. An important part of any test is the Security Target. For EAL 1, this doesn't have to be fully specified.

As the numbers go up, the amount of testing and verification increases. The developer helps by providing documentation, design specifications and more - how much depends on the EAL level. For example, for EAL 2, the Security Target must be fully specified. The developer also provides product manuals, basic design documentation, and some testing results that are independently confirmed. A vulnerability analysis is also performed. By EAL4, the developer provides full interface specifications and a modular design of the product for more formal verification.

Or in other words, the higher the number, the more testing, analysis and checking was performed (and the higher the costs). The Common Criteria explains what must be done to achieve each EAL level.

In our z/OS 2.2, the testing was EAL4+. What's the plus sign? It means there is a little assurance outside of the EAL4 specification. In the case of z/OS, IBM have promised to document and fix any securities issues found (meeting Common Criteria specification ALC_FLR.3).

Let's think about this for a minute. A higher EAL rating doesn't necessarily mean that a product is more secure. It just means that more thorough testing and checking was done to ensure that the product meets the Security Target. To get a real idea of security, we need to combine the EAL with the Security Target: what functionality was checked, and how thoroughly.

z/OS and EAL5+

OK, so z/OS 2.2 got to EAL4+. But didn't z/OS get an EAL5+? No.

z/OS 1.6 achieved EAL3+, subsequent releases up to z/OS 2.2 achieved EAL4+. But there have been tests not only for the base z/OS operating system, but other relevant software configurations. Here are some of them:

• z/VM - z/VM 6.3 conformed to OSPP, OSPP-LS and OSPP-Virtualization: EAL4+
• RACF - RACF with z/OS 2.2: EAL5+
• DB2 for z/OS - DB2 v12 with z/OS 2.2 conformed to the Protection Profile for Database Management Systems: EAL4+
• PR/SM -PR/SM on z13: EAL5+

• z/VM 6.4: OSPP, OSPP-LS, OSPP-Virtualization: EAL4+
• Windows 10: OSPP (EAL not specified)
• Red Hat Enterprise Linux 7.1: OSPP, OSPP-Advanced Management, OSPP-Advanced Audit, OSPP-Virtualizaton: EAL4+
• SUSE Linux 12: OSPP to EAL4+
• Oracle Solaris 11.1: OSPP, OSPP-Advanced Management, OSPP-EIA, OSPP-LS, OSPP-Virtualization: EAL4+

It seems like z/OS is running with the pack in the operating system race. Though as is so often the case, the devil is in the details. To properly compare each certificate, we'd need to closely analyse the Security Target of each test and compare. Let's skip that and accept that most operating systems (other than Windows) adhered to OSPP to EAL4+, some with additional packages.

Operating systems often run under virtualization software. IBMs mainframe-based PR/SM and z/VM have their own certification. The non-mainframe VMWare was certified to EAL2+ in 2015, However any more recent virtualization certifications are hard to be find. So, it can be argued that the PR/SM virtualization (last certified in 2016) has been more rigorously tested and is more secure.

z/OS also has a separate security certification for RACF, which none of the others can match. And this was very well analysed to EAL5+ standards.

This Means

z/OS has been through more certifications than other operating systems. Every version after 1.6 (until 2.2) has met the OSPP with two extra packages to EAL4+. RACF and PR/SM both have exceeded this EAL level with their own separate Security Targets and certification.

Although z/OS has never achieved an EAL5+ rating, EAL4+ isn't bad. What's more, an EAL rating alone isn't a good indicator of security. But a typical z/OS environment with RACF running under PR/SM has been more rigorously tested to Common Criteria standards than any other. This is a good indicator that this environment's security is hard to beat.

David Stephens

LongEx Quarterly is a quarterly eZine produced by Longpela Expertise. It provides Mainframe articles for management and technical experts. It is published every November, February, May and August.

The opinions in this article are solely those of the author, and do not necessarily represent the opinions of any other person or organisation. All trademarks, trade names, service marks and logos referenced in these articles belong to their respective companies.

Although Longpela Expertise may be paid by organisations reprinting our articles, all articles are independent. Longpela Expertise has not been paid money by any vendor or company to write any articles appearing in our e-zine.