EFT evaluation: a craftsman-led approach
by Dr. David F. C. Brewer


This paper examines a security evaluation of an electronic funds transfer system that was carried out over ten years ago in 1986. Even now, the evaluation may still be regarded as innovative. At the time it was a candidate paradigm for the European ITSEC, but was rejected in favour of the "Orange Book". However, it differs from the Orange Book, and all of its derivatives, in two main respects. Firstly, it is based on the overall business needs. It combines risk analysis, evaluation and corrective action. There is no concept of E-levels. There is no concept of failure. The result is a system that has been demonstrated to be adequately secure with respect to the defined business needs. Secondly, it focuses the evaluator's attention on the vulnerabilities that matter, escalating the level of investigation as appropriate. There is no need to evaluate everything to the same depth. There is no need for any special documentation or other bespoke evaluation deliverables. The result is a saving of time and money on behalf of both sponsor and evaluator.


A long standing banking service has been the making of sterling payments to UK beneficiaries on behalf of customers of overseas banks. The customer has his account debited in his own currency whilst the beneficiary is paid in sterling. Payment instructions can be received from the overseas bank in a variety of ways, including telegraphic transfer, mailed banker's order or an electronic funds transfer (EFT) network. In 1986 we were invited to evaluate the security of a system which had been developed with the objective of automating this procedure and thereby providing a better service to the customers.

The objective of any bank, of course, is to make an acceptable level of profit without incurring excessive risk. One of the ways it can do this is to provide an excellent service to its customers and correspondent banks, thereby attracting custom both in the form of high value, high commission transactions and high volumes of low value but straightforward transactions. The system did both and, in 1986 processed in excess of GBP3,000M per day through over 10,000 separate transactions.

One might therefore expect that fraud, or error, would be a major concern; only one transaction would need to go wrong for the bank to have lost GBP1M or, at best, the interest payable on this amount compounded over the time it would have taken to recover the principal. Nevertheless, in the real world goodwill may make it possible to recover the loss in full. For example, one bank will help another in anticipation of the favour being returned, should the need arise. However, while matters are being sorted out the bank will be exposed. It is therefore a question of risk management. The bank therefore wanted to know what risks it ran and whether its systems were adequately secure to counter or otherwise manage those risks.

Not surprisingly, the system had been developed over a decade. Although originally designed with an elegant and effective system architecture, changes forced by changes to the user requirements had produced what could only be considered to be a complicated and confusingly structured system. Traditionally at the time, auditors did not venture into investigating computer systems in any depth, such as examining software. This was felt to be inadequate for a system of this nature. However, the coverage given by auditors and their ability to home into problems that really mattered was regarded as essential to any investigation. Compounded by the complexity and confusing nature of the system, the need to combine the best qualities of auditing and software examination necessitated that a new, and perhaps innovative approach was required. At the time, of course, we did not have the benefit of the ITSEC or Common Criteria, and although the Orange Book had just been published, it did not appear relevant to the evaluation of an electronic funds transfer system. Moreover, although we were to be given access to the test system and the source code, the system documentation was largely un-maintained, and therefore considered to be unreliable. We therefore decided to invent a new approach.

The Approach

The approach that we ultimately adopted is shown schematically in Figure 1. Starting with the "system," (i.e. the EFT system we were asked to evaluate) we devised an architectural model which we called the "pipeline model". This enabled us to partition the system into its constituent business services, for example those concerned with the making of sterling payments to UK beneficiaries. Then, taking the "security policy" (which was essentially the bank's requirement for the management of risk) we derived a set of "control objectives" similar to those commonly used by auditors. The pipeline model had five components, which were intended to work collectively to ensure that every control objective was met. In practice, we found that different components employed different mechanisms to achieve the same objective and that not every component needed to meet every control objective. We were therefore able to identify specific "security requirements for each pipeline component". The "evaluation" sought to establish that the system met the security policy, by demonstrating that the individual (pipeline) components met their respective security requirements. In doing this, the evaluation was able to confirm that the system was secure or where weaknesses were found, countermeasures were identified. To do this, the evaluation made use of a variety of investigative techniques.

Figure 1: Schematic of the evaluation process

(Go back)

The Pipeline Model

We were able to model an EFT transaction quite simply by representing the transaction in the form of a pipeline, see Figure 2. A customer instruction to transfer money, presented at one end of the pipeline, ultimately resulted in some financial effect being produced, in exact accordance with the customer instruction, at the other.

Figure 2: A simple pipeline

(Go back to the Approach)

It was convenient to regard a pipeline as being responsible for processing a single type of transaction. Transaction types are well defined in banking circles, for example SWIFT messages are given message identifiers such as MT100 (which is the customer transfer described in the introduction). In doing this, we were able to focus our attention on the IT aspects of a particular aspect of the business in isolation from all other aspects. It meant, however, that we had to regard the entire system or a bundle of pipelines. Ignoring maintenance functions, such as updating exchange rates, we identified over 130 such pipelines.

Pipeline Structure

The two ends of a pipeline did not need to be collocated, and their location could conceptually even vary with each transaction. Indeed, our model allowed for the two ends of the pipeline to be in different banks and in different countries for each transaction, depending on where the instruction originated and where the payment was to be made, see Figure 3.

Figure 3: Pipeline communications

We constructed the pipeline models using five generic building blocks or components, see Figure 4. We called these:

  1. active processes;
  2. communication processes;
  3. stable data processes;
  4. enquiry processes;
  5. access control processes.

The active processes included all the software that was directly responsible for making the financial effect happen, e.g. printing cheques and posting accounting entries. The communication processes sent and received messages over SWIFT and various national and private networks. The model treated the sending and receiving processes as distinct entities. The stable data processes were responsible for inserting information into the pipeline which was relatively stable for a number a transactions, such as foreign exchange rates and bank sort codes. The enquiry processes were responsible for extracting information from the pipeline, e.g. to assist in answering customer enquiries. The access control processes were responsible for controlling human access to the pipeline.

Figure 4: Pipeline structure

(Go back to the description of the model)

The MT100 Sterling Pipeline

A simplified schematic (identifying only the active and communication processes), in the form of a flow diagram, of the pipeline for "SWIFT MT100 sterling" transactions is shown in Figure 5. Conceptually, an MT100 sterling message was received from SWIFT and validated. If the message was totally indecipherable an error resulted, otherwise the message was accorded a "received" status and progressed further down the pipeline. In practice, all SWIFT messages were received via a dedicated minicomputer and validated by a mainframe computer. Following successful receipt a hard copy of the message was produced. The system might have still, however, found fault with the message, in which case an error resulted. Assuming this not to be the case the message was shown to an operator for checking and deciding how payment was to be made within the UK. Various amendments and additions were permitted. For example it might have been necessary to append the sort code of the branch holding the beneficiary's account.

After checking and possibly modifying the transaction, the operator could then pass it to:

  1. CHAPS (a separate pipeline);
  2. manual processing (e.g. for non-standard requests such as an attempt to credit a customer's account at a branch which has ceased operations);
  3. the next part of the pipeline.

As shown in Figure 5, a bifurcation of the pipeline then results. For transactions less than some nominated value, which had not been amended, a "released" status resulted, otherwise they acquired only "vetted" status and a second operator would have had to approve the transaction for it to be released. An operator would then have to decide the form of payment. There were seven possibilities. Six of these were apparently straightforward and were dealt with by the host computer's local users; for the remaining option the instruction was passed over the bank's private network for payment at a branch. In this case it was possible for the application for payment to be rejected by the recipient branch, hence the cycle shown in the figure. "Recycling" resulted in the instruction acquiring a vetted only status and processing restarting once again from that part of the pipeline. The cycle permitted recovery from payment transfer to a branch which for some reason could not make that payment.

Figure 5: Schematic of the MT100 pipeline

(Go back to The MT100 Sterling Pipeline | The bifurcation of the pipeline | only once semantics | Primary level investigations | Secondary level investigations)

Control Objectives

We asserted that the EFT system as a whole would be secure if the individual pipelines met certain control objectives. We identified 19 such control objectives split into 4 groups. We further determined that each pipeline component did not have to meet every control objective. For example, if an enquiry process could be prevented from writing to the pipeline, it need do very little else. On the other hand, the communications processes needed to satisfy the majority of objectives.

Group I - Only Once Semantics

Where a choice of payment existed (e.g. Figure 5), we required the effect of the pipeline to cause just a single method to be chosen (choice). Clearly to do otherwise would have meant that a particular payment could have been made more than once. However, choice by itself was not enough: we also required that the pipeline could not replicate the effects of a single customer instruction even with the same payment method (no replication), and it could not allow the same customer instruction to be input more than once (no duplication); it could not produce financial effects without a customer instruction having been input (no addition) and it could not loose an instruction (no loss). We referred to these five objectives, choice, no addition, no duplication, no loss and no replication collectively as only-once-semantics.

Group II - Correspondence with Reality and Timeliness

We further required that the pipeline would always produce the desired financial effects in accordance with the instruction, which had to be a genuine instruction. We called this correspondence with reality. This was an important issue. It concerned, for example, verifying that a customer instruction as entered into the system was correct (the so called `check and release function'), inter-bank reconciliation, correspondence of payments to accounts and correspondence of payments to instructions. We also required the financial effects to be produced on the specified value date, no sooner and no later. We treated this intent as two separate security objectives, no acceleration and no delay, since the former could be guaranteed, the latter could not.

Group III - Data Transmission

In keeping with good banking practice regarding telecommunications, we required that privacy was maintained for all communications; that the message entered must be the one received (message integrity), that it must only be received at the destination specified by the sender (correct destination), and that the source was able to be confirmed as authentic (correct source). We also required that there should be an established business relationship between source and destination for the type of EFT transaction sent (valid business relationship), and the recipient of a message could not deny that it had been received (no repudiation).

Group IV - Accountability

In keeping with good banking practice regarding access control, we asserted the need for a general requirement to identify and authenticate users (authenticity), to ensure that users could only make use of the programs and data within the system for which the users had been authorised (right of access), that users did not abuse their authority (no abuse of authority), that all transactions were logged (accountability), and that all such logs were inspected for evidence of malpractice (audit).

Security Requirements

As previously mentioned, not every control objective had to be met by every component. We found that where control objectives were common to several components, the components would often implement them in different ways. For example, with regard to 'no loss' in the context of communication processes, we expressed the requirements in terms of message sequencing numbers; whereas in the context of active processes, we expressed them in terms of the relationships between the acceptance or rejection of inputs and the production of outputs. In total we identified 55 specific security requirements.

The Evaluation

Pipeline Identification and Ranking

Our first task was to identify the pipelines in terms of type of input and financial effect. We did this initially by studying the available documentation and by talking to people. We then ranked the pipelines in terms of the bank's financial and regulatory exposure, so that we could study the most important pipelines first and ignore those which did not pose any real risk to the bank.

Primary Level Investigations

Having selected a pipeline, we then studied it in greater detail, in particular to identify how the generic pipeline components mapped on to real components. For example, being in an IBM environment, we were able to say that "this TCAM queue is a stable data interface; that TCAM queue is an enquiry data interface" and so on. The bank found this pipeline analysis to be of immediate benefit in its own right, allowing maintenance staff, for example, to understand how their programs interacted with those of their colleagues.

For each actual component, we then instantiated the various security requirements to create specific questions in the context of the actual component. For example, the MT100 Sterling (Figure 5) pipeline had four different communication processes: SWIFT, CHAPS and two different private networks. The security requirements for communication processes were therefore instantiated four times.

We then interviewed selected maintenance and operations staff to answer these questions and set about searching for corroborative evidence to support their answers.

We were surprised with the reliability of the live error reports as corroborative evidence. We asked the question (for example based on the 'no replication' control objective) "have you ever seen a cheque printed twice?", to which the answer was "yes, and here are the live reports to prove it". We asked the question (based on the 'no addition' objective) "has the SWIFT Interface Device ever output a transaction that was never input?", to which the answer was "no, there are no such live error reports".

Thus, we were able to reliably answer all our questions, and determine how well each actual component met its corresponding control objectives. We considered our findings both in the context of each component in isolation and in context of the entire pipeline. Thus, we sought to determine whether isolated component weaknesses were adequately countered elsewhere in the pipeline.

Evaluation Results

Our results fell into four categories:

  1. The control objective had been met (unconditionally) by the pipeline.
  2. The control objective would be met by the pipeline, provided that certain procedures were carried out by the users or operations staff.
  3. The control objective might not be met by the pipeline.
  4. The control objective had not been met by the pipeline.

In case (a) no further action was required. In case (b), we recast our findings in the form of a detailed checklist for immediate use by the bank's internal inspection department during subsequent audits. In case (c), we considered the impact that failure of the control objective would have on the pipeline. If the impact could be tolerated, no further action was taken. Otherwise we escalated our analysis to the secondary or tertiary levels of investigation and re-categorised our findings (in practice, as case (a), (b) or (d)) once the result was known. In case (d) we identified what needed to be done to counter the weakness. The result of this process, once completed, was that the bank knew that it had a secure system, with all countermeasures (both technical and non-technical) adequate for the levels of financial and regulatory exposure involved.

Secondary Level Investigations

Our investigations at this level were characterised by performing "goal orientated" penetration tests. In other words, we agreed precisely what we were looking for before any penetration tests were performed. In practice, we found that this approach allowed us to use experienced and trained bank staff to carry out the penetration testing under our direction.

As an example, the Figure 5 pipeline contained a feedback loop. Would it therefore be possible to request a branch transfer for customer instruction to be paid in cash at Branch A, cancel it, request payment in cash at Branch B, cancel that, request payment at Branch C, and so on? Would it be possible to do this without limit? Indeed, would the individual branches ever know that the instruction had been cancelled? Could an accomplice therefore collect the cash at each branch? We grouped such questions together and made them the subject of a series of tests. The goal in this case was to determine whether it was possible to exploit the cyclic nature of the pipeline to perpetrate a fraud.

(Go back to the evaluation results)

Tertiary Level Investigations

Our investigations at this level were characterised by analysing the source code. In particular, we carried out a "semantic analysis" of the access control software. The software was written in COBOL, multi-threaded and the listing was about 13 cm thick! The semantic analysis, which took about three weeks, involved converting the source into what mathematicians call an "acyclic graph", essentially one enormous "case" statement showing every control path through the program for which an access control decision could be made. We verified by testing the existence of all vulnerabilities identified by analysis.

We recognised that the program's use of COBOL "search" constructs was semantically equivalent to "set membership". This allowed the search statements to be replaced by booleans with meaningful names such as "KNOWN_USER", and "LOGGED_ON", etc. As a consequence, the path conditions were invariably expressed in a form of pidgin English (an argot), e.g.



thereby simplifying considerably our understanding of the results.

The path:



was particularly curious, and clearly distinguishable by the absence of the KNOWN_USER, LOGGED_ON and MASTER_TERMINAL_ON predicates. It was in fact a security vulnerability, that would return a branch to full operational capability after a premature close-down. No log-on was required. Just switch on a terminal after everyone had gone home and type DEBUG. There was no audit record either.

The technique also revealed (and solved) some significant, but intermittent live error problems.

The technique was repeatable and reproducible in the EN45000 sense, but we found it hard to train bank staff to apply it to new types of problem.


Soon after the evaluation work had been completed, the author was invited to join the Anglo-French-German-Dutch team that ultimately wrote the ITSEC and the ITSEM. The EFT evaluation experience formed an input to that work. However, after much consideration and public consultation, it was concluded that the approach, particularly at the tertiary level of investigation, was very much an "expert craftsman" approach. It was therefore considered to be generally unsuited for "mass produced" evaluations and hence an unsuitable basis for the ITSEC. However, the concept of the control objectives survived in part as the ITSEC generic functional headings and the approach to combining pipeline security weaknesses was generalised and recast as the "attack path" analysis given in the ITSEM.

The value of the "expert craftsman" approach was recently further underpinned when the bank requested a shortened version of the approach that could be undertaken by bank staff in an afternoon. The method that we devised made use of the pipeline concepts, their financial and regulatory parameters and existing countermeasures to determine whether the countermeasures were inadequate, adequate or excessive. Countermeasures could be added or removed until a "countermeasures adequate" result is obtained. Once again, in contrast to the ITSEC, there is no concept of E-levels and there is no concept of failure. These are characteristics which appear far more acceptable to the commercial world. Indeed, the preliminary results of a study  undertaken in 1997 on behalf of the UK DTI to determine the market opinion of the meaning of the word "trust" in Trusted Third Parties supports this view. Given this, perhaps the time has come to re-visit this type of craftsman led approach, particularly in the context of system assessment.