click here to download paper in PDF format (1.4M)

             
             
  Home About Gamma  Tour our Web Site  Events  White Papers  Services  Visitors' Book  How to contact us
         IMS  Internal Control  ISMS  Smart Cards  Common Criteria
                 

Objective

If the current ICS is not achieving management's objectives, management needs to be able to determine what needs to be changed and plot a course of action to implement those changes.  As the ICS evolves, management again needs to measure the actual ICS to see how its effectiveness is improving.

Apart from the overall ICS, management may focus attention on particular controls.  This may be in response to incidents or changes in threat.

Measurement

There are two types of improvement:

Measurements are generally made using the methods previously described, i.e. using Table 2 to determine the actual class of a control, using Table 3 to determine operational effectiveness and Table 4 to determine cost-effectiveness.  However, individual improvements may require additional metrics as discussed below.

What data do I need?

Time to detect, time to fix, time window (either for individual events and/or averaged across many events of the same class)
Cost of fix and impact penalty (again either for individual events and/or averaged across many events of the same class)
Number and frequency of events
Cost of control, cost of ICS
Whether controls are protected by self-policing procedures and whether those procedures are fail-safe
What incidents there have been that were not anticipated.

Improvement to the overall ICS

There are a variety of improvements that you may wish to make to the overall ICS and conform by measurement, e.g.

Advancement to a higher Category

In implementing such an improvement you will undoubtedly know what changes need to be made to effect the transition.  Theoretically, you only need to re-measure the control class (Table 2) and re-apply the criteria (Table 3) to the changes.  However, it is prudent to identify what criteria are borderline and monitor those as well.  In that way, you can check that the operational effectiveness does not decrease as a result of changes made. 

Removal of redundant controls

A control may be redundant if:

  • The events that the control is designed to trap are universally trapped by some other control.

  • The control does not trap other events.

The latter condition ensures that controls are not removed because they are redundant with respect to one event but not some other.  If a control is truly redundant its removal should lead to improvements in cost effectiveness (which can be gauged using Table 4) together with similar improvements in business efficiency.

The first step would be to identify the controls that are candidates for removal and then monitor/measure how they work.  Can you establish, for example, the number and frequency of the events that each control does trap? The second step is to establish a back-out plan such that, if removal of a control harbingers disaster, the control can be speedily put back! The third step is to remove the control and monitor/ measure how the other controls behave to establish confidence that the removal achieves its objectives.

Increased cost effectiveness

The idea here is to measure the cost effectiveness of either the whole ICS or a selected subset of it, by first acquiring the data necessary to apply Table 4.  If improvements are required, the changes to particular controls are then identified.  These are discussed below.  On making the changes the measurements are repeated to confirm that the changes have met their objectives. Note that the removal of redundant controls is likely to have a positive impact of the overall cost effectiveness of the ICS.

Improvement to an individual control

There are a variety of improvements that you may wish to make to an individual control, or group of controls, and conform by measurement, e.g.

Advancement to a higher Class or within Class

Moving from reactive to detective/ preventive, or moving from detective to preventive, is generally implemented by removing one control and replacing it by another.  On the other hand, moving from one reactive class to another reactive class, or moving from one detective class to another detective class, is generally implemented merely by improving the particular characteristics of the control rather than changing it beyond recognition.

For example, moving from Class 7 to Class 6 may mean documenting what we did last time (or should have done in hindsight).  Moving from Class 6 to Class 5 requires pre-deployment of some parts of the plan.  The plan, however, is essentially the same in each case.

Likewise, moving from Class 4 to Class 3 and hence to Class 2 may be accomplished by improving the time to detect and/or the time to fix though educating and training staff, and essentially little change to anything else.

In each of the above cases we need only to measure the time parameters to indicate improvement or not as case be.

Self-policing and fail-safe properties

The value of a self policing procedure is that it promptly detects a control failure, allowing some other control to take over.  As shown in Figure 6, if a control is not protected by a self-policing procedure, a control failure may go undetected and ultimately manifest itself following expiry of the time window.  Is there evidence of this happening in the ICS?  If there is, you not only have a need for a self-policing procedure but a means to monitor its effect.  Once implemented the number of events trapped as Class 7 should reduce, ideally to zero.  Note that self-policing and fail-safe properties are requirements of the higher order categories of ICS.

Increased cost effectiveness

Advancement to a higher Class or improvements within class are likely to have a beneficial effect on the cost-effectiveness of the control.  Other improvements may be made by paying attention to the costs involved, in particular the cost of the control.  Note that reducing the time to detect (see our cost-effectiveness worked example) may automatically result in a shorter time to correct and a lower cost of fix. Use Table 4.

A Worked Example (continued again)

Let us return to our software company example and this time consider the candidate ICS identified when we were considering the cost-effectiveness problem.  The category for ICS#1 is A, as determined previously.  The chosen ICS (#3) adds a new control, which is Class 2 non-degradable.  By itself this is insufficient to change the category of the ICS.  However, it was chosen because it ought to  increase the cost effectiveness of the ICS.  We need to monitor the cost of the control, the cost of fix and any impact penalty to confirm this.  We can also monitor the improvements in the control itself by recording the time to detect and the time to fix for the events that it discovers.  Other questions that we ought to monitor are:

  • Are there errors that it was designed to detect that escape detection?

  • Are these trapped by the software testing control? 

  • What proportion of these escape detection and are ultimately found by the customer?

             
             
             
 
Gamma is an ISO/IEC 27001:2005 and BS EN ISO 9001: 2000 registered company, certified for the provision of information security consultancy.  BSI certificate numbers IS 85916 and FS  30710.  Please send comments to webmaster@gammassl.co.uk or complete our Visitors'Book. Gamma Secure Systems, Diamond House, Frimley Road, Camberley, Surrey, GU15 2PS, UK Tel: +44 1276 702500 - Fax: +44 1276 692903Copyright © Gamma Secure Systems Limited 2004
 
 
Page last updated: 18 March, 2004