Home	About Gamma	Tour our Web Site	Events	White Papers	Services	Visitors' Book	How to contact us
				IMS	Internal Control	ISMS	Smart Cards	Common Criteria

Objective

Operational effectiveness does not necessarily imply cost effectiveness. To determine the cost effectiveness of the ICS we need to apply other metrics, e.g. Cics, as identified in the fundamental model.

Measurement

Measurement in this case is effected by taking each control and comparing its financial parameters with those that would apply to controls that belong to other classes. There may be a variety of objectives in undertaking these measurement.  Some of the most important, albeit general questions are summarised in Table 4 and discussed in the ensuing subsections.  Other objectives may be very specific.  Our worked example is a case in question and considers the design of an ICS to maximise the profit on a particular contract.

Preventive Control

Should we be using a preventive control? The answer is likely to be yes if the cost of using a preventive control is less than the sum of the cost-to-fix and possible impact penalties for all the events that the preventive control is designed to detect, i.e.

C_{preventive control}     <     Σ_{all events, j, that the preventive control detects} (Cf_j + Ip_j)

The cost of using a preventive control includes the cost to buy/develop, install, configure, commission, operate, train people in its use, audit its use and maintain it. We have not included in the above formula the cost of the controls that would otherwise perform the task of the preventive control as we recommend that they be retained in case of a control failure in the preventive control. An impact penalty will only occur if the corresponding existing controls are Class 4 or lower.

Note the summation over all the events that the preventive control is designed to detect.  In practice, this number will be an estimate as it concerns the future.  If the cost-to-fix and possible impact penalty for each event is constant (or can be considered approximately so), the inequality becomes:

C _{preventive control}     <     N * ( Cf + Ip ) _average

where N is the number of events.  If this number is very small and average cost-to-fix and impact penalty is also small, then it is very likely that a preventive control will not be cost effective.  On the other hand, if either N or the average cost-to-fix and impact penalty is very large, the use of a preventive control is most likely to be a very good idea.

Detective Controls

Should we improve the efficiency of our detective controls? If the control is a Class 4 then it detects the event too late for it to be fixed within the time window.  There is therefore a cost-to-fix and an impact penalty.  If we convert the control to a Class 3 then there is no impact penalty.  Thus, for the upgrade to be worthwhile, the cost of the upgrade must be less than the average impact penalty times the number of events, i.e.

C_upgrading  <     N * Ip _average

We have assumed here that the cost-to-fix is about the same for the two classes of control.  This is a reasonable assumption if the Class 3 allows the event to be fixed just prior to the expiry of the time window while the Class 4 fixes it immediately after.  The cost-to-fix, however, may be dramatically reduced as we reduce the time taken to detect the event, thereby moving from a Class 3 to a Class 2 control.  This upgrade is worthwhile if the cost of the upgrade is less than the average reduction in the cost-to-fix times the number of events, i.e.

C _upgrading  <     N * ( Cf _{in Class 3 case} -  Cf _{in Class 2 case}) _average

If we pre-deploy all or part of a BCP there will be an associated pre-deployment cost and a maintenance cost.  Pre-deployment costs may include equipment purchase/lease, building purchase/hire, insurance, extra staff, training, commissioning, regular tests and practices, etc.

Prior to invoking the BCP there may also be an associated business benefit, B_BCP, which will offset the pre-deployment costs, C_pre-deploy.  For example, a redundant IT installation, kept in a state of readiness  in case the main installation fails, or is otherwise rendered unavailable, could be used for other purposes, e.g. systems development.  Of course, when the BCP is invoked this benefit will be lost, hopefully for a short time but it will be lost.  We need to factor this loss into our cost-effectiveness considerations.  We will do this in a moment.  Let that loss be B_BCPloss.

By pre-deploying the BCP we gain time.  Recovery from abnormal to normal operations will be quicker, and the impact penalty will be reduced.  Let that reduction be IP_reduction.

Arguably for pre-deployment to be cost-effective:

C_pre-deploy - B_BCP   <     IP_reduction   -   B_BCPloss

Invocation of the BCP ought not be a frequent event, otherwise we should be considering Class 2, 3, or 4 controls as our main line of defence.  Suppose we estimate that over a period of Y years the BCP is invoked N times, our inequality then becomes:

Σ _{Y years}   ( C_pre-deploy - B_BCP )   <    N  *  ( IP_reduction   -   B_BCPloss )

BCP Need

Do we need a BCP?  The first question to ask is what would be the impact penalty if the unthinkable was to happen?  Having answered that question, the next question is "is that an acceptable risk".  If it is not, then you need a BCP.

Surprisingly, in response to the second question the answer "if I am still alive, I'll just start up again" is a Class 6 control.  The question is then, how much of this plan should we pre-deploy (clearly following carefully consideration and suitably refinement of this somewhat embryonic/flippant  BCP!).

A Worked Example (continued)

The foregoing allows us to answer general, albeit important, questions about the effectiveness of our ICS.  By way of our worked example, however, we look at something very specific and entirely in tune with what the focus of cost-effectiveness for a commercial organisation ought to be. i.e. profit.

Let us suppose that our small software company is being invited to bid for a new project.  The contract is for a fixed price with stage payments. The development is scheduled to last one year.  The final payment is due on satisfactory completion of the project.  If there is an overrun, there is an impact in the form of a revenue penalty which increases with the extent of the overrun. The agreed schedule is shown in Figure 8 (which we reproduce here for convenience).

Figure 8: The agreed development schedule

As previously mentioned, the company usually relies on an ICS that is predicated solely on program testing.  In particular, there are no formal design/code reviews.  In costing the project, the company anticipates using a full time team of three analyst/programmers of the same grade and salary costs.  Expressed in terms of some arbitrary monetary units (MU), the cost of the project is therefore 36MU plus some allowance for overhead, charged at 5MU per person per year.  This gives a total of 51MU.  To remain competitive the company wishes to charge the client 60MU, yielding an anticipated profit of 9MU.  There is also a one-year maintenance component of 10 MU, which applies for the 12 months following client acceptance.  Its purpose is to fix program bugs that manifest during the operational use of the software.

From experience, the company realises that the most likely worst case scenario is a top level design error (the event) that causes rework affecting 1/3rd of the program modules.  It estimates (see Figure 9) the cost (in MU) to fix the problem as a function of the month in which the error is detected by the ICS.  Past experience also indicates that all of the 10MU maintenance provision is used up, resulting in effectively no profit or loss.

Before agreeing the contract, the company considers alternative forms of ICS.  It also notes that the revenue penalty is quite steep, being 5MU for any overrun, increasing thereafter by 1MU per month.

Figure 9: Cost (in MU) of fix as a function of month in which event is detected

It reasons:

Case 1

Based on past experience, the company believes that if the event occurred, at worst it would be detected in month 11 during the integration testing, but no later.  According to Figure 9, this would correspond to a cost to fix of 6MU.  The company also realises that, given the late detection of the event, there would be an overrun of 2 months, causing a revenue loss of 6MU, together with an overhead component of 3*5/12 = 2.5MU.  Thus, the anticipated profit margin of 9MU is at risk of being reduced to a loss of 5.5MU, should the event occur.

Case 2

The company argues that through the use of certain more sophisticated testing techniques, at an extra staff cost of one extra analyst/programmer, half time from month 5 onwards, plus a one-off software purchase of 2MU, at worst the error would be detected in month 6.  The company also believes that the improved testing techniques will have a positive impact on the maintenance phase, which ought to result in a profit of 5MU, whereas usually there is none.

The increased costs of the ICS are, in this case, 5.7MU.  The cost of fix, should the error occur is 1.5MU.  Thus, the overall profit (inclusive of the maintenance component of the contract) would be 6.8MU should the event occur and 8.3MU should it not.

Case 3

The company recognises that an alternative approach would be to entertain design/code reviews.  The company decides that this can be accomplished with the proper use of certain overhead resources on a very part-time basis in order to moderate the reviews.  The project team must, however, be trained in the techniques that are to be used.  The training cost will be 2MU.  In the worst case, the event should be detected by month 3.

The company also estimates that these techniques will also have a positive benefit on the maintenance component, but perhaps not so great as Case 2.  They estimate a profit of 3MU. 

The increased costs of the ICS are, in this case, 2MU.  The cost of fix, should the error occur is 0.25MU.  Thus, the overall profit (inclusive of the maintenance component of the contract) would be 9.8MU should the event occur and 10MU should it not.

Case 4

Alternatively, the company argues that it can dispense with the training course and use a more experienced person - a chief analyst/programmer - in exchange for one of the analyst programmers. The chief analyst/programmer costs 25% more than an analyst programmer, but is competent in the required techniques and is also experienced in providing successful on-the-job training.  At worst the event again should be detected by month 3.  The same benefit should apply to the maintenance phase as in Case 3.

The increased costs of the ICS are, in this case, 3MU.  The cost of fix, should the error occur is 0.25MU.  Thus, the overall profit (inclusive of the maintenance component of the contract) would be 8.8MU should the event occur and 9MU should it not.

In summary

The company decides upon ICS#3.

An alternative scenario

It is interesting to consider what might happen if the contractual situation was quite different.  What would happen, for example, if the contract was time and materials and there was no penalty clause. Suppose the company elects to charge its analyst programmers at a daily rate, equivalent to 1.67 MU per month, and its chief analyst programmers at a daily rate, equivalent to 2.09 MU per month.  These rates allow for overhead and profit.

Ignoring maintenance, as that being on a time and materials basis, it would appear that the greater the number of bugs, the more maintenance work is required and therefore the greater the profit(!), we have:

Note that in each case, the company makes a greater profit when the event occurs - it is almost as if they are being paid to do badly.  ICS#3 is now the worst option, as it always results in the least profit  However, it does represent the least cost to the client.

With regard to the maintenance component of the contract, the company would charge 11.8 MU, resulting, in Case 1, with a profit of 1.8 MU.  In Cases 3-4 less work is involved and therefore less profit (in fact 0.89, 1.25, 1.25MU respectively).  In view of this, the company could elect to go fixed priced for Cases 3-4. The anticipated profit figures (and client charges including maintenance) would therefore be:

Note that ICS#2 is the best from the perspective of making a handsome profit, whilst still not being the most expensive option from the perspective of the client.  What we see here is the interplay between the client taking the risk during the development phase and the company taking the risk during the maintenance phase.  The company is prepared to do this because of the superior ICSs involved in Cases 2-4. Note also that in a highly competitive situation, Case 3 wins as it allows the company to offer a low price whilst still demonstrating good value and competence.

Comment

Just who takes the risk in these situations is an important decision.  For ICS#1 (see Table 7 above) the client takes all the risk.  If the company (i.e. the contractor) makes an error, the client pays for it. With ICS#2-4, the company has a better ICS, which in Cases 2 and 4 does cost the client more.  However, in all cases the client pays less than in Case 1 if the company does make an error.  The question therefore is "is it in the client's best interest to pay the company to improve the effectiveness of its ICS?" Table 7 suggests that the answer is not only "yes" but also that it can be done for next to nothing (compare ICS#3 with ICS#1).

Perhaps this example gives us an insight as to why customers do put pressure on large suppliers to introduce management systems (whether for risk, or subordinate areas such as quality or information security).