Metrics to improve infosec and risk management

A thoughtful and well-written paper by David Lacey is strong on linking infosec/risk management metrics to organizational objectives, and on using them to improve security practices. David references a paper from the 1930 stating that for every significant safety incident there are around 20 minor incidents and 300 near-misses - an interesting analogy that reminds me of the "days since a lost time accident" boards outside many British factories in the latter half of the 20th Century. I can just imagine a "Days since a major security incident" counter on the corporate intranet, with a click-through to suppporting details on the nature of the last incident and a count of minor incidents, or perhaps even a "security events seismograph" showing the incidence and gravity of incidents. Implicit in this kind of approach, of course, is that someone needs to know about all the incidents and ideally the near misses, meaning that internal reporting must be mandatory. The same principle applies in the public context, hence the reason that many US states already mandate disclosure of privacy incidents, and the UK's Information Commissioner is considering a similar approach.

A few of the infosec metrics suggested in David's paper could be accused of falling into the trap of being easy to count or measure but providing limited value to management, whereas most are more worthwhile. I'm constantly on the lookout for 'elegant' metrics - things that are not too difficult to count, measure or calculate and that have been shown to indicate genuinely useful facets of the efficiency or effectiveness of the organization's information security management system. One of my favourites is the proportion of all system changes processed by IT as emergency changes: this has been shown to correlate closely with the department's process maturity and, I believe, closely reflects the stability and security of the systems.

I like David's suggestions to track compliance exceptions for various categories of control. That ties in neatly with the concept of accountability, namely that anyone who requests a policy exception has to accept personal accountability for the associated risks, quite a burden for any manager. Measuring and reporting exceptions thus provides a mechanism to remind those people carrying the burdens until the exceptions are cleared (either by upgrading the controls or, potentially at least, downgrading the policy requirements) or until incidents occur and they are 'called to account' (aka walked off site).

[For those who don't recognize the name, David Lacey is a visionary, one of the founding fathers of BS 7799 (now the ISO/IEC 27000 family). The first version of 7799 was largely based on the internal security policy manual generously contributed by David's employer at that time - the Royal Duitch/Shell Group.]