Welcome to NBlog, the NoticeBored blog

I may meander but I'm exploring, not lost

Jul 12, 2013

PRAGMATIC Security Metric of the Quarter #5

Example Information Security Metric of the Fifth Quarter

The PRAGMATIC scores for another 3-month's worth of information security metrics examples are as follows:

Example metric P R A G M A T I C Score
Information access control maturity 90 95 70 80 90 80 90 85 90 86%
Security policy management maturity 90 95 70 80 88 85 90 82 88 85%
Number of important operations with documented & tested security procedures 95 96 91 85 95 84 62 90 60 84%
Information security budget variance 70 90 85 77 80 77 80 90 95 83%
% of information assets not [correctly] classified 75 75 97 85 90 80 80 80 80 82%
Policy coverage of frameworks such as ISO/IEC 27002 70 75 90 69 85 76 72 65 85 76%
% of policy statements unambiguously linked to control objectives 92 91 64 60 85 65 45 75 75 72%
Rate of change of emergency change requests 64 71 69 73 78 70 70 69 83 72%
Total liability value of untreated/residual risks 88 98 59 33 96 33 77 38 10 59%
Entropy of encrypted content 78 66 23 78 3 93 74 79 34 59%
Embarrassment factor 26 38 20 50 63 72 40 87 87 54%
% of security policies that satisfy documentation standards 66 47 79 45 74 38 44 50 35 53%
Patching policy compliance 66 52 55 77 19 36 11 8 5 37%

Top of the heap are two maturity metrics scoring 85% and 86%, with a further 3 metrics also scoring in the 80's.

While it is tempting to recommend these and other high-scoring metrics to you, dear reader, please bear in mind that they were scored in the context of a fictional manufacturing company, Acme Enterprises Inc.  The scores reflect the perceptions, prejudices, opinions and needs of Acme's managers, given their current situation.  Things are undoubtedly different for you.  We don't know what's really important to you, your managers and colleagues, about information security.  We have no idea which aspects are of particular concern, right now, nor what might be coming up over the next year or three.  Hence we encourage you to think critically about the way we describe the metrics, and preferably re-score them.   

Furthermore, PRAGMATIC scores alone are not necessarily a sound basis on which to select or reject metrics.  It's not that simple, unfortunately, despite what you may think given the way we bang on and on about PRAGMATIC!  The scores are intended to guide the development of an information security measurement system, a well-thought-out suite of metrics plus the associated processes for measuring and using them.  Considering and scoring each security metric in isolation does not build the big picture view necessary to measure information security as a coherent and integral part of the organization's management practices.

The book describes PRAGMATIC scoring as the heart of a comprehensive method, an overall approach to information security metrics.  The method starts by figuring out your metrics audiences and their measurement requirements, building a picture of what you are hoping to achieve.   Knowing why certain security metrics might or might not be appropriate for your organization is arguably even more important than knowing which specific metrics to choose ... but, that said, the act of gathering, contemplating, assessing and scoring possible metrics turns out to be a productive way both to determine and to fulfil the needs.  It's a deliberately pragmatic approach, a structured method that achieves a worthwhile outcome more effectively and efficiently than any other approach, as far as we know anyway.  Perhaps you know different?