Welcome to NBlog, the NoticeBored blog

I may meander but I'm 'exploring', not lost

Jul 9, 2012

PRAGMATIC Security Metric of the Quarter #1

PRAGMATIC Security Metric of the First Quarter

Having scored and discussed fourteen Security Metrics of the Week during the first three months of this blog, it seems appropriate now to take a step back, review the metrics we have discussed thus far and consider the value of the PRAGMATIC process.  

Here are the fourteen security metrics, tabulated in descending order of their overall PRAGMATIC scores.  Click any of the metrics to read more about them and discover why we scored them thus.


Metric

P

R

A

G

M

A

T

I

C   

Score

Discrepancies between physical location and logical access location

75 76 72 90 82 75 85 83 60 78%
Information security policy coverage

75 82 92 78 80 70 73 60 81 77%
Unowned information asset days

40 51 84 77 74 86 92 94 82 76%
Number of unsecured access points

95 80 90 70 85 77 45 75 55 75%
% of critical controls consistent with controls policy

83 92 80 83 89 82 32 70 35 72%
Days since logical access control matrices for application systems were last reviewed

55 80 95 30 80 85 60 70 80 71%
Number of unpatched technical vulnerabilities

80 64 80 70 80 75 25 85 52 68%
Coupling index

68 85 50 60 72 47 35 61 42 58%
Vulnerability index

74 85 71 74 60 32 46 33 19 55%
System accounts-to-employees ratio

74 67 38 39 68 42 36 83 44 55%
Corporate security culture

80 80 60 55 75 55 10 45 10 52%
% of purchased software that is unauthorized

71 51 90 75 82 35 13 20 6 49%
Security budget as % of IT budget or turnover

13 3 16 2 2 0 4 18 88 16%
Number of firewall rules changed 2 1 1 10 2 33 14 4 17 9%

In simple numerical terms, the metric Discrepancies between physical location and logical access location is the leader of this little pack which qualifies it as <cue drum roll> our first PRAGMATIC Security Metric of the Quarter.  In fact there's clearly not much to choose between the top four metrics in the table in terms of their overall PRAGMATIC scores.  The scores and hence rankings may well have changed if we had made different assumptions in the scoring, or of course if we had altered the specification/wording of individual metrics to address the issues we identified and hence altered their scores.  Furthermore, your scoring of the metrics may differ from ours due to  differences in how we each understand and interpret both the metrics and the PRAGMATIC approach.  We don't have the same experience as you, our biases differ, our presumptions and organizational contexts differ and no doubt we have different potential audiences and purposes in mind for these metrics.

That whole line of discussion is moot, however, since we are not claiming that the PRAGMATIC approach is scientific and objective.  Our top-scoring metric is not necessarily the best of the bunch under all circumstances for all organizations, just as the lowest scoring metric may be appropriate and score more highly in certain situations.  The approach simply offers a rational way to consider the value of and compare various security metrics, to elaborate on their pros and cons, to identify ways in which the metrics might be re-phrased or materially altered to improve them, and most of all to facilitate a more informed and productive metrics discussion with management.  Even if you simply use the process to shortlist the most promising from a bunch of metrics candidates, leaving the final selection to management, isn't that a worthwhile outcome?

There's plenty more to say yet about being PRAGMATIC, including ways to glean further useful information from the data in the scoring table above, but we'll leave that for the book, future blog pieces, seminars and papers.   Meanwhile, do please let us know about your favorite security metric and we'll see we make of it.  We look forward to your comments on this blog and emails, especially constructive criticism and creative ideas to make the PRAGMATIC approach even more effective.  Over.