Welcome to NBlog, the NoticeBored blog

I may meander but I'm 'exploring', not lost

Apr 30, 2012

SMotW #4: Control policy compliance

Security Metric of the Week #4: Proportion of critical controls consistent with controls policy

This week's security metric example measures the proportion or percentage of critical controls that are consistant or comply with the associated policies.

The metric assumes that policies are defined, at least for controls deemed "critical".  Arguably, all controls that have documented policies are critical but not necessarily so, and some critical controls may not have policies. 

Examples of control policies are “access is permitted unless expressly forbidden” and “trust is transitive”.  They might also be termed, or be part of, 'control specifications'.  If such policies/specifications are not formally defined and mandated, implementation is more likely to be inconsistent and perhaps inappropriate or inadequate. 

This is a relatively Costly metric due to the need to assess the consistency or compliance of critical controls with the associated policies.  Such detailed compliance checks can be slow, hence its PRAGMATIC score is also depressed by the Timeliness criterion.  However the metric's overall score works out at a respectable 72%.


P
R
A
G
M
A
T
I
C
Score
83
92
80
83
89
82
32
70
35
72%





If measured and reported consistently, the metric should in theory drive improvements in the consistency/compliance, which in turn should help ensure that critical controls are correctly configured.  However, the metric could prove counterproductive if controls are artificially downgraded from "critical" simply in order to improve the numbers, suggesting perhaps the need to track and report the number of "critical" controls in parallel, or at least to find a way to prevent inappropriate downgrades (e.g. insisting that they are formalized through the change control process).

What do you think?  Would this kind of metric work for you?  Do you measure this already, or do you have a different way to achieve similar ends?

Apr 27, 2012

Traffic light metrics fail in NZ

"Traffic light reporting" generally implies metrics that should make sense to anyone having normal color vision, and an appreciation of the conventional sequence of colors meaning stop, ready and go.  Short of binary yes/no type metrics (such as certified compliance), metrics doesn't get much simpler than traffic-light reporting ... or does it?  According to April's eBulletin from New Zealand's Ministry of Civil Defence & Emergency Management:
"The different meanings of the colours used for tsunami threat level maps in national warnings and those used for tsunami evacuation zone mapping (specifically red, orange and yellow) were identified as confusing in exercises and real events. For instance, in the threat level maps red indicated a ‘severe threat’ of 8m+ amplitude while yellow meant a ‘moderate threat’ of 3-5m. In the Tsunami Evacuation Zones Guideline the opposite applies, with the ‘red zone’ being evacuated for the lower level threat and the ‘yellow zone’ in the extreme (worst) case."
The piece went on to describe a new set of 6 color-coded tsunami threat levels and definitions, developed by the Tsunami Working Group.  While there is a green level, red and yellow/orange/amber are now conspicuously absent.  Perhaps they were too obvious for the bureaucrats.

While it's easy for me to poke fun at the governmental approach to disaster planning, the obvious fact remains that NZ sits in a geologically-hyperactive region.  Metrics relating to those and other incidents, plus scientific data from the geologists and other experts in plate tectonics stream into the government's risk assessment, planning and budgeting, while video footage from the Christchurch quakes, plus the Indonesian and Japanese tsunamis, represents the most graphic metric of all.  Without the government's efforts, it is unlikely that we citizens would be able to coordinate our efforts to prepare for the next Big One.  Let's just hope they have their act together.

Rgds,
Gary

Apr 23, 2012

SMotW #3: Unpatched vulnerabilities

Security Metric of the Week #3: Number of unpatched technical vulnerabilities

This week's security metric is, at face value, straightforward: "simply count the number of technical/software vulnerabilities that remain unpatched."

In practice, as stated, the metric is quite ambiguous.  Are we meant to be counting the number of different vulnerabilities for which patches are not yet applied, or the number of systems that remain to be patched, or both?  If the organization is using a distributed vulnerability scanning utility that identifies missing patches, perhaps the management console gives us the metric directly as a number, or the average number of missing patches per machine.

The Predictability score for this metric would be higher if it also somehow addressed unknown and hence currently unpatchable vulnerabilities, plus nontechnical vulnerabilities (e.g. physical vulnerabilities and vulnerabilities in business processes).  Software vulnerabilities are important, but the organization needs to address risks, meaning other kinds of vulnerabilities as well, plus threats and impacts. 

The metric also disregards vulnerabilities for which no patch is currently available (so-called "oh-days").  If we have the capability to identify them (meaning some form of penetration testing or fuzzing), the metric could be expanded to cover oh-days as well as patchable vulnerabilities.  The problem here is that practically all software has bugs, some of which are security-relevant oh-days.  The number of oh-days we find is a compound function of the quality of the software design, coding and pre-release testing, and the quality of the vulnerability assessment/post-release testing.  The latter, in turn, depends on the amount of effort/resources applied to the testing and the expertise and tools the testers use.

If very carefully specified and rigorously measured so as to standardize or normalize for these variables, the metric might conceivably have value to track or compare different systems and test teams, but that's a lot of effort and hence Cost.  In short, there are so many variables that the more complicated metric would have little value for decision support: it would probably end up being a facile "Oh that's nice" or "That looks bad" metric.  Time lag may be an issue since it takes time to identify and characterize a vulnerability, put the corresponding signature into the scanning tools, scan the systems, examine and assess the output, and finally react appropriately to the findings - meaning a low Timeliness score.  The overall PRAGMATIC score for the relatively simple metric as stated works out at 68%.  Here's the detail:


P
R
A
G
M
A
T
I
C
Score
80
64
80
70
80
75
25
85
52
68%

Apr 16, 2012

SMotW #2: Coupling index

Security Metric of the Week #2: Coupling index

Our second 'metric of the week', Coupling index, is a measure of the degree of interdependency between things such as:
  • IT systems;
  • Business processes;
  • Business units, departments and teams;
  • Organizations.
Coupling has an impact on the risk of cascade failures (known colloquially as 'the domino effect').  In tightly-coupled situations, upstream issues will quickly and dramatically affect downstream elements.  In loosely-coupled situations, by contrast, there is more leeway, more 'slack' so downstream effects tend to be less evident, show up less quickly if at all, and generally have less impact on the organization.

Contrast traditional mainframe-based batch-processing financial systems against real-time ERP systems, for instance: if something causes a single batch to fail on the financial system, it can generally be corrected and rerun without too much trouble, provided it still completes within the batch window.  However, a similar failure on an ERP system can sequentially topple a whole series of highly interdependent operations, bringing the entire ERP and the associated business activities to a crashing halt.



P
R
A
G
M
A
T
I
C
Score
68
85
50
60
72
47
35
61
42
58%






This candidate metric scores a mediocre 58% on the PRAGMATIC scale.  It is quite strong on Relevance but other aspects such as Timeliness and Cost detract from the final score. These criteria are issues because of the practical difficulties of quantifying the 'degree of coupling' in meaningful and comparable terms, especially across such a diverse set of factors as noted in the bullets above. 

COBIT 5 metrics

COBIT version 5, just released by ISACA, suggests numerous sample metrics in the Enabling Processes document - typically four or five metrics per goal of which there are 17 enterprise goals plus another 17 IT-related goals, giving approximately 150 metrics in total.

For example, supporting the third financial enterprise goal "Managed busines risk (safeguarding of assets)", the following three metrics are suggested:
  1. Percent of critical business objectives and services covered by risk assessment.
  2. Ratio of significant incidents that were not identified in risk assessments vs. total incidents.
  3. Frequency of update of risk profile.
The text introduces the metrics thus: "These metrics are samples, and every enterprise should carefully review the list, decide on relevant and achievable metrics for its own environment, and design its own scorecard system."  Fair enough, ISACA, but unfortunately COBIT 5 does not appear to offer any advice on how one might actually do that in practice.  How should we determine which metrics are 'relevant and achievable'?  What is involved in 'designing a scorecard system'? 

Some readers may assume that they should perhaps be using most if not all of the 150 sample metrics.  Others may feel less constrained to the examples, but may also assume that 150 is a reasonable number of metrics.  We beg to differ.

The PRAGMATIC approach is well suited to this kind of situation.  It is quite straightforward to assess and score ISACA's 150 metrics, comparing them alongside suggestions from various other sources in order to identify those that deserve further investigation: we call them 'candidate metrics'.  In conjunction with the people who will be receiving and using the metrics to support business decisions, the shortlisted candidate metrics can be further considered using the PRAGMATIC criteria and refined before finally selecting and implementing the very few security metrics that are actually going to make a positive difference.



Apr 9, 2012

SMotW #1: Unowned information asset days

Security Metric of the Week #1:  Unowned information asset days

We will be introducing, discussing and scoring a "Security Metric of the Week" (SMotW) through this blog for the forseeable future.  We know of literally hundreds, if not thousands of candidate metrics, including those that are proposed in various standards, books and published lists of metrics, along with a few of our own creation.  On top of that, we frequently come across novel metrics during our consulting work or simply in the stream of information that flows past every well-connected professional every day.  If you would like us to consider, score and discuss your favorite security metric, by all means email us, raise a comment on this blog, or join the Security Metametrics discussion forum.

Unowned information asset days is a candidate metric concerning asset ownership, an important governance concept that underpins accountability for the adequate protection of information assets.  

It is a measure of the number of days that information assets remain without nominal owners for various reasons such as:
  • It is a new asset for which an owner has not yet been designated by management;
  • The designated owner has left the organization, or has been given a new set of responsibilities and is no longer accountable for the asset's protection;
  • The information asset ownership practices are not in place or are not working well. 
Unowned or orphaned assets are unlikely to get much attention and care, in other words if nobody feels responsible for their protection, perhaps nobody will bother to risk-assess, secure and generally look after them.  Just look at what happens to 'pool cars' and typical office printers to see what we mean.



P
R
A
G
M
A
T
I
C
Score
40
51
84
77
74
86
92
94
82
76%









Using the PRAGMATIC method, this metric scores a respectable 76%.  It is held back a little by the Predictive and Relevant criteria since ownership and accountability are not sufficient, in themselves, to ensure that information assets are adequately secured.  Driving up asset ownership and accountability should lead to better security in time, but only as part of a coherent and comprehensive approach to information management, governance and security. 
 

Apr 7, 2012

Welcome!

Hello world!

We have started this blog to enhance the SecurityMetametrics website and support both our book (due out in August) and the global community of professionals using, trying, or intending to use metrics to measure and improve information security.

This blog, like the website and the discussion forum, is brand new and will take a while to get up to speed, but we hope it will soon start to deliver real value.  We have some great material to share with you, and definitely welcome your feedback comments and improvement suggestions.  The field of security metrics is wide open for creative, innovative approaches, academic input and practical experience.  As the field slowly matures, join us as we do our level best to nudge things along in the right direction.

We'll kick off by releasing our first "metric of the week" on Monday ...

Kind regards,

Apr 5, 2012

Office printer hacks and security

An infosec blogger describes the fun he had using nmap to analyze typical office printers (that's an excellent Google translation of the Spanish original). 

Most printers have web configuration interfaces on the network and, thanks to having no passwords or (well known) default passwords, hackers can play pranks such as printing junk, resetting the admin pasword or changing the printer's IP address (e.g. deliberately conflicting with another device on the network). 

All pretty juvenile really, little more than geek vandalism, but I guess printing directly to the device might conceivably be of concern if the printer is loaded with check blanks and relies on security on the print server to prevent anyone who feels like writing themselves a big fat check simply doing it.  Given that they would need access to the printer's network, knowledge of the print formatting necessary to put all those zeroes in the right place, some way of slipping past the business process controls normally associated with company checks and, of course, a safe way to cash-out, the probability is quite low, even if the financial impact could be serious.


Something else caught my beady eye though.  Tucked away in the blog is the throwaway line "In the case of Ricoh printers, which have a Document Server, we can see some documents stored (as images) on the printer itself."  Now that could be more serious.  Want to see what the boss - or head of HR - has been printing lately?  Curious to find out what has been sent or received by FAX on the office multifunction device?  Keen to read whatever has been submitted to the Secret printer in the secure corner office?  Mmmm, that could be a more significant risk.

For a more detailed look at printer hacks, see this piece by Adrian "Irongeek" Crenshaw or read this provocative 2002 paper by Ltlw0lf which reminds me that anyone with physical access to a printer and sufficient technical nouse (say, a printer maintenance engineer, or a social engineer pretending to be one - perhaps turning up to fix a printer that says it needs maintenance as a result of him hacking the message on its little display panel) may be able to pull/swap its hard drive and analyze the data at his leisure.  Using a compromised network printer as a launch point for further hacks, and a fairly safe place to store purloined data, is a possibility.  Again, the risks are probably low enough to be insignificant compared to many others for most of us, but there are situations where they may be of concern.


As to what we can do to secure our network printers, the articles are rather light on practical advice.  We should physically secure the printer and implement suitable policies and procedures against social engineering attacks (accompanied by effective security awareness, naturally!).  Changing the admin password seems like A Jolly Good Idea, and of course firewall the network, if not the printer itself.  Monitor the network like a hawk is great advice for any organization that has a surfeit of diligent network security analysts with boundless time on their hands. Other than that, we're largely in the hands of the printer manufacturers and their software engineers.  


By the way, if security patching your printers seems like a good move, consider that the very same software update mechanism might itself represent a vulnerability.  Rock, meet hard place.

Regards,
Gary (Gary@isect.com)