Boiling the O.C.E.A.N.

Metrics projects that are intended to consolidate and report on the state of security for an organization rarely fail for a lack of measures. Information technology systems, processes and projects all throw off an impressive amount of data that can be captured and counted. The Complete Guide to Security and Privacy Metrics suggests over 900 metrics, and NIST Special Publication (SP) 800-55 Rev. 1, Performance Measurement Guide for Information Security extends this analysis from the system level to an executive view by providing a framework for summarizing the results.

So given all of the measures, structure and guidance available, why is it so tough to be successful? The silent killer in this space is often a lack of focus: too many metrics, too much aggregation, and too little analysis connected to business problems and goals to provide useful insight.

Instead, its better start with the stakeholders and focus on fully understanding their goals and decisions without limiting the conversation with assumptions about what is or isn’t going to be measureable.

Consider this subset of stakeholders, and some of their goals:
Executive management – financial health and strategic direction of the organization. Are we profitable and are we executing effectively in the markets we serve?
Risk governance / Security management – are we keeping risk at an acceptable level? Are we making the best use of the security resources we have?
Line Management – are we achieving operational goals, and aligning with strategic initiatives?
These questions become an effective filter for removing the measures that don’t matter, and for finding common measures that, with analysis, can serve many different purposes. Here’s where it may be useful to classify measures from a stakeholder perspective in terms of the types of decisions that they enable:
Output measures - what is the primary deliverable from a given team?
Coverage measures – how many locations, systems or groups are covered by a given process or policy?
Exposure measures – what proportion of the environment stores or processes regulated information?
Activity measures – how many requests have been received during a given reporting period? Addressed?
Null measures – which teams have not provided data?

The last category is an important one, as it highlights the difference between a measure and a metric. A measure is an observation that increases your understanding about a situation and improves the quality of decision making; a metric is a standardized measurement. Inconsistent, incomplete and missing data from key teams or groups are an important measure of program maturity. Sometimes it’s what you can’t count that counts.

Above all else, resist the pressure to measure everything. A few well-chosen measures will allow for versatile and powerful analysis. There are literally dozens of ways to analyze and present a limited number of well-chosen data points. And when captured consistently over time, the correlations between seemingly unrelated activities offer the opportunity to surprise.

Can you afford bad security?

Within the current economic turmoil and uncertainty its becoming clear that the global economy is slowing, pressuring organizations of all sizes to compete more intensely for revenue while taking an even harder look at reigning in costs. These concerns cascade through the overall project portfolio to IT and security in the form of two very basic questions: What do we need? What can we afford?

In a company fighting for its survival, talking to management about improvements in information security may seem as relevant as changing the locks on a burning building. Naturally, fire is an immediate threat to an asset and its contents, but over a longer time horizon so is the risk of theft … or foreclosure.

Bottom line, some organizations can afford bad security. Others can’t. In some situations, immediate survival concerns will temporarily trump long term protection goals. But as the market meltdown in the United States in 2008 is showing us, it is just as plausible to see that relaxing key control requirements for short term profitability puts entire companies, and even markets, at risk.

The only way to get this right is to view security in light of the survival needs of the firm, and measure it to the same standard of every other investment. In the past, information security hasn’t been held to this standard, mostly due to measurement challenges. Hopefully, for the good of the profession as well as the entities we protect, those days are over and we can take up the challenge of proving our value more accurately and more persuasively than we have in the past.

“What the CEO wants you to know”
In 2001 Ram Charan wrote a gem of a book called “What the CEO Wants You to Know,” distilling business acumen into the effective management of five core measures of business health: cash, margin, velocity, growth and customers. Charan: “Cash generation is the difference between all the cash that flows into the business and all the cash that flows out of the business in a given time period …it is a company’s oxygen supply” pp.30-31

Margin is the difference between the price and cost of goods sold, while velocity is the rate at which those goods are sold. Growth includes expansion (more sales) and extension (new markets) while the Customers category represents how well the organization responds and aligns with market demands.

Naturally, some of these needs can become tactical and immediate while others are more strategic in nature. But all must be functioning effectively for a company to succeed, and any threat to these measures ultimately threatens the health of the company.

“What the CISO wants you to know”
If the five factors above represent the keys to a successful business, then good security is important to a company only to the extent that it affects those factors. If there’s no impact on customers, growth, etc. then there’s no value to security. Or, as your CFO probably read in school:

“A potential project creates value for the firm’s shareholders if and only if the net present value of the incremental cash flows from the project is positive.” [Brigham and Ehrhardt, Financial Management: Theory and Practice, 11th Edition, p.389]

Security issues expressed in terms of cash, margin, velocity, growth and customers, and measured in terms of net impact to the company have the best chance of resonating with decision makers.

Gordon and Loeb propose a three dimensional Cybersecurity cost grid as a tool for building that business case. The authors suggest failures of confidentiality, integrity and availability are to be analyzed in terms of direct and indirect costs, as well as explicit and implicit costs.

For me, the distinction between indirect and implicit didn’t seem as compelling as the difference between a net positive or negative effect on security, so I started segmenting the effect of security across Charan’s five categories this way:



Of course, measuring it is the real trick. But there are quite a few resources available to help with that...