Defining Success for Information Security Through KPIs

Monday, March 26, 2012

Rafal Los


I've been a measure it kind of guy since shortly after taking a role at General Electric almost a decade ago.  In a culture where everything is scorecards and metrics I quickly learned that proof requires actual evidence. 

The problem with evidence is that it has to be in the correct context, otherwise what it means to you doesn't necessarily translate to the person you're presenting that evidence to. 

This presented a problem for me many times over the years as I've struggled to build and launch successful software security assurance programs which could, over time, demonstrate their value and success to the business they service.

Imagine that, IT that serves a business purpose and can prove its value.  I know plenty of people who have been jaded over the years and consider this a fool's errand, and will readily refer to "rainbows & unicorns" but I challenge you to have a compelling conversation with a senior IT leader (likely your CIO) without solid evidence of your successes or failures. 

First and foremost I acknowledge that there must be a solid relationship between IT and the business it serves - that cannot be argued - but when you look across all the various IT shops out there we here at HP feel there has to be some way of having a consistent conversation about success (or lack thereof).

To that effect, we've started a community called "Discover Performance" dedicated to this effort.  The team which I'm a part of strongly believes you can't manage up based on 'gut feeling'.  We also acknowledge that while managing "down" is absolutely critical - being able to demonstrate clearly, concisely and positively your impact to the business leadership (or managing "up") is just as critical if not more so.

Some of you may have already read and remember the whitepaper I wrote a while back called "Tracking Performance of Software Security Assurance - 5 Essential KPIs".  This was the start of my effort to enumerate a set of key performance indicators associated with software security assurance, and distil down to a reasonable number the things that come out of the class of "IT" and "business" requirements. 

Let's face it, in the world of software development the business just wants to release fast and functional while the security team would prefer slower and more 'secure'.  These can be at odds with themselves, so as security struggles to positively impact risk and business, I found 5 key performance indicators that stood out as a bridge between the two lands.

It's time to move on from just looking at software security, and as you're probably already noticed over the last 6 months or so, I've taken a step back from just software security into holistic 'security and risk'.  I won't claim to be the guru on either, especially risk where I still have much to learn, but my current project is called "Zosimos"... after the alchemist that back in the 3rd Century AD. attempted to turn lead into gold.  That's essentially the point here - let's turn all these massive piles of metrics (lead) into KPIs (gold).

In Information Security you've probably got no less than 10 dashboards telling you all sorts of critical things about your security posture.  You also already know that those dashboards are often not viable for consumption at the CIO (or higher) level. 

There are already several projects here within our organization that have fantastic IT-level performance dashboards with stellar KPIs (all based on your unique perspective)... but my focus is to refine the security piece of that puzzle. 

So here we go:

  • Objective - Gather, Analyze, and Distill as much information as possible on how information security is reported to senior leadership in IT (presumably the CIO or equivalent), and then simplify to create a set of cross-industry KPIs.
  • Perspective - I'm looking at the CISO <> CIO relationship and addressing how these two roles with very different perspectives and goals communicate, effectively
  • Technical details - What I'm looking to create is a 3-tier view (CIO, CISO, technical analyst) as a roll-up of metrics to KPIs.  Think of it this way- you have firewalls, IPSes, code scanners, project data, DLP and all manner of security and not-directly-security 'stuff' which generates data.  At the practitioner level you likely care about how many critical attacks are generated, but that doesn't mean much to the CIO. 

What we will be doing is taking an approach that gathers all of the data points a security practitioner/analyst cares about, aggregates into KPIs a CISO cares about, then extracts meaningful "is security successful" KPIs that a CIO can look at.  Incorporating the ability to drill down or aggregate up through these three levels, etc ... that's going to be key.

Now - I can't do this without you.  If you have ideas, opinions, suggestions you'll be asked to help contribute to this effort.  I may pick your brain, ask you follow-up questions, etc... remember this is all in a community effort to help you perform better.  It's all about the performance of IT (and security as a component thereof).

The end result will hopefully be at least a whitepaper, some presentations on the topic, and maybe even a tool so that we can all perform better in Information Security.

Cross-posted from Following the White Rabbit

Possibly Related Articles:
Information Security
Risk Management Application Security Secure Coding Analytics metrics Software Security Assurance Information Security key performance indicators KPI
Post Rating I Like this!
Michael Thibodeaux Hello Rafal,

I am currently tasked with developing KPI's and it would be nice to learn what you are doing for IT systems supporting Automation or ICS.

The views expressed in this post are the opinions of the Infosec Island member that posted this content. Infosec Island is not responsible for the content or messaging of this post.

Unauthorized reproduction of this article (in part or in whole) is prohibited without the express written permission of Infosec Island and the Infosec Island member that posted this content--this includes using our RSS feed for any purpose other than personal use.