Breached! Now What? Seven Steps to Avoid Failure Panic

Monday, May 07, 2012

Rafal Los


It is widely acknowledged that a security incident, whether it ends up being a massive breach or not, will happen to virtually every organization that has an Internet presence.

Even those without a presence on the Internet can still expect to have some sort of issue involving security simply because there is almost no such thing as a disconnected organization - I mean completely disconnected - anymore.  

Once you get past the notion that it's not a question of if but rather when a breach or incident occurs - another issue starts to hit you.  What does a breach mean for your security program?

To many organizations, a security breach means a catastrophic failure in security.  A breach signifies a breakdown in the protective mechanisms installed to keep the organization secure, and by its very nature represents failure. The problem with this situation is it represents two failures.  

The first failure in a situation like this is in planning and organizational strategy, more on this in a moment.  The second failure this represents is in the organization's face to customers, investors, and the public.  Failures of this nature are difficult to recover from in both perception and reality - but it doesn't have to be this way.

Let's take an example from morning traffic radio.  If before you leave your house for that big meeting in the morning you hear something like "The inbound I-90 lanes into Chicago are wide open, it's going to be 30 minutes from the airport into downtown" and you get on the highway only to find yourself in an unexpected backup you curse the traffic announcer even though it's rarely their fault traffic backed up.

However if you hear "The inbound I-90 lanes into Chicago look clear for now, with travel times currently at about 30 minutes, but allow yourself for extra time because you can never predict Chicago traffic" and you get into that same traffic jam and miss your meeting you blame yourself rather than the traffic person... right?  

The difference is one of those announcements told you that although things look good now you can't predict the future.  This is an important lesson for information security professionals.

As I mentioned above, the primary failure in a situation where an organization is breached and the resulting event is seen as a catastrophic failure is in strategy and planning.  Failing to account for a breach in your strategic information security and risk management planning is a failure all in itself.

It's at very least arrogant to think you can ever reach some mythical state absolute security in your organization, and in many situations it's irresponsible.  How many CISOs are out there telling their leadership they need more budget to "secure the company" only to completely leave out the "we will never actually reach a level of complete assurance" that should accompany the request?  

The tragedy is that if leadership doesn't understand risk management well enough on the technical end of things to know better, they're telling the board, or even their customers that your security is Fort Knox.  When Fort Knox is breached and the keep is pillaged... then what?  At that point you've completely and utterly failed and there is no coming back from that.

While many organizations that don't plan to fail (experience a breach) recover just fine in the market and with their customers, often times the sacrificial lambs wouldn't agree.  Many CISOs and security staff lose their jobs as a result of a breach because their organizations are expecting security and when that security fails, the employee or program failed and people must be held accountable.

Speaking of accountability, think about how you as a customer feel when someone promises you something and then comes back to tell you that they screwed up, and they're sorry.  Your reaction is typically to get angry, point to previous statements about guarantees and say that there was never any mention of 'oops' planning and to demand accountability.  

Rightfully so.  If your vendor would tell you that they do their best-effort to maintain the privacy of your data but have a strategy which includes failure response you'd at least understand and not expect perfection.  Am I right?

Let's relate this to what we all do, and specifically what I'm focused on which is the security of things like cloud environments.  Many of my customers are providers of cloud services themselves... whether that's to an internal customer through a shared service model or external customer in the case of a small-market cloud provider.  When they promise security to their customers it's important for me to help them work through what that actually means.

Here are the 7 steps I work people through to make sure they're not only strategically prepared, but also past the panic stage when bad things happen...

  1. There are no absolutes - what you're providing is a level of risk reduction to an agreeable, acceptable level for the task
  2. Define tolerances to failure- make sure you've clearly defined your tolerances for everything from downtime, to security incidents to understand clearly when a failure happens
  3. Strategies embrace failure - a good security & risk strategy embraces failure as one of the realistic outcomes, and plans for it, providing a pre-defined next-step in those failure cases
  4. Define failure modes - incidents and breaches happen, but they're not all created equal - make sure you have this pre-defined before you find yourself in the middle of a failure mode asking "how do we explain how bad it is?"
  5. Define a recovery - when planning for a potential failure scenario define clearly what your steps are to recover from the various failure modes you've already defined, and execute
  6. Test recovery methods - there is nothing worse than having a well-defined plan for recovery that you find out is non-workable in the heat of the moment - test, test, test and validate your strategic recovery methods
  7. Validate sentiment - make sure that what you consider acceptable and reasonable, and a sound strategy which includes failure is acceptable by those that matter to you, namely customers, investors and the public

You've heard the phrase "Failing to plan is planning to fail" right?  Let's modify it slightly to say "Failing to plan for failure, is planning to fail catastrophically".

Make sure your security strategy includes at least one (or more realistically severalfailure scenarios.

Cross-posted from Following the White Rabbit

Possibly Related Articles:
Information Security
breaches Enterprise Security Risk Management Security Strategies Best Practices Incident Response Vendor Management Leadership Policies and Procedures
Post Rating I Like this!
Marc Quibell Sounds like you're describing an "Incident Response" program. The GIAC Certified Incident Handler program provides some framework for this as well, such as the steps in the Incident Handling process (Preparation, Identification, Containment, Eradication, Recovery)
The views expressed in this post are the opinions of the Infosec Island member that posted this content. Infosec Island is not responsible for the content or messaging of this post.

Unauthorized reproduction of this article (in part or in whole) is prohibited without the express written permission of Infosec Island and the Infosec Island member that posted this content--this includes using our RSS feed for any purpose other than personal use.