Don't Cry Wolf

I often find myself in a position that's doesn't fit the security guy stereotype, the position of devil's advocate. In an organization that's heavily populated by engineering types, it's fairly common that a spike in traffic or an unexpected outage will be met with cries of "hackers! break-ins! denial of service!" and such. In these cases, I'm very deliberate in making sure that we have the facts before I pull the security incident alarm.

As a security team, we only have so much goodwill to spend on security incidents. The first time we report one, the organization will pull out all the stops to make sure we have the resources we need to address it. As time goes on, the enthusiasm fades. The sure way to make sure we NEVER get any help on an incident is to start reporting false positives. Think about it... we report a denial-of-service, then eventually realize it was actually a misconfigured system causing the bad traffic. If this happens a couple of times, eventually the organization will treat it exactly like we treat a car alarm going off in the parking lot: "ignore it, eventually it'll shut off on its own."

Obviously, if you see data being destroyed or going out the door, feel free to yell as loud as you can and pull out all the stops. But if you're just not sure, grab a couple of people you trust to help, and get the facts first. Spend your resources wisely. When the real thing hits, you'll be glad you did.