Before you start reading, be warned: this is a *very* long post. It probably should have spanned multiple posts.
Last week I sat down with my CISO and had a chat about Google's incident response framework called Google Rapid Response, better known as "grr". I explained that I had installed it on a couple of virtual machines, started testing out the web-based interface and pulling files from remote systems, and that so far it was REALLY cool but that I was having some problems writing a blog post about it.
When he asked why I told him because every time I tried I ended up with two paragraphs covering general digital forensics and incident response (DFIR) and that I didn't think I could jump into a good grr post without spending some time talking about DFIR.
"Have you considered doing a non-technical post?"
The obvious answer was no, I had not. This blog started for me to document stuff I like to work on and so I have an easy-to-follow guide for potentially complicated installations or configurations. I hadn't really thought about covering a "soft topic" for others. I considered his question for about three seconds, tilted my head, grinned and thanked him for showing me how to jump into covering some DFIR tools.
It is simple, really. You can't discuss DFIR tools without first discussing some of the DFIR basics. Which tools do your DFIR team use? How do you create your DFIR team? Where do those tools and people fit into your DFIR process? And, what I consider the most important of all, why do DFIR in the first place?
Each of these questions could be a chapter in a book - if you're an actual forensicator or responder you're probably ready to argue that they could each be their own book, and I agree with you! I won't spend quite that much time answering them but I do want to use this post to address the question that drives all others - to DFIR or not to DFIR?
DFIR - Do We Or Don't We?
Let's say you are a freelance IT consultant and you have several small businesses as clients. Maybe you grew up and opted to stay in a small town so you do the IT work for friends that have become doctors, lawyers, dentists, dry-cleaners, fabricators, landscapers, plumbers, carpenters, publishers or any of the countless other types of small business owners out there. Even though there are significant differences in each of these businesses, they all have some things in common, right? At the very least:
- They all have customers of their own
- They all need to keep track of those customers
- They all bill their customers for services rendered
- They all buy business supplies
- They all have computers in their businesses (or they wouldn't be your client!)
- They all probably use the computers to track their customers and income/expenditures for taxes
- They all may lose business if their business records are leaked to the public
On most days it is business as usual - maybe a broken printer, someone bought a new smartphone and they want to sync their email, any of the myriad of small fires that need to be put out.
Then one afternoon you get a phone call from Bob, the guy who runs the dry-cleaning service. He explains that he was trying to open a video sent by an old classmate, Joe. You and Bob went to school together and you remember Joe, so this sounds pretty legit. Then Bob tells you that when he opened the video he had a warning that he needed to update his Flash player so he clicked "Accept". After that, he saw a little black window pop-up then disappear and now he has a pop-up saying his anti-virus found 300 "critical infected files".
How do you respond? Do you try to "clean up" the system and keep it in production? Do you give him a temporary system while you spend days analysing a forensically-sound copy of the hard drive to determine what malware was downloaded and what it accessed on the system? Do you have a network device in-line for his business that lets you analyse all of his network traffic for potential data exfiltration? Do you advise Bob to file a report with local or regional law enforcement?
Is that advice the same as it would be for Elizabeth, the lawyer, if she had the same thing happen? How does that advice change if the victim is the doctor?
If we're all being honest with ourselves then I think we can safely say that no, the advice will not be the same. The doctor will have HIPAA concerns and may have intimately detailed customer records accessible from each system in his office -- the Federal fines could put him out of business if he can't show protected data wasn't exposed. Bob may only have a first name and telephone number for each client - potentially unprotected information in his jurisdiction so he may not have to report a data breach even if his entire database were exposed.
Some clients will have different statutory and contractual requirements regarding investigation and reporting. In some situations it may be completely acceptable to rely on anti-virus or anti-malware software to "clean" the system. In some situations it may be acceptable to reformat and reinstall the operating system, what we call "nuke and pave." Other situations may require intense, in-depth analysis by trained forensicators spanning several days, weeks or even months. Larger organisations will have to take those same requirements into consideration when making the "do we or don't we" decision.
If You Do
Regardless of the scenario, you're almost certainly going to do something. That "something" will be determined by the leadership of each organisation and those decisions will need to be well-informed. It is easy to say, "yes, we're going to do forensics and/or incident response" but what if there is no funding for that decision? What if there is nobody at the organisation trained in those areas?
At the very least, the people making the decision will need to understand:
- Forensic hardware can be expensive
- Forensic software can be expensive
- This will be a long-term investment
- This is about protecting the organisation's resources, reputation and clients
- Results will not be immediate and can require days, weeks or months
- DFIR is an evolving field, training is never-ending...and can be expensive
Once the decision is made to commit to DFIR, the practitioners should also have some basic understandings:
- Forensic hardware will take training to use correctly
- Forensic software will take training to use correctly
- Both the hardware and software will require regular use to remain proficient
- Processes will have to be documented, validated and then followed - every single time
- DFIR is an evolving field, training is never-ending
Using DFIR To Enhance Security Operations
Summing It Up
Traditional InfoSec thought has been the trite castle and moat analogy, right? The barbarians are at the gates, screaming to be let in, but the bridge is up and at first they can't get across the moat. Then they build a bridge across the moat but that can't breach the walls. Then they push siege towers over the bridge to exit on top of the walls. Then they fight wall-by-wall until they get into the heart of the castle. We keep insisting firewalls (the moat) and VLANs (the walls) are enough to keep attackers out but we're delusional - the problem isn't the barbarians outside, it's the users inside.
What we really have to envision is a dark battlefield, dotted with foxholes. We know there is a perimeter and we try to protect against anyone coming in on the ground with firewalls and IDS. We try to monitor those parachuting in via authentication...but we're left with these thousands of foxholes all over the battlefield. Each foxhole may be occupied by one of ours, one of theirs, both or neither, and the only way we can know is by gathering intelligence. Without real-time intelligence from each host on the network we are essentially blind to that particular foxhole, meaning we don't know how that part of the battleground looks.