Data breaches cost companies $3.92 million on average, according to a 2019 IBM study, which helps explain why companies are so eager to hire security response team engineers.
According to CSO Online, incident response engineers work for companies to monitor for attacks and work on remediation when they are detected. As companies have become more aware of the negative consequences of vulnerabilities, demand for security incident response team (SIRT) engineers has grown.
What is SIRT Cybersecurity?
“Attacks are always there,” said Vikram Chabra, an incident response engineer at NetEnrich.
NetEnrich is a company headquartered in San Jose that offers remote managed security services to customers. Chabra, who has worked as an incident responder for three years, said that incident response engineers usually start their careers in security operations centers (SOC). While SOC team members don’t necessarily have programming skills, they often collaborate with engineering teams.
“An incident response team does a lot of things,” Chabra said. “The first one is immediate detection, the second is to contain, eradicate, recover and minimize the damage. The third is to do deeper analysis, to find out indicators of compromise using memory dumps and hard drive dumps.”
Being an incident response engineer can be hard work. Chabra said part of what makes the position challenging is that there are so many different ways a company can be attacked. Attacks are actually happening all the time — if companies don’t have a plan in place and know which vulnerabilities to devote time and effort toward, it can quickly get overwhelming.
“It depends on your lens of detection,” Chabra said. “You’re kind of observing every cough and sneeze attack on your enterprise systems and your web applications, which might happen on a daily basis. ... There are infinite possibilities of attacks coming in, any kind of attack vehicle.”
Because there are so many attacks, and because attacks vary greatly in terms of technique and severity, the question of how frequent attacks on companies are can be a little complicated. The reality is that a company can only count the attacks it is able to detect, and how many attacks are detected depends on how wide you’re able and willing to cast your net.
Chabra turned the question of frequency around on its head.
“What determines the frequency of attacks is how deep your detection lenses are, how effective your systems are, and how much time you spend doing all this,” he said. The harder you look, the more attacks you’re likely to see.
Different types of attacks also operate on different timelines. Some can find their way into a system and lay dormant, gathering additional information or waiting to be deployed.
“Sometimes we detect it after three months or six months,” Chabra said. “An attack happens and goes through the different stages, and the last stage is where exfiltration actually happens — when your data is out, or your files are corrupted and the hacker is asking you for ransom.”
So How, Exactly, Does Attack Detection Work?
SIRT engineers are looking for “any breach, anything that is suspicious, anything that is bad for an organization,” Chabra said. Incident response engineers keep a list of known attacks that they check against — attacks such as distributed denial of service, ransomware, malware and phishing — but in terms of how they detect attacks, it isn’t so much looking for those specific attacks as looking at what parts of an organization are vulnerable to attacks.
“It depends upon various things that come into the picture whenever we think about a cyber security incident,” Chabra said. “The first thing is the attack vector, meaning what the attacker used as a payload to get into the organization.”
“Do we have the right controls in place for the timely detection of these attacks?”
Once the team detects evidence of an attack, the next step is figuring out how far along the attack is. This is known in cybersecurity as determining where the attack is on a “kill chain.” Not only do SIRT engineers detect and mitigate attacks, they also use the knowledge from learning about a new attack to set up procedures for detecting similar ones in the future.
“The incident response team is responsible for determining where we are in that attack state,” Chabra said. “How bad is the attack, how far has the attacker been able to access? And then the final thing is, when did this attack start, and how is it architected? And do we have the right controls in place for the timely detection of these attacks?”
The SIRT Engineer Toolkit
The teams use a variety of tools to detect, mediate and monitor attacks.
“The primary source is the security information and event management (SIEM) tool,” Chabra said. “The SIEM tool typically collects all the live events and logs from testing everything that an enterprise owns, to make sure that all that is functional.”
Chabra said these tools also receive threat feeds, which are lists of compiled malicious IP addresses and URLs. These threat feeds are built based on known indicators of compromise, pointing incident responders to malicious intrusions or attempts at intrusions.
“Nowadays, threat feeds have become very smart,” Chabra said. “I can actually customize a threat feed and tell it to reach for this geographic location or for this industry vertical. Marrying the information from SIEM, which are your logs, with integrated threat feeds, is where the magic begins.”
The SIEM tool alerts incident response teams when attacks are detected, which kickstarts the rest of the SIRT process. There are also methods used for understanding how attacks work in more detail.
“Because you don’t know when cyber attacks can potentially happen, the best thing you could do is to have the right defenses in place.”
One is known as “honeypotting,” which, similar to its spy cousin, involves setting up a server with false data to attract hackers. Incident response team members can use the honeypot to monitor attackers and understand how they advance through a system, so that team members can better understand how to defend the real system.
Chabra said that, because the job relies on using many different strategies and tools to detect attacks, successful incident responders spend time learning about new detection strategies and adding tools to their toolkits.
“A lot of time that an incident responder spends is trying to sharpen their tools,” he said. “Because you don’t know when cyber attacks can potentially happen, the best thing you could do is to have the right defenses in place.”
Chabra recommended that incident responders spend a few hours a day learning about attacks happening across the world, to “see if there’s any learning that can be taken out of it to enhance our defenses, write more rules for timely detection, and have some preventative steps implemented.”
He said that staying up to date with the world of cybersecurity is important for SIRT engineers because of how dynamic the industry is.
“We wake up with a lot of new attacks every day, and things keep changing,” Chabra said. He said the qualities that make a good SIRT engineer are “a combination of how quickly you can learn something new, out-of-the-box thinking, and how detailed you can go in terms of your investigative capabilities.”
The Pros and Cons of Becoming a SIRT Engineer
Being an incident response engineer is challenging, demanding work, and it is often quite stressful. Because it’s the job of a SIRT engineer to protect enterprises from attacks, the success of a company can be on the line.
“Startups have been closed by experiencing a single cyber attack,” Chabra said.
He said the job can involve long hours and late-night calls, and that even small mistakes can have large repercussions.
“Things can really go wrong,” Chabra said. “If you forgot to just take a back-up, that can potentially completely void the chain of custody, and you would not be able to claim cyber insurance.”
“You might have seen those movies like Die Hard.”
It can be a somewhat thankless job, where “people will not remember a lot of good things that have been done,” but the team will get a lot of attention when something goes wrong, he said.
But for the right person, the job can also be very exciting.
“You might have seen those movies like Die Hard,” Chabra said, describing how it felt at work sometimes. “You can believe that you’re actually helping, you know, rescuing the world and all that. It’s a great feeling to steer an organization out of a cyber attack with minimal damage and defeat an attacker.”
It’s also a growing field with many job opportunities.
“The cybersecurity industry is giving a lot of importance to incident responders,” Chabra said. “There will be a lot of technologies that will be built to facilitate incident response teams. I believe that this is going to be a very important and high-skilled job, going forward.”
