Penetration Testing: What It Is, and How to Do It Well

During Ed Skoudis’ first stint as a penetration tester for a phone company in the early nineties, his colleague turned to him with some “prophetic” career advice.

“My officemate said to me, ‘Look, kid, you’re probably only going to get 10 years out of this cybersecurity career, because we know how to fix all of these vulnerabilities, and people are going to fix them,’” Skoudis said.

The internet was in its infancy back then, and hackers had concentrated their efforts on breaking into phone systems and rerouting phone calls. Skoudis, who had grown up hacking bulletin boards on his old Commodore 64 home console and earned a master’s degree in information networking from Carnegie Mellon University, was hired to figure out how the bad guys were breaking into the system.

While his colleague was right that the cybersecurity team would eventually figure out how to patch the vulnerabilities the hackers exploited to break into phone systems, he overlooked the same thing companies today overlook: As technology grows exponentially, so does the amount of security vulnerabilities.

“We are deploying new vulnerabilities faster than we’re deploying fixes for the ones we already know about.”

Skoudis now works as a fellow at the Sans Institute, where he teaches advanced penetration testing techniques. The internet, smartphones, third-party software, IoT devices, the cloud: All create a web of access points that hackers can use to exploit people and businesses if they aren’t properly secured. Today, even a doorbell can be an entryway into a network if it’s part of a smart system.

As companies struggle to keep up with hackers and technology grows more interconnected, the role of the penetration tester has never been more necessary. “We are deploying new vulnerabilities faster than we’re deploying fixes for the ones we already know about,” Skoudis said.

What Is Penetration Testing?

The concept of penetration testing started in the 1960s when computer science experts warned the government that its computer communication lines weren’t as secure as it had assumed.

To test this theory, the government brought in groups of computer scientists called “Tiger Teams” to try and break into its computer network, according to the InfoSec Institute. The computer network failed the tests, but it did prove the value of penetration testing.

Since then, penetration testing has been used by the government and businesses alike to analyze the security of its technology. At the core, a penetration tester’s job is to act like a hacker and exploit vulnerabilities in a company’s system. It exists under the umbrella of ethical hacking, and is considered a service within the role of white hat hacking.

“The only difference between us and another hacker is that I have a piece of paper from you and a check saying, ‘Go to it.’”

While vulnerability scans can identify surface-level issues, and red hat hackers test the defensive capabilities of blue hat security teams, penetration testers attempt to go undetected as they break into a company’s system. Their goal is to expose and exploit the depths of a company’s weaknesses so that the business can understand its security risks and the business impact, said Joe Neumann, who is the director at the cybersecurity firm Coalfire.

“It’s very common for us to gain a foothold in a network and laterally spread across the network to find other vulnerabilities because of that initial exploitation,” Neumann said. “The only difference between us and another hacker is that I have a piece of paper from you and a check saying, ‘Go to it.’”

Why Is Penetration Testing Important?

While penetration testing has been around for nearly six decades, the practice has only started to grow in popularity among commercial businesses within the past five years, Neumann said. He receives calls every day from companies asking him to perform their first penetration test, and, often, they fail miserably.

Still, after a few years of conducting penetration tests in the private sector, Neumann expected to see the number of new security issues to flatten out. Instead, every test brings up a new batch of vulnerabilities as tech becomes increasingly interconnected.

“There’s just more and more stuff that comes out,” Neumann said. “We’re not getting more secure, and I think now we’re realizing how bad that actually is.”

One of the most common culprits comes from “legacy debt,” or flaws inherited from tech a company acquired, Neumann said. But the rising number of threats is also reflective of the industry’s attitude toward cybersecurity and penetration tests in general.

“We’re not getting more secure, and I think now we’re realizing how bad that actually is.”

Sometimes companies skip testing a product for security flaws to hit the market sooner. Other times, employees cut corners and don’t apply proper security measures, Skoudis said. Security features are still considered a luxury, especially for small-to-midsize businesses with limited financial resources to commit to security measures.

Of course, as cars and homes become more interconnected, this can have dangerous consequences. Two hardware penetration testers showed how easy it is to hack into an internet-connected Jeep and take over the car’s network, in a story for Wired. Over the course of the penetration test, the test-driver found himself helpless as the hackers took over the speakers, the brakes and even the engine.

Despite the risks, most companies wait until they’ve been hacked to reach out for a penetration test, Neumann said. Instead, it’s helpful to think of a penetration test like a preventative visit to the dentist: It can probe the network for soft spots and identify holes in the security network, but it also reinforces a stronger security network as a whole.

Ultimately, the results of a penetration test can only show the scope of a security risk and its business impact. Much like the dentist, the impact will only go as far as the security steps clients are willing to take once it’s over.

“One thing I try to stress to customers is that all the security prep work and diligence they did before the penetration test needs to be done year-round,” Neumann said. “It’s not just a surge thing to be done before a test.”

More on CybersecurityEthical Hacking: Inside the World of White Hat Hacking

201: Strategies for Executing a Pen Test

People like to think what Skoudis does is magic. They imagine a hooded hacker, cracking his knuckles and typing furiously to expose the guts of a company’s network. In reality, Skoudis said the process goes something like this:

“You walk up to a wall, and you start beating your head against the wall. You’re trying to break the wall with your head, and your head isn’t working out, so you try everything you can think of. You scrape at the wall and scratch at the wall, and you spend a couple of days talking to colleagues. Then, finally, you find this little crack in the wall, and you start digging, but it goes nowhere. Several days later, you look over your shoulder and you notice that there’s a little piece of the wall you haven’t seen before and there’s a nick in it. So you reach your finger out and you touch it, and the wall falls over.”

Still, there are a few strategies testers can deploy to break into a network. Before any pen test, it’s important to get a few upfront logistics out of the way. Skoudis likes to sit down with the customer and start an open dialogue about security. His questions include:

• What is your dependence on computers, and how can someone hurt you through them?
• What are you worried about? What keeps you up at night?
• What is it you’re trying to find out about how a hack can hurt you?

This helps him understand the scope of the test they’re looking for. From there, he warns the customer that there is a risk that he will crash their system and that they need to be prepared for that.

“If a pen tester ever tells you there’s no chance they’re going to crash your servers, either they’re outright lying to you — because there’s always a chance — or they’re not planning on doing a pen test,” Skoudis said.

From there, a penetration tester will run a vulnerability scan and make a list of weaknesses to exploit. Skoudis looks at domain names, server records, DNS records, employee emails and third-party software to see the attack surface. Then, he starts digging.

“If a pen tester ever tells you there’s no chance they’re going to crash your servers, either they’re outright lying to you — because there’s always a chance — or they’re not planning on doing a pen test.”

His strategies run the gamut of tricks that a hacker might use. He might send a phishing email and see if an employee will bite, post JavaScript into an HTTP request to access another user’s browser or enter garbage data into various input fields.

“What you’re trying to do is to get the network to cough or hiccup, which might cause an outright crash,” Skoudis said.

Some of the most common issues that pop up are default factory credentials and default password configurations. A tester’s goal is to exploit that low-hanging fruit and then dig deeper into the list to find medium risks that could pose a greater danger to the company, like server messaging box signing, Neumann said.

Throughout the test, it’s important to take detailed notes about the process to help explain the errors and provide a log in case anything went wrong, said Lauren Provost, who is an assistant professor in computer science at Simmons University. It’s up to the tester to provide a post-test summary and convince the company to implement some security changes. When she goes over her reports with a customer, she’ll often guide them into other findings that she discovered outside of the scope they requested and offer resources to fix it.

“The job is to meet the customer’s needs, but you can also gently support education while you’re doing that,” Provost said. “You’re being a resource. You can say, ‘This is what I’ve been doing, but I also noticed this issue over here that you should think about.’ I also like to offer employee education while I’m there.”

Read Case Studies and Think Like a Hacker to Stay Informed

The only way to get ahead as a penetration tester is to think like a hacker. Provost’s expertise is in cybersecurity, and she spends a lot of time in her classes going over case studies of malicious hacks with her students.

Since every penetration test reveals new flaws, it can be difficult to know what to prioritize. The studies can help them identify the patterns and methods malicious actors use. Often, a hacker repeats the same strategies and behaviors from one case to the next.

“A lot of the motivation is the same: financial gain or notoriety,” Provost said. “Understanding the past helps guide us in the future.”

Meanwhile, cybersecurity conferences, news articles and scouring the dark web can also help testers stay up-to-date on the strategies hackers are using. The same is true with late-night tinkering with the latest IoT devices to pinpoint common security weaknesses. Training as a penetration tester never ends, Neumann said.

“A lot of the motivation is the same: financial gain or notoriety. Understanding the past helps guide us in the future.”

Although it’s impossible to be completely informed and up-to-date with the latest trends, there is one security risk that seems to transcend all others: humans. A malicious actor can call an employee pretending to be HR to get them to spill a password. People click on phishing emails, company leaders ask IT to hold off on adding restrictions to the firewall to keep employees happy, and engineers overlook security configurations because they take the security practices of third-party vendors for granted.

Each of these blunders are entry points that can be prevented. So when Provost models penetration tests, she’s thinking about not just how someone will break into a network but also the mistakes people make to facilitate that. “Employees are unintentionally the biggest vulnerability of most companies,” she said.

To fix it, companies must invest in training their employees and make cybersecurity a priority. The best penetration tests help to identify those weak points and give companies the materials they need to start patching their entire cyber ecosystem, from third-party software to internal firewalls to training exercises.

The Future of Penetration Testing

Neumann doesn’t believe security teams will ever catch up to the exploits of hackers. It’s a Sisyphean struggle that has grown more complex with every advancement in technology.

The challenge doubles when companies release consumer IoT devices without the proper security configurations. In an ideal world, security should be easy enough that anyone who buys the device can simply turn it on and operate it carefree. Instead, products ship with security holes, and both companies and customers pay the price.

“I don’t think we’ll ever get to the point where the defender has everything secure because of the sheer volume.”

“I don’t think we’ll ever get to the point where the defender has everything secure because of the sheer volume,” Neumann said. “There will always be that chink in the armor that you’re able to get through. That’s what a pen tester does: try to find that one spot and gets in.”

There is some hope. Larger companies like Amazon, Microsoft, IBM, Oracle and Google all require companies to do penetration tests before they can sell them services, Neumann said. Safeguards like those are changing the culture around cybersecurity and leading others to embrace penetration testing as a preventative measure.

Skoudis, meanwhile, is entering year 25 of his supposed “10-year-career,” and it shows no signs of letting up.

More on CybersecurityHere’s What Happens After a Company Gets Hacked