For some people, QA refers to manual testing — clicking through a program’s user interface, trying to find bugs in unlikely places. For others, QA refers to a highly technical position that requires extensive knowledge of programming and the use of specialized tools to test applications. Neither definition is wrong, exactly: Quality assurance is constantly evolving.
This can be confusing even for QA workers and the companies that hire them.
Jeff Sing, a QA manager at Optimizely, which creates tools for supporting feature flag deployments and A/B testing, said companies sometimes hire QA testers and expect them to fix problems, even though the company doesn’t understand the nature of the problems.
“Things aren’t working, customers are unhappy — I don’t know why, and I need to find out,” Sing said. “Well, my developer said he tested everything, but it’s not working. ... So we hire a QA engineer.”
When companies hire QA engineers, Sing said, they narrow down the pool of candidates by specifying programming language skills. For instance, a company that creates applications in Java will look for QA testers with experience in Java. But hiring QA testers without first understanding the underlying problems may not be the solution — if a company is having problems with user experience, a QA tester running integration tests can only do so much.
“If you have the [programming] background, maybe they’ll hire you, but that doesn’t mean you’re uniquely ready to solve the problem,” Sing said.
The Evolution of the QA Role
QA is a fundamental part of software development, but the two disciplines have had a long-standing on-again, off-again relationship. Software development embodies efficiency through automation, while QA for a long time meant manual testing that is difficult to automate.
“Back in the days, QA really was a business analyst role,” Sing said. “There’s a wall: developers build, QA manually checks — like a waterfall model. As you see the industry mature into more agile roles, [it reached] the point where we decided we don’t really need testing — we can offshore to China or India. And then that completely blew up.... And then the whole concept of ‘shift left’ came up.”
“Traditionally, developers should be at the bottom. They test everything there, and QA should be testing everything on top.”
Software testing used to occur mainly at the end of the software development cycle, but “shift left” refers to the idea that moving it toward the beginning is more cost effective and better. Bugs noticed by customers could result in loss of business, not to mention the extra time and money needed to develop a fix and get it to the customer. Bugs caught at an earlier stage of the development cycle by developers can be fixed immediately, often within the same development sprint.
Shift left is the current trend of the industry. Instead of being the specific domain of QA, responsibility for testing has increasingly moved to developers, who use automation and metrics to help them test. Sing described the different types of testing involved in a given project as the testing pyramid. At the bottom of the pyramid are unit tests, followed by integration tests, then end-to-end tests and user interface testing, and finally manual tests at the top.
“It’s like the food pyramid,” Sing said. “Traditionally, developers should be at the bottom. They test everything there, and QA should be testing everything on top. And the amount of tests obviously is less on top ... but it should be more impactful.”
Automation Is a Large Part of QA
Although integration, end-to-end and manual testing are important QA functions, Sing said those skills alone are not enough to stay competitive as a QA engineer.
“To be successful in this modern era, you need to be evolving,” he said. “If you’re only here to test manually, your food’s going to get eaten because of companies like Rainforest and Applause.”
Rainforest and Applause are software testing companies that use crowdsourcing to perform manual and automated testing on a company’s behalf. Tim Harrison, software QA architect at SQA2, a software testing consultancy, said QA engineers should develop skills such as automated testing process improvement, which focuses on improving a company’s overall testing process.
“To be successful in this modern era, you need to be evolving.”
“In terms of long-term value to the organization and skill sets that you’re growing, that manual testing is only going to get you so far,” Harrison said. “We look at manual testing as a transactional type of thing. You do it, and it’s only as valuable as the test is in that one execution — you’re proving that something works. But when you build automation, you are building something that is a valuable asset to the organization that continues to provide value, especially if the company is running that automation on a nightly basis for every release — that automation is a value add.”
Despite the current trend of developers taking more responsibility for QA, Harrison argues that it’s costly for developers to take over the testing automation process.
“It doesn’t make any sense for a company to pay the big bucks for a developer, and then not have them produce revenue-generating products or services for their company,” he said. “Automation, at the end of the day, is throwaway code. If something changes, half of it needs to be deleted and rewritten to match the new features. If you’re spending the big bucks on having a developer do that, the work that they build is now worthless and deleted.”
Modern QA Involves Requirements Quality
QA engineers can also help product managers and developers understand one another, being both technical and accustomed to taking customer expectations into consideration, Sing said.
“Product is so in line with what customers want, in their mind they have this vision of what it should be, but they’re kind of far away from technical,” he said. “Quality is the bridge. We understand what our customers want, because we see customer complaints, we talk to customers and we know they’re unhappy. We also know the technical side because we’re in the code.”
A very left-shifted QA process that takes advantage of both technical skills and an understanding of customer needs is known as “requirements quality” — checking that a story’s feature requests and acceptance criteria makes sense before any code is even written.
“Requirements quality is very important. It actually works to prevent defects down the road, rather than just detecting defects,” Harrison said. “Making sure that not just the product team understands it, but [QA] has to understand it, and the developers understand it as well.”
“Quality is the bridge.”
One of the strategies QA uses to check requirements quality is to identify test cases immediately after receiving requirements from the product manager, Harrison said. QA then reviews the test cases with the product manager and the developers — a process that often reveals gaps in each role’s understanding of the requirements and kicks off the necessary conversations to bring everyone on the same page.
“When that doesn’t happen, a developer could build something the product owners requested, and it’s completely not what the product owner expected out of it,” Harrison said. “You can only convey so much detail in a requirements document — especially with agile trying to be more lean, product owners are pushed to document things less than the olden days of waterfall when you had these huge documents.”
Strong Technical Skills Are Important for QA
Being able to bridge the gap between the product side and the technical side is a great asset, but QA is vulnerable to being squeezed out between developers who are getting more responsibility over testing and product managers who are the true owners of requirements. For Harrison, the only protection is to be as technical as possible.
“I believe the solution is QA people need to understand technology,” Harrison said. “If I needed to, I could jump in and contribute just as well as any of the developers on my team. That’s where any QA team member should be. [Companies] need a technical QA person that understands the technologies they’re working with, not some keyboard puncher that just tests things all day.”
He recommends staying on top of tech trends by trying out new tech stacks and frameworks outside of work hours, and getting familiar with testing automation.
“There’s definitely value in building a dummy website in the technology stack your company is working in,” Harrison said. “It is very valuable because you can maybe come up with test cases that the typical QA person wouldn’t come up with. Creating automation against the application under test [is also useful]. Not only does that help you learn, that’s also creating long-term value for your company.”
Emotional Intelligence Contributes to the Success of QA
It seems like QA engineers can get stretched pretty thin, between their work responsibilities and the technical skills they develop outside of work hours. QA testers must learn specific testing tools that developers may not need to know, and they may need to learn entirely different tools if they move to a different company. What’s more, support for QA professional development is not nearly as robust as for developers — there are only a few large QA communities, like Software Test Professionals Conference, STARWEST and Ministry of Testing.
Sing’s advice was a little different. He acknowledges that it is necessary to have a strong technical foundation and to understand software architecture. But that doesn’t mean QA pros have to be as technical as developers — there are other important skills to look for in a QA engineer.
“That’s your bread and butter,” Sing said about understanding tech stacks and architectural principles. “You need to be able to understand how to test the stack that you’re on. You need to be up to date. You don’t need to be a super engineer, like, ‘I can write my own microservices, I can write my own API stack.’ That’s not necessary.”
Sing said it’s important for QA engineers to think of their roles as ensuring the complete quality of the company’s product, not limited to the scope of testing.
“You can’t describe quality as testing,” he said. “If that’s the case, then we can outsource testing. Quality means everything from ‘Did we build the right product?,’ to testing, to ‘Did we launch the product correctly?,’ to ‘How do we monitor the product?’ That is the entire scope of the product life cycle that you need to maintain and understand.”
“The most brilliant engineers say things like: ‘How do I communicate certain things? What am I not thinking about?’”
Understanding the whole life cycle of the product and knowing where gaps in quality are is the true role of QA, Sing said.
“If I understand this is the problem we have, I can address it. And that in itself is better than running 500 tests,” he said. “If we build the right product, we build correctly, we have our processes, and our testing is good, and we move fast to deploy, and we have been monitoring — all of these things combined mean we have a beautiful product and our customers are going to be really happy.”
In order to advocate for quality holistically, Sing said QA engineers need to have a well-rounded understanding of the business, which extends beyond technical skills. It also takes emotional intelligence, or EQ, to be a QA engineer.
“The second part is to have some more EQ,” Sing said. “The most brilliant engineers say things like: ‘How do I communicate certain things? What am I not thinking about?’”
He encouraged people who want to get involved in tech but don’t see themselves as stereotypical zeros-and-ones people to consider QA as a career choice.
“We need people who have that kind of mindset, that personality, to come into this role,” Sing said. “A lot of people are like, ‘If I don’t become a developer, then there’s no other goal for me.’ QA could be very rewarding if you understand [EQ] and you can deliver this value proposition. Companies need you, they want you. To have the EQ to understand these problems, and understand that not everything is a one or zero, and how to drive forward, you’ll deliver a lot of ROI to your company.”
