Software developers and software testers have a symbiotic relationship — good code needs to be thoroughly tested, and testing can’t happen without any code.
In practice, however, the two roles have a way of stepping on each other’s toes. Developers tend to have more prestige at companies than testers, who sometimes have trouble getting developers to take their suggestions seriously, or find their concerns pushed to the bottom of projects’ priorities.
But testers say the two roles can productively coexist. The work of testing can look quite different from what developers do, especially in the field of user interface (UI) testing, and the two sides can learn a great deal from each other. Here’s the advice UI testers would like to give their developer colleagues.
ADVICE FROM UI TESTERS
- Thoroughly unit test before handing code to QA. Each higher level on the testing pyramid depends on the robustness of the layers underneath.
- Give more context. Document what you’ve already tested, and what types of problems you expect to see.
- Be careful of asynchronicity. Stress-test asynchronous code to reveal any odd behaviors.
- Share technical knowledge. Testers can benefit from developers’ technical knowledge, and developers can learn from a tester's perspective.
Thoroughly Unit Test Before Handing Code to QA
Because UI testing is focused on the user interface, it can look quite different from other types of testing. In addition to worrying about bugs, UI testers consider user experience factors as well, such as whether the product is intuitive for end users.
John Hill, a UI test engineer at KBR, an aerospace and defense company, said developers can use UI testers as a proxy for the product’s general user base.
“A UI tester is also kind of a user of a system,” Hill said. “You could use them to get some great advice and feedback — what you would likely receive from your end users, who you may have no way of reaching a lot of the time.”
That’s because UI testing mostly consists of broad, system-level tests that are run near the end of a project or working unit of code. UI tests can be automated or manual, and are used to verify that all the code is working together correctly and the program is running as expected.
“If anything foundational is wrong, you have a huge stack [that’s being tested] to try to drill down and isolate what the heck happened.”
Because UI tests sit at the top of the testing pyramid, they are susceptible to being affected by incomplete testing at lower levels of the pyramid, which includes unit and integration tests.
Steve McClure, a UI test engineer at MathWorks, said it’s important for other testers and developers to thoroughly unit test to ensure all the individual code components are bug free, before handing code off for UI testing.
“At that point, if anything foundational is wrong, you have a huge stack [that’s being tested] to try to drill down and isolate what the heck happened — which should have been caught way, way early in a unit test,” McClure said. “‘Defect localization’ and the time to triage is huge in a system-level test, so it’s better to work your way up the pyramid from the bottom.”
Because companies generally have UI testers as a shared resource, allocating one team of testers for many development teams, McClure encouraged developers to run some UI unit tests themselves — it saves UI testers some time, allowing them to focus on the bigger picture across more development teams.
Give More Context
Tension in the tester-developer relationship is a result of the nature of the work. Testers point out problems in developers’ code, who then have to make time to fix them. That’s not inherently problematic, but it can cause tension when testers and developers don’t see eye-to-eye about whether something really needs to be fixed.
UI testing is especially prone to that sort of uncertainty because UI tests are focused on user experience, which is subjective.
“How something should behave when an action is taken is debatable because it’s up to personal experience,” Hill said.
That doesn’t mean developers and testers should throw their hands in the air and declare a stalemate whenever a disagreement occurs, though. Developers can prevent these problems from happening in the first place by including testers in design conversations early in the development process.
“Describe, even just with little one-liners, the areas you explored.”
“Include them as early as you can in the process, and try to get some sense from them of what they would test if they received the code, and then do what you can to document it,” Hill said.
Documentation can help testers better orient themselves within a codebase, especially if developers make an effort to give testers plenty of information, rather than just listing what needs testing. Testers’ jobs are easier when they can get more context around what needs and doesn’t need to be tested.
“One thing you can do to help testers is describe, even just with little one-liners, the areas you explored,” Hill said. “Describe some concerns that you have, like, ‘I’m unsure how this would work, let me know what you think,’ or ‘I don’t think this will work with this other system.’”
It’s important that testers and developers work together and have a good relationship because it can be much easier to come to an agreement on the severity of the bugs testers have found if there is a history of mutual respect and working together.
Be Careful of Asynchronicity
It’s harder to run UI tests with consistent results than other types of tests because UI tests are exposed to more external factors than, for instance, backend unit tests.
“We always want our tests to be able to provide us with useful information, but UI testing is just less deterministic,” said Colin Cahill, lead test engineer at Extend. “We’re dealing with network requests, page rendering, memory — we’re dealing with all of these other things that can often make the tests less useful, and make the information less meaningful.”
Especially in fields like web development, software products are subject to the whims of external factors that can change the results of a UI test from one instance to the next.
McClure said it’s because UI testing often relies on communication that happens over the internet. The timing of when functions get triggered may differ depending on how far away the user is from the server running the code. While the code may be written with the assumption that things happen in a specific order — one function calls another function, which then calls another — the actual order may be different on different occasions.
“UI is notoriously asynchronous — flaky tests, sporadic failures, that sort of stuff,” McClure said. “If you click a button in your webpage, it’s going to have to talk to a few layers, go back to the server, round trip back to the view to say, ‘You’ve clicked me once.’ All of those patches, they’re all kind of chained up, waiting on a response to come back. If you allow it to respond in five seconds, you’re fine; if you rush it and you only allow 0.1 seconds, once in a while you might go to verify that it was clicked, but it doesn’t look like it.”
“Exacerbate the flakiness, just to make sure that it works at scale, under stress.”
That flakiness introduces another element of uncertainty, making it harder to figure out what in the code is causing the unexpected behavior and how to fix it. Cutting down on this uncertainty is another reason to thoroughly test individual parts before sending them for UI testing. Developers can help by running unit tests synchronously, such as mocking out portions of asynchronous code, and making sure individual components are working as expected on their own.
Once individual components are tested, the next step is stress testing the network latency to uncover the different ways asynchronous problems can manifest.
“Exacerbate the flakiness, just to make sure that it works at scale, under stress,” McClure said. “That might be a partnership with the QA, to think of almost off-the-wall ideas of how to really stress the system.”
McClure’s company uses an automated system to test code under heavy network load. He also recommended developers test by running their code remotely rather than on their local machines, and by pairing the component they’re building with another developer’s code, to tease out unexpected behaviors caused by the interactions.
“In production, the backend is several states away, or hundreds of miles or thousands of miles, and all of a sudden, the speed of light is going to weigh you down,” McClure said. “Under typical development models, a round trip may take one second, so I’ll wait for two and I’m safe. But in the real world, everybody has some laggy internet. Once in a while, it’ll be five or 10 seconds, or 30 seconds or a minute. You need to gracefully recover from it.”
Share Technical Knowledge
Software development advances pretty quickly, with new libraries, frameworks and even new languages springing up all the time. It can be a lot for developers to keep up to date, but testers have an even harder time staying on top of changes, while still being affected by them.
“Things change really quickly in the world of browsers and web engines,” Cahill said. “Those changes will absolutely have effects on how we write our UI tests. Oftentimes, developers will know about those, but there’s not necessarily a channel of communication between those engineers and test engineers… That’s a good opportunity for shared knowledge. When we’re dealing with very recent changes that will break code or require change, that kind of information is useful for testers because they have more information about what to write tests against.”
On the other hand, developers can gain a high-level understanding of the ways applications could break by learning from UI testers.
“Not every developer can see every bug, but every tester is going to see it.”
“I’ll end up working on something and just developing this almost encyclopedic knowledge of the types of issues that will occur in certain parts of the application,” Hill said. “Not every developer can see every bug, but every tester is going to see it. You’re just going to get a better appreciation for all the different ways it can fail.”
Testers get to see the full spectrum of bugs — from serious ones that cause production failures down to trivial ones that can be ignored — so they quickly learn which ones are priorities, Hill said. Developers who reach out and learn from testers’ experience can help the two groups develop a more similar set of priorities, and ease any tension caused by disagreements over the importance of problems caught by testers.
“And if the developer and QA can agree on severity in absolute terms, then that contention will go away,” Hill said. “It’ll actually make it easier for everyone to prioritize.”
