In the world of quality assurance, there’s no one set process: Each company — and each QA team — has its own set of requirements that ultimately depends on the product at hand.
However, there are a few noteworthy similarities across the world of QA. For example, without a constant feedback loop between the product and engineering teams, QA can breakdown.
For Jeff Rogers, head of QA at the healthcare technology company Tempus, any changes to requirements need to be circulated amongst relevant teams.
“When requirements change, the agreed-upon success criteria is evaluated by engineering, product and QA team members, and commitments are restated and/or re-forecasted,” Rogers said.
Similarly, at security company Trail of Bits, a good unit-testing feedback cycle is essential for its QA process. According to Assurance Practice Lead Stefan Edwards, this feedback cycle minimizes rework for the QA team as well as keeps the product in the planning stages rather than the reaction stages.
At Bread Financial, a rapidly-growing e-commerce company, documentation, in addition to communication, are key during the QA process. Both allow for each team member to have access to the data in order to understand its purpose, scope and edge cases, said Christina Kung, director of engineering.
Built In recently caught up with 14 leaders to dig deeper into their respective QA processes and how it helps them create a better product for their customers.
QA Best Practices to Keep in Mind
- Work closely with your product development team
- Utilize automation
- Continuously groom the test suite
- Hold peer reviews often
- Integrate agile processes early
- Prioritize bug tickets
- Hold exploratory testing
Erika Hayden
DIRECTOR OF QUALITY ASSURANCE
CCC’s Director of QA Erika Hayden said no QA practice is more important than the other. Working closely with product development, testing quickly and identifying performance issues early are a few strategies that Hayden said have contributed to their success.
What’s the most important best practice your QA team follows, and why?
I will share a few that I believe are the source to our success. First, we work closely with the product development teams to understand our customer and how they use our applications. Second, with such a high demand for the features that we design and develop, the QA team needs to have the ability to test quickly while maintaining the highest level of quality. This is where automation comes into the play to assist with the regression testing. We develop automated tests as soon as the code is stable, which allows us to execute thousands of critical customer workflows in days versus weeks. It is crucial to execute regression tests for a release, as successful tests give us the confidence to move to production.
Lastly, it is critical to identify performance issues early in the development lifecycle. We have a dedicated performance testing team that works closely with the product development, architecture and infrastructure teams with the collective goal to identify performance issues as soon as possible by discussing the changes in upcoming releases.
By building out the performance test harness early in the development lifecycle, we can identify any performance issues early and allow the teams more time to address them. Performance issues may take more time to address, so, starting early is key. It is also important to have a dedicated environment in which you can simulate production use cases and volume and match your production configurations as much as possible.
How do you determine your release criteria?
Working on an enterprise QA team, we receive release candidates from several teams that will ultimately be deployed to production together. Our goal is to have a release at least once a month to meet the needs of our customers. Proper planning is essential for both an efficient and high quality successful release. With the help of our release management team, we start by clearly identifying the release candidates that teams want to go into a release. The release candidates are prioritized with the customer focus in mind.
The QA team then meets with the leads of the product development teams to understand the release candidates in detail and estimate the time to test all features. If there are features that cannot be tested in the release schedule timeframe, those items are planned for the next release. These release candidates are reviewed with the appropriate market and product managers to confirm alignment with current customer needs. If there is an issue, we work collectively with the release management team to identify options to swap priorities.
How do you ensure the QA team stays up to date on shifting requirements, and how do you plan ahead to ensure changes are handled smoothly?
Communication and collaboration are the keys to our success. This has become more important with the entire team working remote. Prior to COVID-19, we had releases planned out and were well on our way to executing against those plans. After COVID-19, we quickly reprioritized the development of features that we knew would be most important to our customers.
We include all impacted parties as soon as we can in the discussions when we are shifting the priorities, and share critical information via collaborative meetings and not just in email. This lets us provide the organization with a clear picture of when the new requirements can be satisfied, and we are confident in our ability to deliver against the new plan.
Parina Madaan
QUALITY ASSURANCE MANAGER
Project requirements can shift quickly, which can lead to confusion across teams. While Morningstar QA Manager Parina Madaan admits that this remains a challenge for her team, one strategy she’s implemented to help is making sure that both the QA and development teams are operating on the same tech stack.
What’s the most important best practice your QA team follows, and why?
It is important to continuously groom the test suite and make sure test cases add value. This creates a focus on quality over quantity. We run smoke tests with every commit and deploy so we can quickly find high-risk issues. This reduces costs and maintains customer satisfaction.
How do you determine your release criteria? Give us an example of your typical process.
It starts with considering quality from a team perspective. Units, integrations and end-to-end tests are run before releases. When both product and engineering meet for a go/no go, we review all test results and confirm that the scope of the sprint is complete. A final regression is run on the most stable environment with the customer in mind, and we run a final smoke test in production after the deployment to confirm the release.
How do you ensure the QA team stays up to date on shifting requirements, and how do you plan ahead to ensure changes are handled smoothly?
That’s a big challenge, but a few things help. It starts with making sure the QA and development teams are using the same tech stack for coding and automation. This allows for more interaction and blurs the line between the disciplines. Product spends a lot of their time focusing on prioritization and communicates this daily during team standups. Technology is allowed to question product. This creates a steady balance where the entire team is focused on the business value for the customer.
Mike
DATA CENTER ENGINEERING LEAD
At the trading firm DRW, the QA team’s most effective tool is its peer review. Mike, who oversees DRW’s data center engineering team, explained why.
What’s the most important best practice your QA team follows, and why?
Peer review is one of the most effective QA tools we have in our arsenal. We’ve built this practice into the project cycle and have our teams constantly rotate through various deployments to ensure iterative reviews throughout the deployment lifecycle. This approach allows for ongoing validation that all changes are accounted for and the install met our high standards. It also benefits our team by consistently exposing each of us to new projects and groups, which provides ongoing opportunities for learning and development. We believe the more deployments we handle, the better we become at anticipating users’ needs and improving our approach on subsequent projects.
Since we are immersed in layer 1, the physical layer, peer review doesn't involve checking code or running code through a simulation. Instead, we follow these simple steps: We utilize our deployment data tools to encapsulate the scope of work, collaborate with other teams to make sure all changes are accounted for and, lastly, we document our deployments in a repository so that we can all see what practices work best for what deployments.
How do you determine your release criteria?
Our process starts and ends with the deployment data tools we have developed with DRW’s software engineers, serving as the communication backbone between our team and the rest of DRW. These tools allow for a constant flow of communication, which enables us to efficiently adapt to changes in internal requirements or evolving external factors in the markets in which we operate.
How do you ensure the QA team stays up to date on shifting requirements, and how do you plan ahead to ensure changes are handled smoothly?
As our team goes through deployment processes, we collaborate independently to make sure all changes are accounted for and installments have met our standards that we have laid out as a team. Evaluation, collaboration, communication and a shared desire to continuously push the boundaries of what we do allows us to effectively deploy equipment at a rapid pace. This brings a tremendous benefit to our firm as we seek to find a new edge in markets around the world.
Aaron Toran
QUALITY ASSURANCE ENGINEER
Topstep’s QA team participates in all of the company’s agile ceremonies, which QA Engineer Aaron Toran said helps his team know ahead of time when transitions in product changes will occur and how they could impact other stories.
What’s the most important best practice your QA team follows, and why?
Having a deep understanding of our business goals and users’ behavior is key to ensuring that we provide the best possible experience for our customers. We continually put ourselves in our customers’ shoes by testing our products’ quality and finding new ways to improve. Getting involved in the agile process in the early stages of a project helps us identify missing criteria and point out edge cases that may not have been considered before.
A few more best practices that we exercise daily include keeping up-to-date with bug tickets and requests, advocating for quality improvements, understanding priority versus severity when it comes to fixing bugs, and keeping time for exploratory testing, which is especially important when there is an extensive integration happening and we want to avoid all of the individual components from breaking down when combined.
How do you determine your release criteria?
It begins in the planning phase, where we plan out the entire roadmap of a project, including development work, quality assurance, integration and staging testing, release and product testing. The goal is to have each story ready for release when development and QA work is completed. We discuss the acceptance criteria during the grooming process and find edge cases and potentially blocking stories. There’s also the possibility of more stories being needed for the expected functionality, so additional time is set aside for supplemental planning as required.
It’s not uncommon for an epic to be too big for a single release. When this happens, we stage a series of dark releases wrapped in feature flags. Although this process increases the amount of work required for testing, it also improves the overall release confidence because we are now testing both new and old flows.
How do you ensure the QA team stays up to date on shifting requirements, and how do you plan ahead to ensure changes are handled smoothly?
Communication is critical, and we are continually collaborating with multiple departments company-wide when new changes are being made. Because we participate in all of the agile ceremonies, we know ahead of time when transitions will occur and how they could impact other stories. This process also gives us the ability to update test plans and formulate questions regarding the end state of the feature and how it works as a whole. In addition to our daily department meetings, we also have weekly QA sync meetings to go over the stories we have planned for and groomed. The weekly meetings ensure that each team member is up to date on project priorities and avoids any single points of failure if someone falls a little behind. We also use this time to go over automation testing and address any process pain points.
Bedford West
QUALITY ASSURANCE MANAGER
At BenchPrep, the learning management system’s QA team created a QA coordinator role that gathers feedback from each member of the QA team during a given sprint cycle and relays information between QA, engineering, product, design and support teams. QA Manager Bedford West said this role has reinforced the importance of communication on his team.
What’s the most important best practice your QA team follows, and why?
We see frequent and transparent communication as our most important best practice. We do our best work when every member of our team is fully informed and feels safe to share their ideas or to question the status quo. This practice of communication isn’t simply a guiding principle, it’s concretely baked into our process and execution. Our QA coordinator role, which rotates every sprint, supports our emphasis on communication. We also conduct a QA retrospective at the end of every sprint in addition to our scrum team retrospectives so we can align on best practices across our discipline.
How do you determine your release criteria? Give us an example of your typical process.
Releasing a product is an all-team decision. We don’t view QA as the gatekeepers of our release criteria. Reinforcing our communication best practice above, we strive to openly communicate the current customer and business risks of releasing our code, data and configuration to production. We do this on a story-by-story basis by creating an organic test plan via testing notes for each JIRA ticket. We treat these testing notes as a lightweight, malleable checklist by which we can communicate remaining risk in a sprint. Our QA coordinator takes point on communicating this out to the broader technology and product teams to ensure everyone is on the same page prior to hitting “go.”
How do you ensure the QA team stays up to date on shifting requirements, and how do you plan ahead to ensure changes are handled smoothly?
The only way to ensure QA is looped into changing requirements is to make them part of the conversation early and often. As equal members of our scrum teams, QA participate in ideation, refinement and planning. To ensure changes are handled smoothly, each team has QA run a test planning meeting at least once a sprint for upcoming work. In this way, everyone can brainstorm and align on how changing requirements can be adequately addressed through testing and development. When requirements change mid-sprint, we try to similarly discuss these items in our daily standups and Slack channels.
Neil Duggan
QA DIRECTOR
Technology and design company Work & Co defines and launches digital experiences that transform companies all over the world. At the company, writing test cases is one of their most important QA practices. “The act of writing test cases enables our QA team to sit down and think critically about what they’re testing, how they test each feature, and the personas and scenarios required to test the functionality fully,” QA Director Neil Duggan said.
What’s the most important best practice your QA team follows, and why?
One of our most important best practices is writing test cases. While test cases are not currently seen as “cool” in many circles, they are a fundamental part of our testing process for a few reasons. The act of writing test cases enables our QA team to sit down and think critically about what they’re testing, how they test each feature, and the personas and scenarios required to test the functionality fully. Test cases are a valuable tool for regression testing at the end of a project, and they form an essential part of our handoff to a client. At Work & Co, we build our products to live on after launch, so handing off test cases that our clients can utilize for future iterations of the product is a duty of care.
Quality can be a subjective term, so the most important thing is to make your release criteria specific and measurable.”
How do you determine your release criteria?
Quality can be a subjective term, so the most important thing is to make your release criteria specific and measurable. As part of our QA process at Work & Co, we take a two-tiered approach to platform support. Tier one platforms are fully supported — these are usually the most up-to-date browsers, newest devices and operating systems, whereas tier two is typically one OS or browser version back.
This helps us when it comes to defect prioritization. We classify defects into four priority categories: blocker, critical, major and minor. At Work & Co, we commit to releasing software with zero tracked blocker and critical defects left open, and a maximum of 10 percent of the total remaining major defects open. We work closely with our clients throughout their user accepting testing (UAT) phase to ensure their satisfaction with our product, and only when both teams are happy are we ready to release the product.
How do you ensure the QA team stays up to date on shifting requirements?
Work & Co’s model is based on our entire team sitting together and collaborating closely. Our QA team is a part of the project team and sits side by side with our strategists, designers, developers and product managers. This allows the whole team to collaborate and adjust together as requirements change. Since March, our teams have been working remotely from our offices due to COVID-19, so we’ve had to embrace technologies like Zoom, Slack and Dropbox to ensure we remain just as collaborative apart as we are when we’re physically together in our offices.
From a planning perspective, the QA team is fully integrated into our project lifecycle. Most notably, we have them begin their work simultaneously with the development team. We’ve found this increases the ability to properly account for the QA process on each project, rather than reactively responding to tickets as they are assigned.
Chris Griffin
CO-FOUNDER
Financial technology and insurance company Narmi helps thousands of community banks and credit unions compete with megabanks by creating a better online banking experience that’s built on the foundation of collaboration and teamwork. According to Co-Founder Chris Griffin, a tight communication loop between the product and engineering teams is key for the QA and release processes. As new requirements come in from the product side, engineers complete each one and pass it back to product for QA.
What’s the most important best practice your QA team follows, and why?
Our applications are used for everyday banking, so dependability is critical. Before a change has even been merged into our codebase, it has gone through a number of checks and balances that help prevent and catch issues well before they could negatively impact production.
Every pull request is peer-reviewed, all changes are tested, and changes go through unit, integration and acceptance tests before being merged. We use feature flags to allow us to develop in smaller chunks, and team members are encouraged to test-drive their code.
After the merge, changes are deployed to staging and customer-specific UAT environments that allow us to test the change against various configurations of our platform. Once any bugs are discovered and fixed, we deploy to production, confident that the code we’ve just released is safe for our customers to use.
How do you determine your release criteria?
As a service provider for banks and credit unions, our release process is tailored to the needs of our customers. Internally, we deploy continuously to staging, which helps to identify classes of bugs quickly and encourages a more iterative QA process.
Externally, our production release process is tailored to the needs of our customers. Some financial institutions are very forward-thinking, while others prefer the software they know and love as they proceed through their digital transformation.
We’ve taken the best of both worlds to create a release process that allows us to QA and release bug-free software to financial institutions that want the latest features while allowing more comfortable customers to change only every couple of months. This keeps both sides of our customer base happy while letting us continue to develop, merge code and QA new features for everyone.
We make sure releases are smooth by keeping a tight loop of communication between product and engineering.”
How do you ensure the QA team stays up to date on shifting requirements?
Communication is key! We make sure releases are smooth by keeping a tight loop of communication between product and engineering. As new requirements come in from the product side, engineers complete each one and pass it back to product for QA. We find that having the people who create the requirements be the same ones who confirm them really tightens up our process and makes sure that what’s requested is what gets delivered.
Jeff Rogers
HEAD OF QA
Tempus enables physicians to deliver personalized patient care through an interactive and analytical machine learning platform that makes data accessible and useful. At Tempus, effective QA practices require all hands on deck. “Our products are built, managed, sold and supported via an equal partnership between our engineering, product management and operations team,” Jeff Rogers, head of QA, said.
What’s the most important best practice your QA team follows, and why?
Pairing continuous delivery with an extreme focus on our foundational principles: patients (our customers), security of their data and regulatory compliance. We do this by limiting the blast radius for changes in product design and optimizing our test strategies around the various failure modes. This allows us to push the envelope on speed to market and continuous improvement.
There is no standard-quality bar for release across all systems; each group assesses each change against the potential risks to our core principles.”
How do you determine your release criteria?
Tempus products are built, managed, sold and supported via an equal partnership between our engineering, product management and operations team. Each group brings its own perspective on quality and the criteria for change and release, and testing activities are aligned and negotiated as necessary. There is no standard-quality bar for release across all systems; each group assesses each change against the potential risks to our core principles.
How do you ensure the QA team stays up to date on shifting requirements?
At Tempus, QA is embedded in product scrum teams, which are tasked with consulting and coaching on adequate success criteria for all requirements, with an eye towards risk, compliance and customer impact. When requirements change, the agreed-upon success criteria is evaluated by engineering, product and QA team members, and commitments are restated and/or re-forecasted. If and when issues arise, QA is accountable for all root cause findings and remediations.
Christina Kung
DIRECTOR OF ENGINEERING
E-commerce company Bread Financial aims to transform the world of paper credit card applications and hidden interest rates by providing leading point-of-sale financing options for merchants. Because Bread’s QA team has grown so rapidly, communication is the key for its success. “This rapid growth means documenting our features clearly so that anyone can look at our work and understand its purpose, scope and edge cases,” Director of Engineering Christina Kung said.
What’s the most important best practice your QA team follows, and why?
This past quarter, we’ve built a new QA team from scratch, onboarding a new QA lead and five new QA engineers — and we are still hiring. This rapid growth reminds us to take communication seriously. This means documenting our features clearly so that anyone can look at our work and understand its purpose, scope and edge cases. The most important practice is for our QA to ensure that everyone is filling out acceptance criteria and test cases as thoroughly as possible, and secondly keeping them up to date as the project evolves.
How do you determine your release criteria?
We have prioritized automating as much as possible. From unit tests to smoke tests, our goal is to have early and consistent feedback for any code that is committed. For example, whenever a pull request (PR) is opened our unit and integration tests are run. When the PR is merged to master our smoke tests are run, and then rerun, that’s when it goes to staging.
Of course, our automation suite is always a work in progress and it is never 100 percent. While we continue to fill in the gaps in test coverage, wherever they may be, the entire team is encouraged to do exploratory testing on features before they go to production. When our QA lead gives the blessing, we then move to production.
The most important practice is for our QA to ensure that everyone is filling out acceptance criteria and test cases as thoroughly as possible.”
How do you ensure the QA team stays up to date on shifting requirements?
This is a nod to the earlier question on best QA practices — we are diligent in keeping our acceptance criteria up to date. The ticket details are our single source of truth for engineers, product managers and QA. Whenever these details are changed we are notified, and we can also use daily standups as an opportunity to discuss those changes.
Stefan Edwards
ASSURANCE PRACTICE LEAD
Security company Trail of Bits helps secure some of the most targeted organizations and products around the world by combining high-end security research with a real-world attacker mentality to reduce risk and secure code. According to Stefan Edwards, assurance practice lead, having a good unit-testing feedback cycle is essential. This feedback cycle minimizes rework for the QA team, as well as keeps the product in the planning stages rather than the reaction stages.
What’s the most important best practice your QA team follows, and why?
Having a good unit-testing feedback cycle to minimize rework. For example, if QA catches something, and you generate a new test case and find a new breakage, making sure it feeds back to a unit test so QA doesn’t have to test that again is huge.
Make sure your QA testing practices are as “engaged left” as possible, meaning they focus heavily on the planning or proactive stages rather than the reaction stages.”
How do you determine your release criteria?
When clients ask how to determine these sorts of things, I’d say you need really good tasking and prioritization because you need to understand how your software development life cycle (SDLC) leads to actual work being done. Once you’re as close as possible to completing that set of goals, you’re at version 1.0.
How do you ensure the QA team stays up to date on shifting requirements, and how do you plan ahead to ensure changes are handled smoothly?
Generally, when you QA and test things, it’s extremely difficult to stay up to date with those requirements. So, make sure your QA testing practices are as “engaged left” as possible, meaning they focus heavily on the planning or proactive stages rather than the reaction stages.
Ricky Otero
DIRECTOR OF QUALITY ASSURANCE
IntelePeer enables companies to communicate better by leveraging omnichannel automation and self-service AI and analytics through its cloud platform. To ensure shifting requirements run smoothly, the QA team meets every morning to discuss daily requirements and priorities to ensure each member stays up-to-date on any changes that have occurred, Ricky Otero, director of quality assurance, said.
What’s the most important best practice your QA team follows, and why?
Our QA team at IntelePeer has many practices that we follow, but one of the most important is our regression testing. We execute regression testing for each software prior to production release so we can ensure existing customers not only receive the latest and greatest product updates but also that the system still performs to 100 percent specifications.
Communication from our management and tagging Jira tickets with release and sprint info ensures the QA team stays up to date on shifting requirements.”
How do you determine your release criteria?
Our release criteria is determined by our product management team. The release depends upon the functionality they determine is needed in production at the release date. For example, by acceptance of successful use case completion, that signals to QA that a customer will not be impacted by new product updates from the collaboration of product management, development and QA before each release.
How do you ensure the QA team stays up to date on shifting requirements, and how do you plan ahead to ensure changes are handled smoothly?
Our team meets every morning to discuss daily requirements and priorities. Communication from our management and tagging Jira tickets with release and sprint info ensures the QA team stays up to date on shifting requirements. We also have a Jira board that contains items or tickets for each release with applicable user stories. The Jira board assists in helping with shifting requirements to see tickets that have been moved to the next release or updated with more information. This information helps us to plan ahead and begin creating test cases for new functionality and existing regression.
When it comes to QA best practices, the team over at InfluxData prefers to keep things simple.
“The most important best practice is not to have a separate QA team,” Barabara Nelson, head of application engineering, told Built In San Francisco.
Instead of making QA a separate function within the engineering department, InfluxData places the onus for developing quality code squarely on the shoulders of its developers, who write their own automated tests. According to Nelson, this practice has led to higher-quality code from developers and a greater feeling of ownership over the products they work on. It also helps the team maintain its breakneck release pace, with Nelson estimating that InfluxData has somewhere between 10 to 20 deployments per day.
InfluxData is certainly not the first company to move away from QA teams and likely won’t be the last, especially given the case Nelson makes for putting developers in charge of quality assurance.
What’s the most important best practice your QA team follows, and why?
Nelson: We do not have a QA team and instead hold the development team responsible for the quality of what they deliver. Every team is responsible for developing the tests to ensure that the code they deliver is reliable and works as expected. This mindset has led to higher-quality code and gives developers a much stronger sense of ownership of the product. One thing that has helped us make this shift is the evolution of test frameworks — like cypress.io, for example — and continuous delivery pipelines, which make it easy for developers to write automated tests that are run on every code change.
We also rely heavily on feature flags and encourage developers to release very small code changes at a time.”
How do you determine your release criteria, and can you give us an example of your typical process?
We have an automated CD pipeline and are constantly deploying new features into our cloud service. We have sets of tests that are triggered during the pipeline and if the tests fail the deployment fails, the team is notified.
For example, we have a set of unit tests that run prior to merging code to master. We have another set of tests that run when the master branch is deployed to staging, then again in pre-production and then in production. If the tests fail, we roll back the deployment. So, the release criteria is pretty simple: If the tests pass, you can release. We also rely heavily on feature flags and encourage developers to release very small code changes at a time. We can have the code in production but hidden behind a feature flag until we are ready to expose the new feature to our customers.
Project requirements can change rapidly, particularly in Agile development cycles. How do you ensure the QA team stays up to date on shifting requirements, and how do you plan ahead to ensure changes are handled smoothly?
We avoid part of this problem by not having a separate QA team. Every feature request needs to have clearly defined acceptance criteria before the developer starts working on the feature, and we use the acceptance criteria to define the tests that need to be written for that feature.
Defining the acceptance criteria typically happens shortly before the developer starts work on the feature, so we avoid having stale acceptance criteria if the requirement changes. If a requirement changes during development, we update the acceptance criteria and the developer updates their code and their tests to match the new requirements.
Melissa Bruckner
SENIOR QA ENGINEER
Software company Documoto helps equipment manufacturers, their network and their equipment owners to “keep the world’s machines working,” with its proprietary SaaS solution. Communication across teams has been the key to the small company’s success, Senior QA Engineer Melissa Bruckner said, resulting in engineering and product teams being highly integrated during the development process.
What’s the most important best practice your QA team follows, and why?
One of the best practices that we follow is using a manual and automated approach to testing. We are a small team, and it is not practical to run a full manual regression for every ticket completed. We take tickets as engineers, complete them and thoroughly manually test them to ensure all acceptance criteria are met. Once we are confident that the ticket has met all the requirements, we then add in limited automated tests to be for future automation.
How do you determine your release criteria?
My typical release criteria are met on a ticket-by-ticket basis. Our product team does a great job of making sure tickets have clear and concise acceptance criteria. Our engineering team does a great job of noting areas of the code that were touched. I focus heavily on the areas of code that were changed by manually testing. I use existing automation to ensure the rest of the app is still doing what it should be doing.
Communication has been the key to our success. If a requirement changes, we are generally in the know right away to change our test cases.”
How do you ensure the QA team stays up to date on shifting requirements, and how do you plan ahead to ensure changes are handled smoothly?
Because Documoto is a small team, the engineering and product teams are heavily integrated. This gives us the ability to approach testing early and at numerous stages of development. Communication has been the key to our success. If a requirement changes, we are generally in the know right away to change our test cases. We will also run automation right away on new or changing functionality, so we are aware if the change affects anything else in the application.
Priya Narayan Rao
MANAGER OF QUALITY ASSURANCE TESTING
Testing from the beginning state of the development process helps the QA team at CSG detect bugs early and address the defects, said Priya Narayan Rao, a manager of QA testing. Rao shared how else this key practice impacts the customer engagement platform’s products and features.
What’s the most important best practice your QA team follows, and why?
Our most important best practice is to have our QA team involved with product development as early as possible. We ensure this happens by following this process: user automation test; release; retrospective and feedback; project starts with requirement gathering; grooming; estimation; planning; development and prepare test scenarios; and testing. By testing early, we’re catching bugs early and we’re improving the quality of the software, which reduces the cost of quality maintenance.
How do you determine your release criteria?
CSG’s software is ready to be released when business requirements and acceptance criteria are met, unit testing is completed — which involves creating tests in isolation specific to independent units — and integration testing is performed. Then we prepare document release notes and have the entire QA team sign off on it saying the software is ready to be released into production. A formal document is then attached to the release CRQ.
How do you ensure the QA team stays up to date on shifting requirements, and how do you plan ahead to ensure changes are handled smoothly?
We test our products continuously. Continuous testing is the only way to ensure that progress is being made on the product. Also, we collect and provide ongoing feedback to guarantee that the product meets the needs of the business. To ensure changes are being handled smoothly, we use less documentation and have a reusable checklist in addition to automated testing. This allows our team to focus on testing as opposed to incidental details. Finally, we have flexibility designed into test scenarios and focus less on detailed test plans. The team works on initial automated testing on application aspects that are most likely to remain unchanged. They then take time to analyze the requirement and have a backup plan to work on changes if needed.