Be an Early Applicant
We harness the power of AI to tackle life’s complex questions with precision and insight.
The Role
As a Senior QA Engineer, you will oversee quality strategies and automation for AI systems, collaborate with teams, maintain documentation, and mentor engineers.
Summary Generated by Built In
Senior QA Engineer (AI Systems & Automation)Essential Software Inc. (ESI)OverviewEssential Software Inc. is a trusted partner to federal agencies, including the National Cancer Institute (NCI), delivering secure, cloud-based platforms that support large-scale cancer data and biomedical research.As a Senior QA Engineer (AI Systems & Automation), you will lead quality strategy and test automation for critical data platforms and AI-powered experiences. You will ensure both traditional software and AI/agentic systems are reliable, explainable, and safe in a federal, mission-driven environment.You will:
- Own end-to-end quality for complex web, API, data, and AI/ML-powered features
- Design AI-aware test strategies and automation that leverage GenAI and agentic frameworks
- Mentor QA engineers and collaborate closely with cross-functional teams and government partners
- Develop and maintain test plans, test cases, traceability, and test data for product and AI features
- Execute manual and automated tests for web applications, RESTful APIs, data workflows, and AI/ML features
- Own automated regression suites, release readiness criteria, and provide clear go / no-go quality signals
- Participate in agile ceremonies, validate end-to-end functionality, and ensure user stories (including AI features) meet acceptance criteria
- Manage the full defect lifecycle, including triage, prioritization, root cause analysis, and verification of fixes
- Maintain QA documentation, runbooks, and quality dashboards
- Design and execute test strategies for AI/LLM-powered capabilities, including virtual agents, chatbots, copilots, and RAG-based systems
- Use LLM-powered tools (e.g., ChatGPT, Claude, Copilot) to accelerate test design, data generation, exploratory testing, and script authoring
- Build and refine QA-focused AI agents that can:
- Scrape UI and verify DOM structures
- Validate data against backend or ground-truth sources
- Auto-generate and maintain test scripts
- Run self-correcting / autonomous test flows
- Evaluate and integrate agentic frameworks (e.g., OpenAI Assistants API, AWS Bedrock Agents, LangGraph, MCP) into QA workflows
- Define and monitor AI-specific quality metrics (accuracy vs. ground truth, hallucination and error rates, safety / policy adherence)
- Ensure AI and virtual agent experiences are accurate, consistent, and high quality in a federal context
- Plan and execute performance, load, and scalability testing (e.g., JMeter or equivalent)
- Validate data integrity and transformation quality across complex biomedical data pipelines and AI-enhanced workflows
- Partner with engineers and data scientists to ensure AI/ML models and integrations are testable, observable, and measurable post-deployment
- Mentor QA team members in both traditional and AI-augmented QA practices
- Collaborate with development, DevOps, product, UX, and data teams to improve testability, shift-left quality, and increase automated coverage
- Integrate automation into CI/CD (e.g., GitHub Actions, Jenkins, Azure DevOps, GitLab CI), monitor test health and flakiness, and address coverage gaps
- Communicate quality risks, trends, and mitigation plans to technical and non-technical stakeholders, including government partners
- Bachelor’s degree in computer science, Information Technology, Engineering, or related field
- 5+ years of software QA experience (manual and automation) in production environments
- 2+ years providing technical or process leadership (e.g., lead QA, primary product QA owner, mentor, or manager)
- Strong experience with UI automation tools (Selenium WebDriver, Playwright, or Cypress)
- Experience testing RESTful APIs and microservices architectures
- Hands-on experience integrating automated tests into CI/CD pipelines (GitHub Actions, Jenkins, Azure DevOps, or GitLab CI)
- Professional proficiency in Python or JavaScript for test automation
- Hands-on use of GenAI tools (e.g., ChatGPT, Claude, Copilot) for QA tasks such as test-case generation, data creation, and exploratory testing
- Understanding of AI/agentic concepts:
- Tool-calling / function invocation
- Multi-step / chain-of-thought workflows
- Autonomous / self-healing test flows
- AI-driven data comparison and validation
- Experience with performance / load testing (e.g., JMeter or equivalent)
- Proficiency with Jira or similar issue tracking tools
- Strong written and verbal communication skills, including the ability to explain AI-related quality risks to stakeholders
- Ability to prioritize, multitask, and operate effectively in complex, mission-driven environments
- AWS Cloud Practitioner certification
- Experience with modern automation stacks (Playwright or Cypress) and API testing tools (Postman, REST-assured, pytest, or similar)
- Experience testing AI/ML-powered features (LLM applications, RAG systems, agents, recommendation engines, or chatbots)
- Experience with one or more:
- LangChain or LangGraph
- AWS Bedrock Agents or OpenAI Assistants API
- MCP (Multi-Context Protocol) or similar orchestration frameworks
- Experience designing or testing internal QA copilots or automation bots for test authoring or execution
- Familiarity with test management tools (e.g., TestRail, Zephyr)
- Knowledge of accessibility standards (WCAG) and basic security testing practices
- Prior QA experience in healthcare, life sciences, biomedical informatics, or other regulated data environments
- ISTQB or similar certification
- Mission driven and customer focused, working directly with scientists, product owners, and data experts
- Curious and learning oriented, with encouragement to experiment with new AI tools, patterns, and technologies
- Work on cutting-edge biomedical informatics and AI projects with real-world impact
- Full-time role; primarily remote, with approximately four days per month onsite in Rockville, MD
- Competitive benefits, professional development opportunities, and a collaborative, supportive culture
Top Skills
Aws Cloud
Azure Devops
Cypress
Genai
Github Actions
Gitlab Ci
JavaScript
Jenkins
Jmeter
Playwright
Python
Restful Apis
Selenium
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Essential Software Inc. helps customers make scientific breakthroughs faster. Our integrated solutions enable our partners—including leading Research and Development (R&D) funded federal agencies—to foster collaboration in the scientific community. We offer solutions that empower innovation, advance the fight against cancer, and improve public health.
