The Role
Lead QA automation strategy and frameworks across unit, integration, API, UI, performance, and security testing. Introduce AI-assisted testing and LLM evaluation, integrate quality gates into CI/CD, mentor engineers, define QA KPIs, and partner with product, security, and DevOps to ensure release readiness and measurable quality improvements.
Summary Generated by Built In
AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
WHY JOIN US
If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!
ABOUT THE ROLE
We are looking for a Senior/Lead QA Automation Engineer to own and evolve automated testing frameworks, introduce AI-assisted QA practices, and define quality engineering strategy across an AI-driven product development environment. You will architect automation across the full testing stack — unit, integration, API, UI, performance, and security — integrate quality gates into CI/CD pipelines, design evaluation frameworks for LLM-based features, and mentor engineers in modern automation and AI-augmented workflows.
WHAT YOU WILL DO
- Own and evolve the organization’s automated testing framework and QA tooling ecosystem;
- Define and standardize AI-assisted testing practices, including test case/data generation, coverage discovery, and responsible AI guardrails;
- Implement intelligent test selection, regression analysis, and measurable quality metrics to optimize release quality and reduce risk;
- Ensure AI-augmented tests remain deterministic, maintainable, production-ready, and supported by traceability, validation, and drift detection;
- Architect and scale automation across unit, integration, API, UI, contract, performance, and security testing;
- Champion modern testing principles, integrate automation into CI/CD, and enforce quality gates and release readiness criteria;
- Design evaluation frameworks for AI-powered features, including prompt robustness, deterministic validation, and human-in-the-loop review where appropriate;
- Lead root-cause analysis for systemic quality issues and continuously improve automation ROI and test effectiveness;
- Mentor engineers on modern automation, AI-assisted workflows, prompt engineering, and review standards for AI-generated code;
- Define and track QA KPIs, provide data-driven Go/No-Go recommendations, and improve testing strategy based on measurable outcomes;
- Partner with Engineering, Product, Security, and DevOps to embed quality early, influence architecture, and integrate accessibility, security, and privacy testing throughout the SDLC.
MUST HAVES
- 5+ years of experience in software quality engineering, test automation, or SDET roles;
- 3+ years leading automation strategy or serving as a senior technical QA authority;
- Hands-on experience using AI tools for test generation, refactoring, and automation maintenance;
- Experience evaluating LLM-based systems (prompt robustness, structured outputs, drift detection, and versioned regression suites);
- Strong proficiency in at least one programming language (TypeScript, Java, Python, etc.) and modern test frameworks (e.g., Playwright, Cypress, Selenium);
- Demonstrated experience designing, maintaining, and scaling test automation frameworks and CI/CD quality gates;
- Strong understanding of distributed systems, HTTP lifecycle, and database querying (SQL/NoSQL);
- Experience mentoring engineers and leading quality initiatives with measurable outcomes;
- Upper-intermediate English level.
NICE TO HAVES
- Working knowledge of secure testing practices (OWASP Top 10) and performance testing strategy;
- Familiarity with compliance, privacy, or regulated industry SaaS environments;
- Experience implementing performance/load testing strategies;
- Experience with accessibility testing strategy and tooling.
PERKS AND BENEFITS
- Professional growth: Mentorship, TechTalks, and personalized growth roadmaps.
- Competitive compensation: USD-based pay with education, fitness, and team activity budgets.
- Exciting projects: Modern solutions with Fortune 500 and top product companies.
- Flextime: Flexible schedule with remote and office options.
Skills Required
- 5+ years of experience in software quality engineering, test automation, or SDET roles
- 3+ years leading automation strategy or serving as a senior technical QA authority
- Hands-on experience using AI tools for test generation, refactoring, and automation maintenance
- Experience evaluating LLM-based systems (prompt robustness, structured outputs, drift detection, versioned regression suites)
- Strong proficiency in at least one programming language (TypeScript, Java, Python)
- Experience with modern test frameworks (Playwright, Cypress, Selenium)
- Demonstrated experience designing, maintaining, and scaling test automation frameworks and CI/CD quality gates
- Strong understanding of distributed systems, HTTP lifecycle, and database querying (SQL/NoSQL)
- Experience mentoring engineers and leading quality initiatives with measurable outcomes
- Upper-intermediate English level
- Working knowledge of secure testing practices (OWASP Top 10) and performance testing strategy
- Familiarity with compliance, privacy, or regulated industry SaaS environments
- Experience implementing performance/load testing strategies
- Experience with accessibility testing strategy and tooling
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
