Sr. QA Engineer

About HerculesAI

HerculesAI helps finance and operations leaders solve problems that are too complex, large-scale, or time-consuming for human teams to manage alone. Its platform automates the validation and verification of data across millions of high-volume, rules-based transactions, improving billing accuracy, reducing costs, and accelerating cash flow. Built on a modular, multi-AI agent architecture, HerculesAI delivers industry-specific solutions for staffing, insurance, government, and financial services. Its accuracy and consistency enable enterprises to achieve levels of precision and speed that were previously out of reach. 

Headquartered in the United States, HerculesAI also has offices in the United Kingdom, Armenia, Canada, and Portugal. 

About the role

As a Sr. QA Engineer at HerculesAI, you’ll lead the charge in embedding quality throughout our entire software development lifecycle. You’ll own end-to-end product quality—from defining success criteria and automation strategies to managing releases and ensuring post-deployment stability. You’ll work closely with Engineering, Product, AI, and DevOps teams to ensure every release meets our standards for reliability, performance, and user impact—making quality a shared responsibility across the organization.

What you'll do

• Lead end-to-end product quality and integrate QA across the SDLC (shift-left, CI/CD quality gates, test evidence as part of “definition of done”).
• Own Release Management: plan releases, cut release candidates, manage freeze windows, lead go/no-go, coordinate phased rollouts (flags/canary), and execute rollback plans.
• Design and maintain automation for UI, API, component, and E2E tests in partnership with all Engineering teams; establish non-functional baselines (performance, security, accessibility, resilience).
• Design, own, and evolve performance and stress testing (load, capacity, scalability): define SLIs/SLOs, create repeatable perf/stress suites, profile bottlenecks, and gate releases via CI/CD and PRV.
• Translate business initiatives into clear acceptance criteria and measurable Success Criteria Docs (KPIs, telemetry, rollout/rollback triggers).
• Partner with Engineering, Product, AI, and DevOps to ensure observability, PRV (post-release verification), incident retrospectives, and continuous improvement of quality KPIs.

Qualifications

• 6–10+ years in QA/Software Engineering, including ownership of release management and large-scale test automation.
• Required: Proficiency in Python (test harnesses, AI/LLM eval tooling, CI utilities) and ReactJS (TypeScript preferred) for UI testability reviews and building test fixtures/mocks.
• Hands-on with modern delivery stacks (microservices, containers/K8s, CI/CD) and test frameworks (Playwright/Cypress, PyTest/JUnit/TestNG, k6/JMeter for performance).
• Demonstrated experience validating AI/LLM features (prompt testing, guardrails/red-teaming, offline/online eval alignment).
• Strong systems thinking and risk-based testing; fluency in telemetry/observability (logs, metrics, traces) and security/a11y/performance gates, including capacity planning and perf profiling.
• Excellent communication skills for turning ambiguous requirements into unambiguous, testable criteria and leading cross-functional quality reviews.

Success Metric Examples
(The Sr. QA Engineer will owns definition, targets, and reporting of these KPIs.)

• Shift-Left & SDLC Health: % stories with testable criteria at grooming ↑; pre-merge test pass rate ↑; escaped defects from unit/component layers ↓; lead time for changes ↓.
• Release Quality: change failure rate ↓; rollback/hotfix frequency ↓; PRV (post-release verification) pass rate ↑; incident MTTR ↓.
• Automation & Coverage: automated regression coverage ↑; flaky test rate ↓; time to fix flaky tests ↓; security/a11y/perf gate pass rate ↑.
• Performance & Resilience: p95/p99 latency ↓; throughput ↑; saturation/error budget burn ↓; successful load/stress/soak test pass rate ↑; resiliency checks (retry/backoff, timeouts) pass rate ↑.
• Observability & Evidence: % launches with telemetry + dashboards + alert thresholds = 100%; quality evidence attached to releases ↑; data-driven retros with action closure rate ↑.