QA Automation Engineer

Position Overview

We're seeking an experienced QA Automation engineering to establish and lead our quality assurance function. This is a critical role responsible for building comprehensive test automation frameworks, defining quality processes, and ensuring the reliability and security of our AI-native financial platform.

You'll be the first dedicated QA responsible for creating our testing strategy from the ground up, building a high-performing QA team, and establishing quality standards across frontend, backend, API, and AI/ML components. This role requires both deep technical expertise in test automation and leadership skills to build and scale a world-class QA organization.

• Build from Scratch - Define QA strategy, frameworks, and processes from day one
• AI/ML Testing - Test cutting-edge LLM applications and agentic workflows
• Full Stack Ownership - Cover frontend (React), backend (Python/FastAPI), APIs, and databases
• Leadership Impact - Build and lead a QA team; shape quality culture
• Ensure quality for mission-critical financial operations

________________________________________

Key Responsibilities

Test Automation Strategy & Framework (35%)

• Design and implement comprehensive test automation framework
• Build API test automation (REST, GraphQL)
• Implement frontend E2E testing (Playwright, Cypress)
• Create backend/integration test suites
• Establish CI/CD integration for automated testing
• Define test data management strategy
• Implement performance and load testing frameworks

Quality Leadership & Team Building (25%)

• Build and lead QA team (2-5 engineers initially, scaling to 8-10)
• Define QA processes, standards, and best practices
• Establish quality metrics and KPIs
• Conduct 1-on-1s and performance reviews
• Mentor QA engineers on automation and testing practices
• Drive quality culture across engineering

AI/ML & LLM Testing (20%)

• Develop testing strategies for LLM-powered features
• Test agentic workflows and AI decision-making
• Validate model outputs and prompt engineering
• Test AI accuracy, consistency, and edge cases
• Implement monitoring for AI behavior in production
• Ensure responsible AI practices (bias, fairness)

Security & Compliance Testing (10%)

• Implement security testing (SAST, DAST)
• Test authentication, authorization, and RBAC
• Validate data encryption and privacy controls
• Ensure SOC 2, GDPR compliance requirements
• Conduct penetration testing coordination
• Test multi-tenant isolation and security

Release & Deployment Quality (5%)

• Define release testing procedures
• Implement smoke and regression test suites
• Validate staging and production deployments
• Coordinate with DevOps on deployment gates
• Post-deployment validation and monitoring

Collaboration & Communication (5%)

• Partner with Product on acceptance criteria
• Work with Engineering on testability
• Report quality metrics to leadership
• Identify and escalate critical issues
• Document testing processes and runbooks

________________________________________

Technical Stack

Test Automation Tools

• API Testing: Postman, REST Assured, PyTest
• E2E Testing: Playwright, Cypress, Selenium
• Backend Testing: PyTest, unittest, integration tests
• Load Testing: JMeter, Locust, k6
• Performance: Lighthouse, WebPageTest

Programming & Frameworks

• Languages: Python, JavaScript/TypeScript
• Backend: FastAPI, PostgreSQL, REST APIs
• Frontend: React, Next.js, TypeScript
• Test Frameworks: PyTest, Jest, Mocha

CI/CD & DevOps

• CI/CD: GitHub Actions, Jenkins, GitLab CI
• Containerization: Docker, Kubernetes
• Cloud: AWS (EC2, ECS, EKS, Lambda)
• Monitoring: Datadog, CloudWatch, Prometheus

Security & Quality Tools

• SAST: SonarQube, Snyk, CodeQL
• DAST: OWASP ZAP, Burp Suite
• Test Management: TestRail, Zephyr, Jira
• Version Control: Git, GitHub

________________________________________

Required Qualifications

Technical Expertise

• 10+ years of QA/test automation experience with 3+ years in leadership
• Expert-level test automation skills (API, UI, backend)
• Strong programming in Python and JavaScript/TypeScript
• Proven experience building test automation frameworks from scratch
• Deep knowledge of testing methodologies (functional, integration, E2E, performance)
• CI/CD integration expertise with automated test pipelines
• API testing expertise (REST, GraphQL, WebSockets)

• Backend testing: Python, FastAPI, PostgreSQL, SQLAlchemy
• Frontend testing: React, TypeScript, Playwright/Cypress
• Performance testing: Load testing, stress testing, scalability validation
• Security testing: SAST, DAST, vulnerability scanning
• Database testing: SQL, data validation, database performance
• Cloud platforms: AWS (preferred), GCP, or Azure

• Experience testing LLM applications or AI/ML systems
• Knowledge of prompt engineering and model validation
• Understanding of AI testing challenges (non-determinism, bias)
• Familiarity with LangChain, LangGraph, or similar frameworks

Leadership & Management

• Proven track record of building QA teams (3+ people)
• Experience establishing QA processes and standards from scratch
• Strong mentorship and coaching skills
• Excellent communication with technical and non-technical stakeholders
• Experience with cross-functional collaboration (Product, Engineering, DevOps)
• Startup experience preferred - ability to work in fast-paced environment

Domain Knowledge

• SaaS platform testing experience (multi-tenant architecture)
• FinTech or financial services domain knowledge (preferred)
• Compliance testing experience (SOC 2, GDPR, HIPAA)
• Agile/Scrum methodology experience

Nice to Have

• ISTQB Advanced or similar certifications
• AWS Certified Developer or Solutions Architect
• Experience with chaos engineering and resilience testing
• Knowledge of observability tools (Datadog, New Relic)
• Open-source contributions to testing frameworks
• Technical writing or conference speaking experience

________________________________________

Success Metrics

First 90 Days

• Assess current testing landscape and create comprehensive QA strategy
• Implement API test automation framework with 50+ test cases
• Set up E2E testing framework (Playwright/Cypress) with critical path coverage
• Integrate automated tests into CI/CD pipeline
• Hire 1-2 QA engineers
• Establish test data management approach
• Define quality metrics and KPIs

First 6 Months

• Achieve 70%+ API test coverage
• Achieve 60%+ E2E test coverage for critical user flows
• Build QA team to 3-4 engineers
• Implement performance testing framework
• Establish security testing practices (SAST/DAST)
• Reduce production defects by 40%
• Implement AI/ML testing strategy for LLM features
• Achieve <10 minute test execution time in CI/CD

First Year

• Scale QA team to 5-8 engineers
• Achieve 80%+ overall test coverage
• Reduce production defect escape rate to <5%
• Implement continuous testing with <15 minute feedback loops
• Establish quality gates for all releases
• Achieve 99.5%+ test pass rate
• Build comprehensive test documentation and runbooks
• Establish SOC 2 compliant testing processes