Position: Senior QA Automation Engineer (AI-First)
Location: Remote
Contract Type: Full-time vendor (Direct contract with Inallmedia.com)
Time Zone Alignment: Central Time (CT) ±2 hours
🧭 About Inallmedia.comInallmedia.com is a global technology and design firm focused on building impactful digital solutions through remote, distributed teams across LATAM. We partner with international clients across industries, providing long-term technical expertise, product innovation, and team augmentation.
🚀 Project OverviewYou will join a forward-thinking engineering team building an innovative data center power management platform designed to optimize energy consumption and relieve grid pressure. As a Senior QA Automation Engineer (AI-First), your primary focus will be designing, building, and enforcing a robust automated testing strategy across distributed systems, Python microservices, event pipelines, and non-deterministic agentic AI workflows.
🔍 Key Responsibilities- Comprehensive Test Strategy: Own the automated test strategy across unit, integration, contract, end-to-end, and edge-deployment validation.
- UI & Workflows: Build and maintain end-to-end test suites in Playwright for operational UIs and human-in-the-loop approval workflows.
- Backend & Pipeline Coverage: Build API, contract, and integration testing for Python microservices and Kafka event pipelines, including replay and failure-injection scenarios.
- CI/CD Quality Enforcement: Integrate all test suites into CI as enforced quality gates with strict thresholds rather than advisory reports.
- Suite Health & Root-Cause Triage: Monitor execution time, flake rate, and critical path coverage; perform root-cause analysis to permanently fix test flakiness rather than relying on simple retries.
- AI & Agentic Evaluation: Build evaluation suites for agentic AI workflows (validating tool selection, arguments, escalation behavior, and operating envelope limits).
- Non-Deterministic Testing: Define regression testing for non-deterministic components, including golden-set evaluations, tolerance thresholds, and drift detection post-prompt/model changes.
- Edge & Boundary Verification: Automate test suites for edge deployments (offline/reconnection scenarios) and verify human-in-the-loop security boundaries under adversarial conditions.
- Process & Mentorship: Convert operational SOPs and incomplete specs into executable acceptance criteria alongside Technical Business Analysts, and coach engineers on test ownership.
- Experience Level: 5+ years of hands-on experience owning test automation for distributed systems.
- Core Automation Stack: Deep production experience with Playwright (fixtures, parallelization, trace analysis, CI integration) and strong proficiency in both Python and TypeScript.
- Backend & Messaging: Proven experience in API, contract, and integration testing, service virtualization/mocking, and event-driven architectures like Kafka.
- CI/CD & Infrastructure: Practical experience setting up CI quality gates in GitHub Actions or Azure DevOps, with basic Kubernetes literacy for containerized testing environments.
- AI & Problem-Solving: Hands-on experience with AI-assisted testing methodologies, critical judgment of AI-generated code/tests, and strong root-cause debugging skills.
- English Proficiency: Fluent English for daily written and verbal communication.
- AI Agents & Automation: Experience with Playwright MCP, AI coding agents running non-interactively in CI, or the Claude Agent SDK.
- LLM Product Testing: Prior experience testing LLM or agent-based products in production, non-deterministic systems, or building MCP servers for test observability.
- Resilience & Security Testing: Experience with performance/load testing for high-volume telemetry, chaos/fault-injection, or security testing in CI (dependency scanning/dynamic analysis).
- Domain Knowledge: Background in energy systems, smart grid technology, industrial operations, or data center environments.
The role requires collaboration with teams aligned to Central Time (CT). Flexibility to overlap with North America business hours is expected.
💬 LanguageAll interviews, documentation, and daily communication will be in English.
#LI
Skills Required
- 5+ years of hands-on experience owning test automation for distributed systems
- Deep production experience with Playwright, including fixtures, parallelization, trace analysis, and CI integration
- Strong proficiency in Python and TypeScript
- Experience with API, contract, and integration testing
- Experience with service virtualization or mocking
- Experience with event-driven architectures such as Kafka
- Experience setting up CI quality gates using GitHub Actions or Azure DevOps
- Basic Kubernetes literacy for containerized testing environments
- Hands-on experience with AI-assisted testing methodologies
- Strong root-cause debugging skills and ability to critically evaluate AI-generated code and tests
- Fluent English for daily written and verbal communication
- Availability to align with Central Time within approximately two hours and overlap with North American business hours
- Experience with Playwright MCP, AI coding agents in CI, or the Claude Agent SDK
- Production experience testing LLM or agent-based products, nondeterministic systems, or MCP servers for test observability
- Experience with performance or load testing, chaos or fault injection, or security testing in CI
- Background in energy systems, smart grids, industrial operations, or data centers
What We Do
In All Media is a cutting-edge technology and innovation consulting organization, facilitating digital transformation in multiple industries. We have partnered with top-tier organizations such as NBCUniversal, Sony DADC, Aclara Technologies, and Dun & Bradstreet, among other leading companies worldwide. In All Media harnesses the collaborative synergy of all our international talent pool using Agile methodologies, to deliver excellence efficiently and with the utmost quality. At In All Media, we believe in diversity and autonomy. Our new model brings together a global managed community, with experts guiding and managing talented engineers; working to bring together the best talent in the world, regardless of where they are. We are not only evolving in terms of technology, but also in terms of In All Media as a re-imagined technology consultancy in step with the Fourth Industrial Revolution. Our community of 200+ experts and engineers are very motivated by different challenges in their field, and apply continuous learning to stay ahead of the curve in their expertise on new developments and innovative technologies. Our delivery model is based on Agile, On-site, and Near Shore capabilities to maximize team integration and provide the best value to our clients. We partner holistically with our clients in the co-creation of services and products, and follow streamlined Agile processes throughout. This makes In All Media a truly global company driven by a community of dedicated, talented individuals self-organizing into multi-disciplinary teams, producing the best possible output for our world-class clients. We're hiring! Email us your CV: [email protected] Contact Us: [email protected]
