Convo is looking for QA Automation Engineers with 5+ years of experience in test automation to build robust automated testing frameworks for APIs, agent workflows, tool calls, and critical UI experiences. The ideal candidate should have strong expertise in Python, pytest, API testing, Playwright, CI/CD, contract testing, and asynchronous workflows, with experience creating reusable test fixtures, mocks, and simulators to ensure reliable and scalable product quality.
Technical missionAutomate regression testing for agent tool calls, planner outputs, APIs and the operator-facing agentic console.
Key responsibilitiesCreate API and workflow regression suites for orchestrator, planner and tool-registry behavior.
Build deterministic assertions around schemas, tool-call sequences, constraints and expected side effects.
Implement pytest-based service tests and Playwright tests for relevant console workflows.
Create mocks, simulators and fixtures for tools, model responses and external dependencies.
Integrate suites into CI, publish actionable results and manage flaky-test remediation.
Maintain traceability from technical requirements and defects to automated coverage.
Minimum experience: 4+ years in test automation, including 2+ years testing APIs, asynchronous workflows or distributed services.
Strong Python, pytest and API-testing experience.
Playwright or comparable browser automation for critical UI workflows.
CI pipeline integration and test-result reporting.
Contract/schema validation for REST and event-driven interfaces.
Ability to test asynchronous, non-deterministic and retrying workflows without masking defects.
Experience designing reusable test fixtures, mocks and service simulators.
Agent/LLM workflow evaluation and tool-call trace analysis.
Performance, resilience and fault-injection testing.
OpenTelemetry or distributed-trace-based test diagnostics.
Automated regression suites for planner, tool calls, APIs and critical UI paths.
CI integration with quality thresholds and readable failure evidence.
Reusable mocks, fixtures and simulators.
Coverage and defect-leakage reporting with maintained documentation.
Works with Agent Runtime, DevOps, AI Test & Dataset Engineering and product teams consuming the shared framework.
- Great compensation package, medical benefit for you and your family, free lunch, annual performance-tied increments & performance recognition awards and a great lean and agile work culture!
- Convo endorses a culture of diversity in all aspects and aims to build a diverse team of amazing individuals!
Skills Required
- 4+ years of experience in test automation
- 2+ years testing APIs, asynchronous workflows, or distributed services
- Strong Python experience
- Strong pytest experience
- Strong API-testing experience
- Playwright or comparable browser automation experience
- CI pipeline integration and test-result reporting experience
- Contract and schema validation experience for REST and event-driven interfaces
- Ability to test asynchronous, non-deterministic, and retrying workflows without masking defects
- Experience designing reusable test fixtures, mocks, and service simulators
- Agent or LLM workflow evaluation and tool-call trace analysis
- Performance, resilience, and fault-injection testing experience
- OpenTelemetry or distributed-trace-based test diagnostics experience
What We Do
Convo provides an enterprise social collaboration platform that enables easy, secure conversations between desk / non-desk workers to accelerate company productivity and engagement. Unlike existing email-focused or chat-centric collaboration platforms, only Convo combines the ease of social networks with rich collaboration capabilities to simplify and optimize work interactions for all employees -- no email required.
Gallery
