What You’ll Be Doing
1.Quality Engineering Leadership Across the Platform
- Represent and elevate the Central QE community, partnering closely with product and engineering teams to embed quality at every stage of development.
- Contribute to a modern Quality strategy for a fast-scaling AI First SaaS architecture.
- Improve CI/CD workflows with better test isolation, data setup, and execution reliability.
2. AI-Focused Test Strategy & Execution
- Develop primarily deterministic and contribute into non-deterministic testing strategies for LLM-based agentic systems.
- Build automation to validate the offline and online evaluation pipelines for correctness, safety, drift, and reliability.
- Design and implement test harnesses for agent endpoints, tool-calling flows, and planning sequences.
- Build automation to validate AI artifacts including prompt libraries, golden datasets, rubric scoring, eval outputs and model versions.
- Validate RAG pipelines end-to-end and integrate AI-specific metrics into CI/CD.
- Build safety, compliance, and guardrail validation mechanisms.
3. Functional & Cross-Team Testing
- Partner across teams to ensure timely, high-quality delivery of roadmap commitments.
4. Quality Metrics, Telemetry & Observability
- Reporting discipline for DORA metrics, test coverage, test effectiveness, and health execution.
- Extend metrics with AI-specific quality indicators.
- Expand observability for AI systems including degradation patterns and anomalous outputs.
5. Culture, Leadership & Collaboration
- Promote a quality-first mindset across Engineering.
- Lead cross-functional initiatives to ensure quality influences design, development, deployment, and incident reviews.
What You Will Bring
- 5+ years experience in Quality Engineering within cloud-native SaaS organizations.
- Expertise in automated testing infrastructure and CI/CD pipelines.
- Strong understanding of testing strategies across the pyramid.
- Solid frontend and full-stack testing collaboration skills.
- Familiarity with observability tooling.
- Experience testing LLMs, AI agents, RAG pipelines.
- Knowledge of AI safety risks including hallucinations and prompt injections.
- Bachelor's/Master's in Computer Science, AI, or related; certifications- Strong communication, leadership, and coaching skills. C1 english.
Perks & Benefits
- Contrato a termino Indefinido with all the legal benefits
- Prepaid Medicine
- Life insurance and funeral assistance
- Internet allowance
- Home office stipend
- Competitive compensation — above the market average
- 100% remote work environment and an excellent work-life balance
- Opportunity to work for a growing global SaaS leader company
- A culture that promotes independence, innovation, trust, and accountability
- Open space to be creative, innovative, and strategize for the future
- Mentorship by a highly experienced professional
- Budget for training, we want you to grow
- 5 Personal Time Off days per year
- Sick Leave Top up to total 100% of salary paid by the employer from Day 3 to 90.
- Recognition Award, additional paid time off in recognition of the corresponding year of service
- Upgrade vacation starting at 5 years of service
- AI-first environment: Be part of an AI-first engineering organization that embraces modern tools, automation, and AI-driven ways of working.
Skills Required
- 5+ years experience in Quality Engineering within cloud-native SaaS organizations
- Expertise in automated testing infrastructure and CI/CD pipelines
- Strong understanding of testing strategies across the pyramid
- Solid frontend and full-stack testing collaboration skills
- Familiarity with observability tooling
- Experience testing LLMs, AI agents, RAG pipelines
- Knowledge of AI safety risks including hallucinations and prompt injections
- Strong communication, leadership, and coaching skills
- Bachelor's/Master's in Computer Science, AI, or related
What We Do
Caseware is the leading global provider of cloud-enabled audit, financial reporting and data analytics solutions for accounting firms, corporations and government regulators. Caseware’s innovative tools and platforms help more than half a million customers in 130 countries work smarter, dig deeper and see further as they transform insights into impact.
