About InvoiceCloud:
InvoiceCloud is a fast-growing fintech leader recognized with 20 major awards in 2025, including USA TODAY and Boston Globe Top Workplaces, multiple SaaS Awards wins for Best Solution for Finance and FinTech, and national customer service honors from Stevie and the Business Intelligence Group. Judges also highlighted our mission to reduce digital exclusion and restore simplicity and dignity to how people pay for essential services, as well as our leadership in AI maturity and responsible innovation. It’s an award-winning, purpose-driven environment where top talent thrives. To learn more, visit InvoiceCloud.com.
- Owns end-to-end AI quality strategy for LLM, RAG, and AI-powered platform features.
- Establishes disciplined validation routines to ensure production readiness and release confidence.
- Ensures compliance with data governance, privacy, security, and responsible AI standards.
- Drives CI/CD quality automation, issue triage, drift detection, and resolution.
- Maintains accountability for AI correctness, observability, and long-term test sustainability.
- Builds automated regression suites, benchmarking tools, and evaluation workflows for AI systems.
- Defines and tracks quality KPIs related to accuracy, performance, latency, safety, and reliability.
- Implements continuous validation and monitoring to reduce manual testing and release friction.
- Partners with engineering teams to improve testability, telemetry, and quality insights across pipelines.
- Uses automation and data-driven signals to accelerate feedback loops and reduce time-to-value.
- Designs and implements automation for prompt testing, semantic scoring, and safety checks.
- Uses modern AI frameworks (Azure OpenAI, LangChain, Semantic Kernel, vector databases) to create reusable quality tooling.
- Experiments with new approaches to AI validation, observability, and model evaluation.
- Applies emerging AI quality techniques to strengthen platform scalability and reliability.
- Leverages AI-assisted tools to enhance test coverage, insight generation, and development speed.
- Validates AI behaviors that directly improve payer, agent, and customer service experiences.
- Measures business impact through accuracy, call deflection, reduced handle time, and outcome quality.
- Advocates for safe, transparent, and trustworthy AI behaviors across the platform.
- Partners with product teams to ensure AI quality aligns with real user needs and expectations.
- Ensures AI systems behave predictably and responsibly in customer-facing scenarios.
- Experience in test automation or SDET roles within SaaS or distributed systems
- Hands-on experience validating AI/ML, NLP, or analytics-driven applications
- Strong proficiency in Python and CI/CD automation practices
- Familiarity with Azure OpenAI, LangChain, Semantic Kernel, or vector search technologies
- Knowledge of responsible AI practices, privacy, RBAC, and security testing
- Ability to define quality metrics, automate validation, and support production monitoring
- Strong analytical, communication, and cross-team collaboration skills
- Health & Wellness Coverage: Comprehensive medical insurance for employees and their families, including spouse, children, and parents.
- Leave & Time Off: Generous paid time off (PTO), and public holidays, along with flexibility for personal and family needs.
- Parental Benefits: Paid maternity and paternity leave, along with additional family support programs for new parents.
- Recognition & Rewards: Participation in global and local recognition programs that celebrate contributions and milestones.
- Employee Engagement & Culture: Access to Employee Resource Groups (ERGs), team events, and volunteering initiatives that promote inclusion and community.
InvoiceCloud is committed to providing equal employment opportunities to all employees and applicants. We do not tolerate discrimination or harassment of any kind based on race, color, religion, age, sex, nationality, disability, genetic information, veteran or military status, sexual orientation, gender identity or expression, or any other characteristic protected under applicable laws.
This commitment applies to all aspects of employment, including recruitment, hiring, placement, promotion, termination, layoff, recall, transfer, leave, compensation, and training.
If you require a disability-related or religious accommodation during the application or recruitment process, and wish to discuss possible adjustments, please contact [email protected].
Click here to review InvoiceCloud’s Job Applicant Privacy Policy.
For recruitment agencies: InvoiceCloud does not accept unsolicited resumes from agencies. Please do not forward resumes to our job aliases, employees, or any other company location. InvoiceCloud is not responsible for any fees associated with unsolicited submissions.
Similar Jobs
.jpeg)
What We Do
InvoiceCloud provides a complete, simple, and secure electronic bill presentment and payment solution. Our SaaS platform provides flexible and always-up-to-date online payment solutions that can be configured to meet the unique needs of your organization. And our simple-to-use interface engages customers throughout the payment process to deliver your highest ever e-payment adoption rates.
