Job Description:
Serve as the technical authority for AI quality, evaluation, and risk controls within the Commercial AI Center of Excellence (CAI CoE).
Define and operationalize technical mechanisms that ensure Vertex AI systems meet tax-grade standards for accuracy, defensibility, and compliance.
Design and implement AI evaluation frameworks, including gold datasets, regression testing, and CI gates; define metrics across accuracy, confidence calibration, citation coverage, and abstention correctness.
Lead model validation, error analysis, telemetry, and drift detection strategies; establish best practices for GenAI and ML hybrid systems.
Partner with Legal, Security, and Risk teams to translate policy into enforceable technical mechanisms; review high-risk AI use cases and guide product teams on mitigation strategies.
Influence standards and practices across large engineering organizations while partnering with product teams to improve model performance, reliability, and adoption.
Required Qualifications
15+ years of experience in AI/ML engineering, with demonstrated impact at Senior Principal or equivalent level.
Deep hands-on experience designing and implementing AI/ML evaluation frameworks in production.
Expertise in model validation, drift detection, telemetry, and automated testing pipelines.
Strong understanding of AI governance, auditability, and risk controls.
Experience building evaluation harnesses, gold datasets, and CI-integrated regression testing.
Proven ability to partner cross-functionally with Product, Legal, Security, and Risk teams.
Strong system-level thinking in regulated SaaS environments.
Preferred Qualifications
Bachelor's degree in Computer Science, Engineering, or a related discipline; advanced degree preferred.
Industry experience in tax or other highly regulated domains (insurance, fintech, or healthcare).
Experience with vector databases and retrieval optimization.
FinOps for GenAI: experience modeling and optimizing LLM token and inference costs.
Data science or classical AI/ML background (beyond prompt engineering).
Strong executive communication and technical storytelling skills.
Equivalent combination of education, training, and relevant professional experience accepted in lieu of a formal degree.
Disclaimer:
The above statements describe the general nature and level of work performed in this role. Other duties may be assigned.
Other Qualifications:
The Winning Way behaviors that all Vertex employees need in order to meet the expectations of each other, our customers, and our partners.
• Communicate with Clarity - Be clear, concise and actionable. Be relentlessly constructive. Seek and provide meaningful feedback.
• Act with Urgency - Adopt an agile mentality - frequent iterations, improved speed, resilience. 80/20 rule – better is the enemy of done. Don’t spend hours when minutes are enough.
• Work with Purpose - Exhibit a “We Can” mindset. Results outweigh effort. Everyone understands how their role contributes. Set aside personal objectives for team results.
• Drive to Decision - Cut the swirl with defined deadlines and decision points. Be clear on individual accountability and decision authority. Guided by a commitment to and accountability for customer outcomes.
• Own the Outcome - Defined milestones, commitments and intended results. Assess your work in context, if you’re unsure, ask. Demonstrate unwavering support for decisions.
Comments:
The above statements are intended to describe the general nature and level of work being performed by individuals in this position. Other functions may be assigned, and management retains the right to add or change the duties at any time.
Pay Transparency Statement:
Base pay offered to new hires may vary based upon factors including relevant industry and job-related skills and experience, geographic location, and business needs.* The range displayed does not encompass the full potential of the role, which allows for further growth and career progression.
In addition, as a part of our total compensation package, this role may be eligible for the Vertex Bonus Plan (VOB), a role-specific sales commission/bonus, and/or equity grants.
Learn more about Life at Vertex and connect with your recruiter for more details regarding Vertex's compensation and benefit programs.
*In no case will your pay fall below applicable local minimum wage requirements.
Skills Required
- 15+ years of experience in AI/ML engineering
- Deep hands-on experience designing AI/ML evaluation frameworks
- Expertise in model validation, drift detection, telemetry, and automated testing pipelines
- Strong understanding of AI governance, auditability, and risk controls
- Experience building evaluation harnesses and gold datasets
- Proven ability to partner cross-functionally with various teams
- Strong system-level thinking in regulated SaaS environments
- Bachelor's degree in a related discipline, advanced degree preferred
- Industry experience in tax or highly regulated domains
- Experience with vector databases and retrieval optimization
- Experience modeling and optimizing LLM token and inference costs
- Strong executive communication and technical storytelling skills
Vertex, Inc. Compensation & Benefits Highlights
The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Vertex, Inc. and has not been reviewed or approved by Vertex, Inc..
-
Leave & Time Off Breadth — Flexible time off, including unlimited vacation for some roles, paid volunteer time, bereavement leave, sabbaticals, and observed holidays is emphasized. Occasional company shutdowns further expand time-away options.
-
Parental & Family Support — Benefits span paid parental leave, adoption assistance, childcare support, fertility benefits, and structured return-to-work programs. Backup care and daycare tuition discounts reinforce family-focused support.
-
Retirement Support — A 401(k) with employer matching and immediate vesting anchors long-term savings. Profit sharing and defined contribution programs complement financial security offerings.
Vertex, Inc. Insights
What We Do
The Trusted Leader in Tax Technology. We're Vertex (VERX). A pioneer in tax automation for more than 45 years. We proudly serve over 4,000 customers worldwide with distinction and provide comprehensive tax solutions that enable global businesses to transact, comply, and grow with confidence. Our collaborative, friendly culture is the foundation of everything we do and who we are. We're proud to be an employer guided by a common purpose by building trusted relationships at work, in business and in our communities. We strive to be a values-driven employer of choice who attracts, retains, and inspires talented professionals to achieve their full potential. We employ over 1,500 full-time professionals across three continents: US, Europe, and Brazil. Our software, content, and services address the increasing complexities of global commerce and compliance by reducing friction, enhancing transparency and enabling greater confidence in meeting indirect tax obligations. As a result, our software is ubiquitous within our customers’ business systems, touching nearly every line item of every transaction that an enterprise can conduct. Our software is fueled by over 300 million data-driven effective tax rules and supports indirect tax compliance in more than 19,000 jurisdictions worldwide. We partner with the world’s most respected companies and harness their strengths to deliver the best tax technology solution to businesses across the globe. We integrate with key technology partners that span ERP, CRM, procurement, billing, POS and e-commerce platforms. We also work closely with over 50 tax, accounting and consulting firms to provide the integrated tax technology solutions.
Why Work With Us
Innovative, agile, and growth-minded professionals thrive at Vertex. Our remote-first culture offers flexibility and connection, while our focus on indirect tax technology and global expansion creates real impact. With leadership development and a strong emphasis on well-being, careers flourish here.
Gallery
.jpg){kind=logo}
