Principal Manager - AI Quality Engineering

Marketing Statement:

Tokio Marine North America Services (TMNAS) provides professional support services to Tokio Marine Group companies in the United States, including Tokio Marine America, First Insurance Company of Hawaii, and Philadelphia Insurance Companies. Customer service is our focus and our passion! Tokio Marine North America Services adds value in all that we do, providing the best professional service to our business partners, allowing them to achieve their goals. Our talented team delivers actuarial, audit, corporate communications, financial and investment, human capital services, information technology, legal, and payroll expertise to our customers.

Job Summary:

In support of the VP of Quality Assurance, this role works closely with IT and business leaders to formulate and implement comprehensive QA Automation strategies and policies. The Principal Manager, Quality Assurance provides strategic leadership and hands-on technical expertise in Automation and Performance engineering. This pivotal role is responsible for building and advancing intelligent, Java/Python-based automation frameworks and performance testing solutions while driving AI-augmented testing practices. The role combines strong leadership, strategic planning, technical oversight, and active hands-on contribution to drive innovation in quality engineering practices across the organization.

Performs essential Quality Engineering functions as applicable to the specific position, including:

Automation Strategy & Execution

• Heads the Automation Engineering and Performance Engineering functions for the QA organization, providing strategic direction, technical leadership, and hands-on guidance to build scalable, efficient, and intelligent automation frameworks.
• Owns and drives the Enterprise business process automation coverage target across critical applications, establishing baseline metrics, gap analysis, and phased delivery roadmaps.
• Architects and maintains end-to-end process automation that spans multiple integrated systems (policy administration, claims, billing, document management, data reporting) validating complete business transactions.

AI/ML-Driven Testing Innovation

• Drives innovation in test automation by actively evaluating, piloting, and implementing AI/ML-powered testing capabilities including intelligent test case generation, self-healing test scripts, AI-assisted defect prediction, and automated root cause analysis.
• Evaluates and integrates AI coding assistants (GitHub Copilot, Claude Code, Cursor) into QA development workflows to accelerate script development, code review, framework and process modernization.
• Designs and implements AI/ML model testing frameworks covering data quality validation, model performance testing (accuracy, precision), bias and fairness testing, and production drift monitoring — critical for insurance industry AI applications in insurance domain applications.
• Leverages Large Language Models (LLMs) and Generative AI for test case generation from requirements, synthetic test data creation, automated test documentation, and intelligent test prioritization.
• Implements Retrieval-Augmented Generation (RAG) solutions for building QA knowledge bases and test asset repositories.

Hands-On Technical Contribution

• Serves as a hands-on technical contributor in Python/Java-based automation development. Leverages strong proficiency in Python along with key automation packages including Selenium WebDriver, Playwright, pytest, RestAssured, Appium, Pandas, NumPy, and AI/ML libraries (scikit-learn, TensorFlow/PyTorch, LangChain, SHAP, Fairlearn) to develop scalable automation solutions.
• Leads the migration of legacy automation scripts (UFT/QTP) to modern Automation frameworks supporting ReactJS, Unqork and modern web application architectures.
• Designs and implements Page Object Model (POM) frameworks, data-driven testing architectures, and API testing solutions (RestAssured, Postman) for comprehensive multi-layer validation (UI, API, database, integration).
• Establishes CI/CD pipeline integration for automated test execution using Azure DevOps, Jenkins, or GitHub Actions with parallel execution capabilities.
• Implements test data management strategies including synthetic data generation, data masking, and environment provisioning.

Quality Strategy & Process

• Formulates and implements QA strategies and policies to ensure product quality and compliance with industry standards and regulations applicable to P&C insurance.
• Supervises the development and execution of test plans, including manual and automated testing, to identify defects and ensure product quality.
• Works closely with Application Development, Engineering, Data Operations, and Customer Service to ensure quality standards are integrated throughout the product lifecycle
• Identifies potential risks related to product quality, develops mitigation strategies, and ensures these are communicated and managed effectively.
• Drives continuous improvement initiatives within the QA department, leveraging industry best practices, emerging technologies, and data-driven methodologies to enhance quality and efficiency.
• Establishes AI/ML governance frameworks for model validation, explainability, and regulatory compliance in collaboration with Risk, Compliance, and Actuarial teams.
• Ensures conversion of business requirements into verifiable use cases/test cases and maintenance tasks.
• Provides guidance for the planning and execution of User Acceptance Testing (UAT).
• Ensures that products meet all relevant regulatory and industry standards.

Leadership & Team Development

• Leads and mentors a team of QA engineers, fostering a culture of continuous learning, innovation, and technical excellence.
• Designs and executes team upskilling programs for AI/ML testing, modern automation frameworks, and cloud-native testing practices.
• Builds high-performance teams through effective training, mentoring, delegating, and providing constructive feedback.
• Manages vendor relationships for QA tools, AI testing platforms, and contractor augmentation.

Other Functions

• Complies with proper internal controls as necessary to conduct job functions and/or carry out responsibilities and/or administrative activities.
• Performs special duties and other projects as may be assigned.
• Establishes and builds strong working relationships and partnerships with IT domain units and senior management.
• Leads change initiatives and operations.
• Sets Service Level Agreements, ensures SLAs are met; when SLAs are not met, leads improvement efforts.
• Serves as focal point of communication for major service outages/escalations.

Required

• 12+ years of progressive Dev/QA experience with a proven track record of leadership and management of teams for at least 5 years.
• 5+ years of hands-on experience building and maintaining automation frameworks in Python and/or Java.
• Strong proficiency in Selenium WebDriver, pytest/TestNG, and at least one API testing tool (RestAssured, Postman, ReadyAPI).
• Experience with CI/CD tools (Azure DevOps, Jenkins, GitHub Actions) and version control (Git).
• Demonstrated experience leading large-scale automation initiatives with measurable coverage and ROI outcomes.
• Working knowledge of AI/ML concepts including supervised/unsupervised learning, model evaluation metrics, data quality, and bias detection.
• Experience using AI coding assistants (GitHub Copilot, Claude Code, or equivalent) for development productivity.
• Experience with data analysis libraries (Pandas, NumPy) and SQL for test data management and validation.
• Excellent communication skills with ability to present technical strategies to executive leadership.

Preferred

• Bachelor’s Degree in Computer Science, Engineering, Information Technology, or related field.
• 3+ years of experience working in P&C insurance or financial services.
• Experience with AI/ML testing frameworks and tools (Great Expectations, Evidently AI, Fairlearn, SHAP).
• Experience with LLM/GenAI applications (prompt engineering, LangChain, RAG architectures).
• Experience with cloud platforms (Azure, AWS, GCP) and containerization (Docker).
• Experience with performance testing tools (JMeter, Gatling, LoadRunner).
• Experience migrating legacy automation (UFT/QTP) to modern frameworks.
• Familiarity with MLOps practices (MLflow, model monitoring, data drift detection).
• Understanding of P&C insurance business processes (policy lifecycle, billing, claims, underwriting).

Certifications (Preferred)

• AWS Certified Machine Learning — Specialty
• Microsoft Certified: Azure AI Engineer Associate
• TensorFlow Developer Certificate (Similar hands-on coding certification)
• (Any) Industry recognized AI certification

Salary range $130,000 to $180,000. Ultimate salary offered will be based on factors such as applicant experience and geographic location. Our company offers a competitive benefits package and bonus eligibility on top of base.

EEO Statement:

Tokio Marine Group of Companies (including, but not limited to the Philadelphia Insurance Companies, Tokio Marine America, Inc., TMNA Services, LLC, TM Claims Service, Inc. and First Insurance Company of Hawaii, Ltd.) is an Equal Opportunity Employer. In order to remain competitive we must attract, develop, motivate, and retain the most qualified employees regardless of age, color, race, religion, gender, disability, national or ethnic origin, family circumstances, life experiences, marital status, military status,  sexual orientation and/or any other status protected by law.