Senior Principal Engineer - IT Quality & Assurance Testing

Posted 3 Days Ago
Be an Early Applicant
Building End, Uttlesford, Essex, England, GBR
In-Office
Senior level
Professional Services • Consulting
The Role
Leads quality engineering, automation, manual testing, and AI/LLM testing across web, API, database, RAG, and AI-powered applications. Defines testing strategies, automation frameworks, AI evaluation practices, quality metrics, and CI/CD integration. Manages QA teams, mentors engineers, mitigates delivery risks, drives continuous improvement, and collaborates with Product, Engineering, AI/ML, Data Science, Security, and business stakeholders.
Summary Generated by Built In
Company:Marsh

Description:

We are seeking a talented individual to join our MMB LAC L3 Squad team at Marsh People and Investments, a Marsh business. This role will be based in Pune/Gurugram. This is a hybrid role that has a requirement of working at least three days a week in the office.


Your Experience:

A visionary forward thinker who views Quality holistically and can drive a Quality Engineering mindset across traditional software and AI-powered applications. Understanding industry trends, tools and practices and implementing the right approaches to deliver high-quality, scalable and reliable software.


  • 10+ years' experience in Software Testing / Quality Engineering with experience in managing end-to-end project delivery.
  • Proven experience leading QA teams across Manual, Automation and AI/LLM testing.
  • Strong understanding of STLC, SDLC, Agile, test strategy, test planning, risk management, defect management and quality metrics.
  • Experience defining testing strategies for Web/UI, API, Database, AI/LLM, RAG and AI-powered applications.
  • Strong automation experience using Python, JavaScript/TypeScript, Java, Playwright, Selenium and WebdriverIO (WDIO).
  • Good understanding of API testing, SQL, relational databases and data validation.
  • Understanding of RAG architecture, embeddings, vector databases, semantic search, retrieval and reranking.
  • Experience with Generative AI, LLM evaluation, prompt engineering, AI agents and AI safety testing.
  • Experience with AI/LLM testing and evaluation frameworks such as PromptFoo, Ragas, DeepEval, LangSmith and PyRIT, or similar tools.
  • Strong understanding of AI quality and security risks including prompt injection, jailbreaks, hallucination, data leakage, guardrails and responsible AI.
  • Experience with modern DevOps and test automation practices, including Git, GitHub, GitHub Actions and CI/CD pipeline integration.
  • Strong communication, stakeholder management, analytical and people leadership skills.

Things you will do:

  • Input into planning and subsequent resource planning – including skills profiling and identifying training requirements for team members.
  • Define and drive the Quality Engineering, Automation and AI Testing strategy across projects.
  • Lead testing across UI, API, Database, Automation and AI/LLM applications.
  • Establish AI testing and evaluation practices covering accuracy, relevance, faithfulness, groundedness, hallucination, safety and security.
  • Drive testing of RAG applications, vector databases, retrieval pipelines, prompts and AI agent workflows.
  • Evaluate and adopt appropriate AI testing, evaluation, observability and security tools based on project requirements.
  • Lead automation initiatives using Playwright, WebdriverIO, JavaScript/TypeScript and Java, integrated with CI/CD and GitHub Actions.
  • Establish scalable and maintainable automation frameworks, reusable utilities and test assets.
  • Define and monitor test coverage, automation coverage, AI evaluation metrics, defect leakage and release quality.
  • Drive shift-left testing, continuous testing, risk-based testing and quality engineering practices.
  • Ensure effective root cause analysis, defect prevention and continuous quality improvement.
  • Mentor and develop QA engineers across manual testing, automation and AI testing.
  • Work closely with Product, Engineering, AI/ML, Data Science, Security and Business teams.
  • Manage team performance, delivery risks, conflicts and stakeholder expectations.
  • Ensure consistent testing standards, processes and quality practices across teams.

What you will bring:

  • Strong understanding of Quality Engineering, Manual Testing, Automation and AI/LLM Testing.
  • Strong experience with Playwright, WebdriverIO/Selenium, Python, JavaScript/TypeScript and Java.
  • Good knowledge of UI, API, SQL/Database and data validation testing.
  • Understanding of RAG, embeddings, vector databases, semantic search, retrieval and LLM evaluation.
  • Experience with AI/LLM evaluation and testing tools such as PromptFoo, Ragas, DeepEval, LangSmith and PyRIT.
  • Understanding of AI security, prompt injection, jailbreaks, hallucination, guardrails, red-team testing and responsible AI.
  • Experience with GitHub and CI/CD pipelines.
  • Experience building and maintaining scalable automation frameworks.
  • Proven experience managing, mentoring and developing QA/Quality Engineering teams.
  • Strong communication, stakeholder management, analytical and problem-solving skills.
  • Ability to evaluate emerging technologies and introduce effective testing solutions.
  • Passion for AI, automation, Quality Engineering and continuous improvement.

 

Values and Culture:

Our team comprises all sorts of people from all sorts of backgrounds. We believe everyone brings value and a unique perspective and want to promote a fun and diverse culture. To help us succeed we hope that you can demonstrate:

  • A genuine passion for technology
  • A clear commitment to creating brilliant products that will give our clients the edge
  • A willingness to work fluidly and respectfully with our talented team

Marsh (NYSE: MRSH) is a global leader in risk, reinsurance and capital, people and investments, and management consulting, advising clients in 130 countries. With annual revenue of over $27 billion and more than 95,000 colleagues, Marsh helps build the confidence to thrive through the power of perspective. For more information, visit marsh.com, or follow us on LinkedIn and X.

Marsh is committed to providing an inclusive work environment. We aim to attract and retain the best, most qualified people and value colleagues of all backgrounds, without regard to age, background, caste, disability, ethnic origin, family duties, gender orientation or expression, gender reassignment, marital status, nationality, parental status, personal or social status, political affiliation, race, religion and beliefs, sex/gender, sexual orientation or expression, skin color, or any other characteristic protected by applicable law.

Marsh is committed to hybrid work, which includes the flexibility of working remotely and the collaboration, connections and professional development benefits of working together in the office. All Marsh colleagues are expected to be in their local office or working onsite with clients at least three days per week. Office-based teams will identify at least one “anchor day” per week on which their full team will be together in person.

Skills Required

  • 10+ years of experience in software testing or quality engineering, including end-to-end project delivery
  • Experience leading QA teams across manual testing, automation, and AI/LLM testing
  • Strong understanding of STLC, SDLC, Agile, test strategy, test planning, risk management, defect management, and quality metrics
  • Experience defining testing strategies for web/UI, API, database, AI/LLM, RAG, and AI-powered applications
  • Strong automation experience with Python, JavaScript or TypeScript, Java, Playwright, Selenium, and WebdriverIO
  • Knowledge of API testing, SQL, relational databases, and data validation
  • Understanding of RAG architecture, embeddings, vector databases, semantic search, retrieval, and reranking
  • Experience with generative AI, LLM evaluation, prompt engineering, AI agents, and AI safety testing
  • Experience with PromptFoo, Ragas, DeepEval, LangSmith, PyRIT, or similar AI testing and evaluation tools
  • Understanding of AI security risks including prompt injection, jailbreaks, hallucination, data leakage, guardrails, red-team testing, and responsible AI
  • Experience with Git, GitHub, GitHub Actions, CI/CD pipelines, and scalable automation frameworks
  • Experience managing, mentoring, and developing QA or quality engineering teams
  • Strong communication, stakeholder management, analytical, problem-solving, and people leadership skills
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: New York, New York
9,026 Employees

What We Do

Oliver Wyman is a global leader in management consulting. With offices in more than 70 cities across 30 countries, Oliver Wyman combines deep industry knowledge with specialized expertise in strategy, operations, risk management, and organization transformation. The firm has more than 7,000 professionals around the world who work with clients to optimize their business, improve their operations and risk profile, and accelerate their organizational performance to seize the most attractive opportunities. Oliver Wyman is a business of Marsh McLennan [NYSE: MMC].

Similar Jobs

Shield  Logo Shield

Senior Manager, UK Security (R6499)

Aerospace • Artificial Intelligence • Machine Learning • Robotics • Software • Defense Technology
In-Office
London, Greater London, England, GBR

Shield  Logo Shield

People Experience Ops Partner (R6561)

Aerospace • Artificial Intelligence • Machine Learning • Robotics • Software • Defense Technology
In-Office
London, Greater London, England, GBR

Shield  Logo Shield

Systems Administrator

Aerospace • Artificial Intelligence • Machine Learning • Robotics • Software • Defense Technology
In-Office
London, Greater London, England, GBR

Shield  Logo Shield

Communications Manager - Europe (6635)

Aerospace • Artificial Intelligence • Machine Learning • Robotics • Software • Defense Technology
Hybrid
London, Greater London, England, GBR

Similar Companies Hiring

Cravath, Swaine & Moore LLP Thumbnail
Legal Tech • Other • Professional Services
New York, NY
1200 Employees
VCA Animal Hospitals Thumbnail
Pet • Professional Services
Los Angeles, CA
35000 Employees
RapidScale Thumbnail
Cloud • Information Technology • Consulting
Raleigh, NC
600 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account