Module Lead, Data Science (LLM-as-a-judge)

BOLD is seeking a senior professional to implement and run Testing and Quality Assurance specifically designed for AI-driven products and Data Science workflows. In this role, you will bridge the gap between traditional software testing and modern machine learning evaluation. You have a great team management and organizational skills. You’ll have a broad mastery of capabilities, techniques, technologies and tools in the Testing and QA domains. You’ll be handling QA teams of various sizes and expertise.

ABOUT THIS TEAM

The Data Science department at BOLD is responsible for discovering patterns and trends in datasets to get insights, creating predictive algorithms and data models, improving the quality of data or product offerings by utilizing machine learning techniques, distributing suggestions to other teams and top management, and using data tools such as Python and SQL.

The Data Science team actively collaborates with other vertical teams such as Engineering, Portals, BI, Product, Legal. Most of the projects are focused around problems that require a mix of natural language processing and machine learning. Some of the active projects are resume parsing, ranking, summary generation, data quality and scoring, content generation, job recommendations, and conversion analysis. Apart from the business initiatives, the team also explores state of the art methods and keeps them up to to date with technology.

WHAT YOU’LL DO 

• Develop and own the QA strategy for AI-driven features, focusing on model accuracy, data drift, and performance metrics (Precision, Recall, F1-score).
• Implement state-of-the-art AI tools (e.g., GitHub Copilot, or custom LLM agents) to automate test case generation, code analysis, and synthetic data creation.
• Oversee the testing of Generative AI outputs, implementing "LLM-as-a-judge" frameworks to evaluate hallucination rates, toxicity, and prompt adherence.
• Develop and own BOLD’s quality framework from product development through to production
• Should be able to drive the initiatives (both Business and QA) in his area along with needed smooth coordination across the board to ensure a timely achievement of goal.
• Take part in Requirement Analysis, Sprint planning, Bug triage meetings.
• Liaise with product, development, and infrastructure teams to collaborate on delivery of high quality, high performance, and high availability applications
• Develop quality scripts with needed coverage to ensure defect free releases.
• Manage and mentor team members to maximize output and efficiency and promote a team culture of quality and fun
• Motivate, Encourage, Guide Teams maintaining equilibrium and satisfaction level.
• Give technical/logical solutions to problems faced by the teams and remove hurdles blocking releases.
• Evaluate and suggest new tools and technologies

WHAT YOU’LL NEED

• Solid understanding of the Machine Learning lifecycle (EDA, Training, Evaluation, Deployment).
• Experience using LLMs (GPT, Claude, Llama) and AI-coding assistants to accelerate the testing lifecycle.
• Experience in a high volume, high throughput web/ecommerce environment.
• Hands-on experience with UI/API automation (Selenium, Playwright, or Cypress) integrated with AI-driven healing capabilities.
• Experience with web service technology like XML, REST, SOAP
• Knowledge on any of the query language like SQL, JQL etc.
• Experience in testing complex web and mobile apps.
• Strong application architecture and technical knowledge
• Strong knowledge of web concepts, including web performance infrastructure.
• Able to play the role of a scrum master, with strong coordination skills
• Knowhow of basic UX standards, SEO/ SEM concepts and key tracking matrices
• Strong communication skills 

EXPERIENCE- 

• Module Lead, QA (AI/ML & Automation)- 5 years+