AI Technical Lead

Posted 26 Days Ago
Be an Early Applicant
Pune, Maharashtra, IND
In-Office
Expert/Leader
Professional Services • Software • Analytics • Business Intelligence
The Role
Leads technical delivery for AI workstreams across regulated client engagements, balancing hands-on engineering with oversight of three to five engineers. Designs and reviews LLM systems, retrieval and agent architectures, extraction pipelines, evaluation gates, security controls, and offline deployments. Mentors engineers, manages design and code reviews, communicates delivery risk to leadership and clients, and builds reusable Center of Excellence assets. Requires production LLM experience, deep Python and system design expertise, cloud depth, evaluation rigor, and technical leadership.
Summary Generated by Built In
The role

We need senior hands now. The AI practice runs several concurrent client platforms in regulated industries, and the engineering team supporting them is early-career. The Technical Lead becomes the day-to-day technical owner of that team: the person who turns architecture decisions into implementable designs, runs the design gate, reviews the code, unblocks the hard problems personally, and tells the Chief Architect and delivery leadership the truth about risk.

Expect the split to be roughly half hands-on engineering and half oversight. The hands-on half is the difficult half: retrieval architecture, agent orchestration, extraction accuracy, performance and cost. The oversight half is where you multiply — three to five engineers with one to two years of experience become independent owners of their workstreams within six months because of how you review, mentor and set standards.

What will you do
  • Own technical delivery of AI workstreams across concurrent client engagements: scope, design, sequencing, quality and honest estimates.

  • Run the pre-implementation design gate: review engineers' design notes, apply the tiered checklist, maintain the decisions register, and approve or send back before code is written.

  • Set and enforce engineering standards for LLM systems — evaluation suites as release gates, prompt and model versioning, cost and latency budgets, guardrails and injection defenses, human approval gates, evidence and audit trails.

  • Personally build the hardest components: retrieval and re-ranking architecture, tool-calling and agent orchestration on Temporal, document extraction pipelines, on-premises and offline deployments.

  • Review code at scale, including code produced with AI coding tools; catch invalidated model output, leaked secrets, missing tests and silent behavioral change before they merge.

  • Mentor early-career engineers with a deliberate plan: pair on design, grade their design notes, expand their ownership as they earn it.

  • Represent engineering with clients: explain trade-offs in plain language, push back on unsafe or unbounded asks, and turn vague requirements into a bounded phase-one scope.

  • Hold the security posture: secrets management via Vault and Key Vault, data residency, least-privilege access, and client confidentiality on on-premises work.

  • Build reusable Centre of Excellence assets — evaluation harness, RAG reference implementation, guardrail library, extraction patterns — so that the method travels across clients even when the code cannot.

  • Assess codebases and delivery risk candidly and communicate upward early, including when the news is unwelcome.

What you must bring
  • Two or more LLM systems that you shipped to production and operated, each with measurable quality gates. You can describe the incidents, what you measured, what you changed, and what you would do differently.

  • Deep Python and strong system design: multi-tenant SaaS, asynchronous pipelines, event- or workflow-driven orchestration, data modelling on document and relational stores.

  • Evaluation rigor as a habit: you have built eval datasets, chosen the metrics, and used them to block a release.

  • Cloud depth on Azure preferred (Azure AI Foundry, Azure OpenAI, AKS or App Service, Key Vault); AWS or GCP equivalents acceptable if the principles transfer.

  • Led a team of three or more engineers through delivery, including design reviews and code reviews you were accountable for.

  • Judgment about where a model does not belong, and the standing to say so to a client who wants one everywhere.

  • Clear written communication: design documents, decision records, risk memos. Comfortable in client-facing technical conversations.

What would set you apart
  • Temporal in production; durable, idempotent, resumable workflow design.

  • Document AI at depth: tiered OCR strategies, layout analysis, extraction accuracy measurement on real-world scanned records.

  • Computer vision for inspection or verification use cases (DETR-family detectors, DINOv2 backbones, ensembling and box-fusion methods) and release gates expressed as per-class recall.

  • Compliance frameworks applied to software delivery: GxP and 21 CFR Part 11, SOC 2, ISO 27001, HIPAA, PCI-DSS.

  • Test automation strategy with AI-assisted authoring (Playwright), and the governance of AI coding tools inside an engineering team.

  • Experience running a small engineering practice or CoE: standards, reusable assets, training, hiring.



Skills Required

  • Two or more LLM systems shipped to production and operated, with measurable quality gates and experience explaining incidents, measurements, changes, and lessons learned.
  • Deep Python expertise and strong system design skills.
  • Experience with multi-tenant SaaS, asynchronous pipelines, event- or workflow-driven orchestration, and document and relational data modeling.
  • Experience building evaluation datasets, selecting metrics, and using evaluations to block releases.
  • Cloud expertise, preferably Azure; AWS or GCP equivalents are acceptable if transferable.
  • Led a team of three or more engineers through delivery, including accountability for design and code reviews.
  • Judgment about appropriate and inappropriate uses of AI models and ability to communicate decisions to clients.
  • Clear written communication, including design documents, decision records, and risk memos.
  • Comfort with client-facing technical conversations.
  • Production experience with Temporal and durable, idempotent, resumable workflow design.
  • Deep Document AI experience, including OCR strategies, layout analysis, and extraction accuracy measurement on scanned records.
  • Computer vision experience with DETR-family detectors, DINOv2 backbones, ensembling, box fusion, and per-class recall release gates.
  • Experience applying GxP, 21 CFR Part 11, SOC 2, ISO 27001, HIPAA, or PCI-DSS to software delivery.
  • Test automation strategy using Playwright and governance of AI coding tools.
  • Experience running an engineering practice or Center of Excellence, including standards, reusable assets, training, or hiring.
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
9 Employees
Year Founded: 2009

What We Do

Benchmarkit helps B2B SaaS companies make metrics-informed decisions through industry benchmarks, research, media, and events. Formerly known as RevOps Squared, the company aims to enable B2B SaaS organizations to increase revenue growth efficiency and enterprise value by providing access to timely, contextual benchmarks and industry best practices across the entire customer journey, including customer acquisition, retention, and expansion.

Similar Jobs

LRN Logo LRN

Technical Lead

Edtech • Information Technology
In-Office
Mumbai, Maharashtra, IND
780 Employees

Mastercard Logo Mastercard

Lead Product Management - Technical - AI Transformation

Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Hybrid
Pune, Maharashtra, IND
38800 Employees

Fold Health Logo Fold Health

Technical Lead

Artificial Intelligence • Healthtech • Software
In-Office
Pune, Maharashtra, IND

Perficient Logo Perficient

Consultant

Information Technology
In-Office or Remote
2 Locations
3295 Employees

Similar Companies Hiring

Revel Thumbnail
Aerospace • Hardware • Robotics • Software
Marina Del Rey, California
70 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
33 Employees
VCA Animal Hospitals Thumbnail
Pet • Professional Services
Los Angeles, CA
35000 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account