Senior Principal Application Support Engineering Lead – AI Products & Agents

At Lilly, the work is demanding because patients are waiting. We unite caring with discovery to help make life better for people around the world, knowing that every decision, every detail, and every day matters. Headquartered in Indianapolis, Indiana, our over 50,000 employees around the globe take on complex challenges to discover and deliver life-changing medicines, strengthen how health is understood and managed, and support the communities we serve. This is hard, urgent, selfless work—but it’s work worth doing. If you’re driven by purpose and ready to bring your best to work that truly matters for patients, we invite you to join us. 

About Tech@Lilly

Senior Principal Application Support Engineering Lead– AI Products & Agents

Job Description

Job Level: R4 – Senior Principal

Experience: 10–14+ years

Location: Hyderabad (Onsite)

Employment Type: Full-time

Job Family: Operations / Application Support / Production Engineering / SRE-aligned Support

About the Technology Organization

Technology at Lilly builds and operates mission-critical digital products and platforms that support the discovery, development, and delivery of medicines that make life better for people around the world. Our teams operate in highly regulated, high-availability environments, where operational excellence, reliability, and quality are non-negotiable.

The Software Product Engineering (SPE) organization applies a product, platform, and reliability-first mindset, ensuring that operational capabilities scale sustainably across the enterprise.

Role Summary

As a Senior Principal Application Support Engineering Lead (R4), you are the senior-most operational authority for a technology team or portfolio of applications. You will lead Support Operations end-to-end, owning operational outcomes across availability, incident management, readiness, and continuous reliability improvement.

This role is both strategic and hands-on. You are accountable for:

• The health, stability, and operability of production systems
• The effectiveness and maturity of support operations
• Shift leadership and execution during critical operational windows
• Influencing engineering, product, and platform teams to prevent incidents—not just respond to them

At R4, success is measured by organizational impact, operational predictability, and the ability to scale reliability through others.

What You'll Be Doing (Key Responsibilities)

1) Support Operations Leadership & Shift Ownership

• Lead end-to-end support operations for a technology team, ensuring consistent execution across shifts and time zones.
• Act as the primary operational leader during assigned shifts, accountable for incident response quality, prioritization, and decision-making.
• Ensure effective shift handovers, operational continuity, and shared accountability across global support teams.
• Establish and evolve shift-level operating models, escalation paths, and decision frameworks.

2) Major Incident Command & Executive Escalation

• Serve as the incident commander for the most complex, high-impact production incidents.
• Lead war-room execution, cross-team coordination, and recovery strategy across engineering, product, platform, security, and vendor teams.
• Provide clear, timely, and confident communication to senior technology and business stakeholders during outages.
• Ensure incidents are handled with rigor, consistency, and accountability.

3) Enterprise Problem Management & Defect Elimination

• Own Problem Management for recurring and systemic issues across the supported technology landscape.
• Drive high-quality Root Cause Analysis (RCA) and ensure corrective actions address root causes—not symptoms.
• Hold teams accountable for long-term fixes and track outcomes to measurable reliability improvements.
• Identify cross-product failure patterns and influence architectural or platform-level remediation.

4) Reliability Strategy & Operational Excellence

• Define and drive operational reliability strategy for the technology team, aligned with SRE and production engineering principles.
• Influence the adoption of SLIs, SLOs, error budgets, and reliability reporting across products.
• Champion improvements in availability, performance, scalability, resilience, and recovery capabilities.
• Establish and enforce operational readiness standards (runbooks, rollback plans, monitoring coverage, post-release validation).

5) Observability, Automation & Toil Reduction

• Set direction for observability strategy across logs, metrics, and traces, ensuring actionable insights and high signal quality.
• Drive automation initiatives that significantly reduce manual effort, human error, and MTTR.
• Promote standard tooling, reusable runbooks, and automated remediation patterns across teams.
• Ensure support operations scale through systems and automation—not heroics.

6) Deployment, Change & Release Governance

• Provide senior operational oversight for releases and deployments, including risk assessment and go/no-go decisions.
• Partner with Engineering and Platform teams to improve CI/CD operational safety and post-release validation.
• Ensure operational risks are identified early and mitigated before production impact.

7) Compliance, Security & Regulated Environment Readiness

• Ensure support operations comply with Lilly standards and applicable regulatory requirements.
• Promote secure operational practices, auditability, and proper handling of sensitive data during incidents.
• Act as a trusted operational leader in regulated and validated environments.

8) Organizational Leadership & Talent Development

• Set the operational bar for the organization through standards, expectations, and role modeling.
• Mentor and develop R2/R3 engineers, building deep operational expertise and leadership capability.
• Influence managers, architects, and engineering leaders through credibility and outcomes rather than authority.
• Contribute to the evolution of enterprise-wide support and reliability practices.

How You Will Succeed (R4 Success Profile)

At the Senior Principal (R4) level, success is defined by breadth of impact and sustained outcomes:

• Be recognized as the senior operational authority for your technology area.
• Demonstrate measurable, sustained improvements such as:
• Reduced major incidents and repeat failures
• Improved MTTR and detection times
• Higher operational predictability and release confidence
• Reduced dependency on manual intervention
• Influence decisions across multiple teams and leaders through expertise and trust.
• Scale reliability and operational excellence through systems, standards, and people, not individual effort.

What You Should Bring (Qualifications)

Required

• 10–14+ years of experience in Application Support, Production Engineering, SRE, or Software Engineering with deep operational ownership.
• Extensive experience leading high-severity incident response and operational execution.
• Deep hands-on troubleshooting across distributed applications, integrations, databases, and cloud platforms.
• Strong experience with monitoring, logging, and alerting platforms (e.g., Datadog, Splunk, ELK, AppDynamics, CloudWatch).
• Advanced scripting and automation skills (e.g., Python, Bash) to drive operational efficiency.
• Proven ability to operate in regulated and enterprise environments.
• Proven ability with skills like Power Automate, LLM models, Gen AI, Agentic, Gen AI.
• Exceptional communication and leadership skills under pressure.

Preferred / Nice to Have

• Experience operationalizing SLIs, SLOs, and error budgets.
• Familiarity with containers, Kubernetes, CI/CD pipelines, and Infrastructure as Code concepts.
• Experience leading globally distributed support teams or shift-based operations.
• Track record of influencing enterprise-wide operational standards.

Leadership Expectations

• Acts with enterprise-first mindset, beyond individual products or teams.
• Drives accountability, clarity, and calm during high-pressure situations.
• Builds trust through consistency, technical depth, and follow-through.
• Raises the capability of the organization—not just personal output.

Additional Information

Availability to work flexible work hours is/may be required. This team will support continuous operations across two shifts and therefore, this role will require non-standard work hours, and some work on weekends and holidays. Appropriate adjustments in benefits will be provided for employees working non-standard hours where applicable.

Candidate should be open to work on different shifts - 6AM to 2PM and 2PM to 11PM when required.

Lilly is dedicated to helping individuals with disabilities to actively engage in the workforce, ensuring equal opportunities when vying for positions. If you require accommodation to submit a resume for a position at Lilly, please complete the accommodation request form (https://careers.lilly.com/us/en/workplace-accommodation) for further assistance. Please note this is for individuals to request an accommodation as part of the application process and any other correspondence will not receive a response.

Lilly does not discriminate on the basis of age, race, color, religion, gender, sexual orientation, gender identity, gender expression, national origin, protected veteran status, disability or any other legally protected status.