The Sr. Staff Platform / Data Reliability Engineer is responsible for making Databricks a reliable, secure, scalable, and operationally mature platform for enterprise use. This role focuses on the platform itself rather than domain-specific business modeling, ensuring the lakehouse can support multiple domains, regulated data handling, and a growing set of production workloads without becoming fragile, expensive, or hard to govern.
This role partners closely with the Senior Staff Data Engineer, who focuses more heavily on ingestion and medallion implementation patterns, and with the existing Cloud & Infrastructure team, which owns cloud accounts, networking, and foundational infrastructure. The Platform / Data Reliability Engineer owns the Databricks operational layer that sits above that foundation: reliability, observability, deployment standards, compute and job policies, and platform enablement patterns for internal users.
What you'll do:
- Own operational excellence for the Databricks platform, including monitoring, alerting, observability, incident response support, and production runbook patterns for data jobs and platform services.
- Define and maintain CI/CD and promotion standards for Databricks assets, including workflows, jobs, notebooks, code packages, infrastructure configuration, and environment promotion from dev to prod.
- Design and maintain platform standards for job orchestration, cluster and compute policies, service principal usage, environment isolation, and production execution reliability.
- Establish reusable operational templates and enablement patterns for new domains onboarding to Databricks, including logging conventions, job tagging, metadata capture, and support handoff expectations.
- Partner with the Senior Data Engineer to ensure ingestion and medallion patterns are implemented in a way that is observable, recoverable, cost-aware, and secure in production.
- Work with the cloud/infrastructure team to align Databricks configuration and usage patterns with broader enterprise cloud standards, especially where commercial and future government-hosted environments are involved.
- Help enforce technical controls for data segregation, access boundaries, and operational compliance in a highly regulated environment.
- Track and improve platform health metrics such as job success rates, incident trends, data pipeline reliability, cost efficiency, and environment drift.
- Document platform standards, operational expectations, and support models so the Databricks platform can scale beyond a small founding team.
- Mentor internal engineers who are growing into platform responsibilities, helping expand Databricks operational knowledge within the team.
Required qualifications:
- 12+ years of relevant experience in data platform engineering, platform operations, site reliability engineering, or modern cloud data infrastructure.
- Hands-on experience with Databricks or a closely related cloud data platform in production environments.
- Experience designing or operating CI/CD, environment promotion, version control, and deployment automation for data platforms and pipelines.
- Strong understanding of platform operations concepts such as observability, monitoring, alerting, incident management, and reliability engineering.
- Experience with compute policy design, workload isolation, service principals, and secure production execution patterns on cloud data platforms.
- Ability to work effectively in a regulated or security-sensitive environment with strong expectations around access control, auditability, and operational discipline.
- Strong collaboration skills and comfort partnering with cloud/infrastructure, security, data engineering, and analytics stakeholders.
Preferred qualifications:
- Databricks certification and/or strong demonstrated expertise with Delta Lake, Unity Catalog, Workflows, and Databricks Asset Bundles.
- Experience with infrastructure-as-code and platform automation in enterprise environments.
- Experience supporting commercial and government or otherwise segregated environments with different compliance and access requirements.
- Experience in defense, aerospace, federal, or another regulated industry.
Skills Required
- 12+ years experience in data platform engineering, platform operations, site reliability engineering, or cloud data infrastructure
- Hands-on experience with Databricks or a closely related cloud data platform in production
- Experience designing or operating CI/CD, environment promotion, version control, and deployment automation for data platforms and pipelines
- Strong understanding of observability, monitoring, alerting, incident management, and reliability engineering
- Experience with compute policy design, workload isolation, service principals, and secure production execution patterns on cloud data platforms
- Ability to work effectively in a regulated or security-sensitive environment with strong access control, auditability, and operational discipline
- Strong collaboration skills with cloud/infrastructure, security, data engineering, and analytics stakeholders
- Databricks certification and/or strong demonstrated expertise with Delta Lake, Unity Catalog, Workflows, and Databricks Asset Bundles
- Experience with infrastructure-as-code and platform automation in enterprise environments
- Experience supporting commercial and government or otherwise segregated environments with different compliance and access requirements
- Experience in defense, aerospace, federal, or another regulated industry
What the Team is Saying
Shield AI Compensation & Benefits Highlights
-
Healthcare Strength — Healthcare coverage includes medical, dental, vision, and mental-health support, with company materials describing excellent coverage. Feedback suggests these offerings are comprehensive and consistently highlighted across official and third-party benefit lists.
-
Equity Value & Accessibility — Equity is granted to all full-time hires, with RSU structures and tools like Carta Tax intended to improve understanding and tax timing. Feedback suggests this broad-based ownership approach is a notable component of total rewards.
-
Parental & Family Support — Benefits include paid parental leave, fertility support, childcare benefits, family medical leave, and onsite resources such as a Mother's Room. Feedback suggests the family-oriented offerings are more expansive than basic coverage.
Shield AI Insights
What We Do
At Shield AI, you won't wait years to see your work reach the field. You'll build hardware and software that operates in the real world right now, in the hands of the people who depend on it. Hivemind, our AI pilot, has been flying since 2018. It has flown more than 30 platforms, including an F-16, and it now sits under a U.S. Air Force production contract for Collaborative Combat Aircraft. When you write code or shape a system here, you contribute to technology with a proven flight record and a clear production future. V-BAT flies intelligence, surveillance, and reconnaissance missions with an operational record that stretches from Ukraine to the Indo-Pacific. It delivers eyes where they matter most, in the most demanding conditions on earth. The teams behind it watch their work get tested where the stakes are real. X-BAT takes its first flight this year. It's an AI-piloted fighter that needs no runway, built to operate where traditional aircraft can't. Join now and you help shape a program at its earliest, most formative stage. That's the kind of ground-floor work that defines a career. Do the most impactful work of your life, on problems that matter. Autonomy at this level asks a lot of you. You'll take on problems in perception, planning, and control that few teams anywhere are equipped to solve. You'll work across disciplines, from aerospace and robotics to machine learning and systems engineering, alongside people who hold themselves to an exacting standard and expect the same from you. Our mission is clear: protect service members and civilians with intelligent systems. That purpose runs through every decision, every design review, and every deployment. It's why the work here carries a weight you can feel. Ready to join our mission? Explore our open roles and find where you fit.
Why Work With Us
Founded in 2015 by a former Navy SEAL, Shield AI builds AI pilots and uncrewed aircraft. Veterans aren't an afterthought here, they're at every level. It's why the work carries weight: AI pilots and uncrewed aircraft flying real missions, from Ukraine to the Indo-Pacific, protecting service members and civilians.
Gallery
%203.jpg)
Shield AI Teams
.jpg)
Shield AI Offices
Hybrid Workspace
Employees engage in a combination of remote and on-site work.
