At Nielsen, we are passionate about our work to power a better media future for all people by providing powerful insights that drive client decisions and deliver extraordinary results. Our talented, global workforce is dedicated to capturing audience engagement with content - wherever and whenever it’s consumed. Together, we are proudly rooted in our deep legacy as we stand at the forefront of the media revolution. When you join Nielsen, you will join a dynamic team committed to excellence, perseverance, and the ambition to make an impact together. We champion you, because when you succeed, we do too. We enable your best to power our future.
Job DescriptionStrategic Mandate
As the Data Platform Lead, you are the technical owner and steward of Nielsen's Databricks Lakehouse for Corporate function. You will drive the strategy, architecture, and operational maturity of our Databricks ecosystem end to end — from workspace and Unity Catalog design to cost governance and platform reliability. This role sits at the intersection of platform engineering and data governance: you will define the guardrails, standards, and reusable patterns that let every engineering and analytics team build confidently on a secure, scalable, and cost-efficient foundation. You will act as the "Platform Authority" for Databricks, ensuring the lakehouse scales with the business while remaining governed, observable, and audit-ready. Critically, you will build an AI-driven data platform — architecting AI-native capabilities (feature stores, vector search, GenAI-ready pipelines) directly into the platform, while also using AI-assisted and agentic engineering practices to build, manage, and scale the platform itself faster and more reliably
QualificationsCore Goals & Responsibilities
- Lakehouse Platform Ownership: Own the end-to-end architecture of the Databricks Lakehouse, including workspace topology, cluster policies, compute strategy (job clusters, SQL warehouses, serverless), and multi-cloud deployment patterns (AWS/Azure/GCP).
- Unity Catalog & Governance: Design and enforce the global data governance model — catalogs, schemas, access controls, lineage, and audit logging — via Unity Catalog, ensuring consistent metadata management and fine-grained access across business units.
- Data Engineering Standards: Define and enforce best practices for Delta Lake table design, Delta Live Tables (DLT) pipelines, medallion architecture (bronze/silver/gold), and performance optimization (Z-ordering, liquid clustering, partitioning, file compaction).
- Platform Reliability & Automation: Partner with CloudOps and DevOps to industrialize the platform through Infrastructure-as-Code (Terraform/Databricks Asset Bundles), CI/CD pipelines, and monitoring, targeting 99.99% platform availability.
- FinOps & Cost Governance: Own cost transparency and optimization across the Databricks estate — cluster right-sizing, workload isolation, chargeback/showback models, and proactive budget alerting — bringing "Strategic Foresight" to cloud spend.
- Cross-Functional Enablement: Work directly with Data Engineering, Analytics, Data Science, and business stakeholders (Finance, HR, Product) to translate requirements into platform capabilities and self-service patterns.
- Governed AI Enablement: Own the governed rollout of AI use cases and native AI features (e.g., Databricks Genie, AI/BI Dashboards, Mosaic AI) on the platform. You hold the keys to unlocking GenAI and natural-language analytics for the business, safely, within Unity Catalog's governance and access-control boundaries.
- Community & Mentorship: Build and lead a community of practice for Databricks developers across the organization — publishing standards, running enablement sessions, and unblocking delivery friction for engineering teams
Expertise & Technology Stack
- Core Engineering: 8–10+ years of deep experience in data engineering, data platform architecture, or distributed systems, with 3+ years specifically architecting and operating Databricks environments at scale.
- Databricks Platform Engineering: Proven hands-on experience building, managing, running, and scaling a production-grade Databricks Data Platform end-to-end — including Unity Catalog, Delta Lake, Delta Live Tables, Databricks Workflows, Databricks SQL, and cluster/compute policy design.
- AI-Native Platform & AI-Assisted Engineering: Experience architecting AI-native platform capabilities (feature stores, vector search/indexes, GenAI-ready pipelines) into the platform, combined with using AI-assisted and agentic engineering practices (e.g., AI coding copilots, automated testing/ops agents) to build, manage, and scale the platform itself.
- Distributed Compute: Deep hands-on expertise with Apache Spark (batch and structured streaming) for large-scale data processing and performance tuning.
- Frameworks & Languages: Expert-level proficiency in Python and SQL, with working knowledge of dbt for transformation-layer standardization.
- Infrastructure & Automation: Proven experience with Terraform (or Databricks Asset Bundles) and CI/CD tooling to codify and automate platform provisioning and pipeline deployment.
- Security & Governance: Strong grasp of data security, RBAC/ABAC, PII handling, and compliance requirements within a governed lakehouse.
- Analytical Thinking: Ability to challenge status-quo assumptions, evaluate architectural trade-offs, and provide a pragmatic pivot when technical or cost risks arise
Requirements & Qualifications
- Experience: 8–10+ years in a Senior or Lead Data Engineering / Data Platform capacity, including direct ownership of a production Databricks environment.
- Education/Certification:
- Required: Proven track record of building, managing, running, and scaling large-scale, multi-tenant Databricks Data Platforms.
- Preferred: Databricks Certified Data Engineer Professional or Databricks Certified Platform Architect.
- Communication: A "Bottom Line Up Front" (BLUF) communication style. You must be able to engage with both technical teams and non-technical stakeholders (HR/Finance) and clearly explain the "how" and "why" behind platform decisions
Please be aware that job-seekers may be at risk of targeting by scammers seeking personal data or money. Nielsen recruiters will only contact you through official job boards, LinkedIn, or email with a nielsen.com domain. Be cautious of any outreach claiming to be from Nielsen via other messaging platforms or personal email addresses. Always verify that email communications come from an @nielsen.com address. If you're unsure about the authenticity of a job offer or communication, please contact Nielsen directly through our official website or verified social media channels.
Skills Required
- 8–10+ years of experience in data engineering, data platform architecture, or distributed systems
- 3+ years architecting and operating Databricks environments at scale
- Senior or Lead Data Engineering/Data Platform experience
- Direct ownership of a production Databricks environment
- Experience building, managing, running, and scaling large-scale, multi-tenant Databricks data platforms
- Hands-on experience with Unity Catalog, Delta Lake, Delta Live Tables, Databricks Workflows, Databricks SQL, and cluster/compute policy design
- Expert-level proficiency in Python and SQL
- Deep hands-on Apache Spark experience, including batch, structured streaming, and performance tuning
- Working knowledge of dbt
- Experience with Terraform or Databricks Asset Bundles and CI/CD tooling
- Strong understanding of data security, RBAC/ABAC, PII handling, and compliance requirements
- Experience architecting AI-native platform capabilities such as feature stores, vector search, and GenAI-ready pipelines
- Ability to communicate clearly with technical and non-technical stakeholders
- Databricks Certified Data Engineer Professional or Databricks Certified Platform Architect
Nielsen Compensation & Benefits Highlights
The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Nielsen and has not been reviewed or approved by Nielsen.
-
Parental & Family Support — Paid parental leave is described as generous for all parents, with birthing parents receiving substantially longer time, and the program is complemented by adoption, fertility, and surrogacy support. Employer-verified entries and company communications consistently highlight family benefits as a standout.
-
Leave & Time Off Breadth — Paid time off is considered solid, with some teams using flexible or unlimited PTO models and additional offerings like paid volunteer time. These time-away policies often contribute meaningfully to work-life balance.
-
Wellbeing & Lifestyle Benefits — Smart Work flexibility, an Employee Assistance Program, and broader well-being initiatives are emphasized as part of a holistic package. These elements are frequently cited as strengths that enhance day-to-day experience.
Nielsen Insights
What We Do
For over 100 years, Nielsen has been a global leader in audience measurement, data and analytics, shaping the future of media. Measuring behavior across all channels and platforms to discover what audiences love, we empower our clients with trusted intelligence that fuels action. As the global media industry continues to evolve, Nielsen will be at the forefront of delivering audience insights that empower marketers and content developers, and ultimately help connect consumers to what matters most to them. We will be guided by our values of inclusion, courage and growth; and we will continue to innovate our products and technology while investing in our talent to drive sustainable and scalable long-term growth.
Why Work With Us
At Nielsen, we have big dreams and aspirations, all of which are achievable if we work as a team. Our culture helps us come together and clarifies how we work together to achieve these goals
