The Role
Leads Azure Databricks platform solutioning, architecture, implementation, security, governance, networking, Infrastructure-as-Code, CI/CD, and data pipeline design. Establishes Unity Catalog, scalable ETL/ELT workflows, and lakehouse modernization strategies. Provides technical leadership through mentoring, hiring, delivery oversight, client pitches, RFP responses, and practice development across data engineering, AI, machine learning, and AI operations.
Summary Generated by Built In
- Solutioning – developing solutions for specific client requirements. Coming up with ideas/solutions/proposals for pitching and help in conversion conversations
- Developing Reference implementations on new technologies – the technology maybe new in the market or maybe new to the client context
- Lead hands-on implementation of Databricks environments including workspace configuration, cluster size and policies, networking, security perimeters, Infrastructure-as-Code, CI/CD and Azure cloud integration.
- Architect and implement Unity Catalog for unified data governance, including metastore setup, catalog/schema design, data lineage, access control policies, and cross-workspace sharing.
- Define and enforce platform-level security standards covering identity management, row/column-level security, credential management, private endpoints, and audit logging.
- Design and review scalable ETL/ELT pipelines using Delta Live Tables, Databricks Workflows, and open-source frameworks.
- Team management responsibilities – technical mentoring of the team, interviewing and identifying right candidates for the team and training to develop the skills
- Technical review, mentoring and oversight of technical deliverables.
- Thought leadership – Develop a robust Data Engineering practice in conjunction with areas such as AI / Gen AI/ ML / AI Ops etc.
- Collaborate on client pitches, RFP responses, and solution proposals specific to Databricks-based platform modernization engagements.
Requirements
- 12-15 years of experience in Data Engineering, architecting scalable solutions.
- 3+ years hands-on Azure Databricks (Unity Catalog, DLT, Workflows, Delta Lake)
- Databricks Certified Data Engineer Professional or Platform Administrator certification preferred
- Strong technical experience in SQL, Python, PySpark, SparkSQL
- Experience with legacy to modern data platform migration
- Hands on experience with Open-source stack for Lakehouse or Warehouse implementation.
- Working experience with big data (gigabytes to petabytes) in prior projects
- Demonstrated Tech leadership roles in recent past with significant exposure to pre-sales, solutioning and managing clients.
- Hands-on experience and good understanding of Data Modelling, Data Governance and Data Security tools and solutions.
- Strong verbal and written communication skills.
Benefits
- Competitive salary and performance-based bonuses.
- Comprehensive insurance plans.
- Collaborative and supportive work environment.
- Chance to learn and grow with a talented team.
- A positive and fun work environment.
Skills Required
- 12–15 years of experience in data engineering and architecting scalable solutions
- At least 3 years of hands-on Azure Databricks experience, including Unity Catalog, Delta Live Tables, Workflows, and Delta Lake
- Databricks Certified Data Engineer Professional or Platform Administrator certification
- Strong technical experience with SQL, Python, PySpark, and SparkSQL
- Experience migrating legacy data platforms to modern data platforms
- Hands-on experience with open-source stacks for lakehouse or data warehouse implementations
- Experience working with big data ranging from gigabytes to petabytes
- Recent technical leadership experience with significant exposure to presales, solutioning, and client management
- Hands-on experience and strong understanding of data modeling, data governance, and data security tools and solutions
- Strong verbal and written communication skills
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Prescience is a Danish SaaS company that provides collaborative supply-chain execution software for industrial businesses. Its platform helps customers and suppliers plan production, track manufacturing progress, monitor execution, manage capacity, improve quality assurance, collect as-built documentation, and coordinate inventory and inbound and outbound logistics. By integrating supplier data in real time, Prescience delivers visibility, risk detection, and operational control across dispersed global supply chains.
