Job Summary
We are seeking a skilled Databricks Engineer with minimum of 5+ years of hands-on in Databricks who can design, develop, and optimize scalable data platforms and analytics solutions using the Databricks Lakehouse Platform. The ideal candidate will have expertise in data engineering, ETL/ELT development, cloud technologies preferably Azure, and big data processing to support enterprise analytics and AI initiatives.
Job Description: Azure / Databricks Data Engineer (9-15 Years Experience)
Senior Azure Databricks Data Engineer
Experience
9-15 Years of IT Experience
Location
Hybrid
Job Summary
We are seeking a skilled Databricks Engineer with minimum of 5+ years of hands-on in Databricks who can design, develop, and optimize scalable data platforms and analytics solutions using the Databricks Lakehouse Platform. The ideal candidate will have expertise in data engineering, ETL/ELT development, cloud technologies preferably Azure, and big data processing to support enterprise analytics and AI initiatives.
Key Responsibilities
- Design, develop, and maintain data pipelines using Databricks, Apache Spark, and cloud-native services.
- Build and optimize ETL/ELT workflows for large-scale structured and unstructured data.
- Develop data models and implement data quality, validation, and governance frameworks.
- Integrate data from multiple sources into a unified Lakehouse architecture.
- Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency.
- Implement security controls, access management, and data governance using Unity Catalog.
- Collaborate with business, analytics, and AI/ML teams to deliver trusted data products.
- Monitor, troubleshoot, and resolve data pipeline issues.
- Support CI/CD, DevOps, and infrastructure automation practices.
- Maintain technical documentation and best practices.
Required Technical Skills Core Technologies
- Databricks Lakehouse Platform
- Apache Spark / PySpark
- Delta Lake
- SQL
- Python
Data Engineering
- ETL / ELT Development
- Data Modeling
- Data Warehousing
- Data Quality & Validation
- Streaming & Real-Time Processing
Governance & Security
- Unity Catalog
- Data Lineage
- Row-Level Security
- Access Control & Compliance
- Data Governance Frameworks
Cloud & DevOps
- Azure / AWS / GCP
- Terraform
- GitHub Actions / Azure DevOps
- CI/CD Pipelines
Analytics & AI
- Semantic Layers
- Data Products
- BI Platforms
- Machine Learning Support
- Generative AI & RAG Architectures
Required Qualifications
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field.
- 9-15 years of experience in Data Engineering and Data Warehousing.
- Minimum 5+ years of hands-on experience with Azure Data Engineering technologies.
- Minimum 4+ years of hands-on experience with Azure Databricks and Spark ecosystem.
- Strong understanding of data lake, lakehouse, and cloud-native architecture patterns.
- Experience in handling large-scale structured and unstructured datasets.
- Strong analytical, problem-solving, and troubleshooting skills.
Preferred Qualifications
- Microsoft Certified: Azure Data Engineer Associate (DP-203).
- Databricks Certified Data Engineer Associate/Professional.
- Experience with Snowflake, Power BI, or Microsoft Fabric.
- Experience in real-time streaming solutions using Kafka/Event Hubs.
- Exposure to Data Governance and Master Data Management initiatives.
Soft Skills
- Strong stakeholder management and communication skills.
- Ability to lead technical initiatives and drive architecture discussions.
- Experience working in Agile/Scrum environments.
- Excellent documentation and presentation skills.
- Strong mentoring and team leadership abilities.
Nice to Have
- Microsoft Fabric
- Power BI
- Azure Event Hubs
- Kafka
- Machine Learning data pipelines
- Data Governance tools such as Purview
ResponsibilitiesSupport critical and nonproduction databases in multitenant, highstress environments.
Support applications Installation, Configuration, Management, and Monitoring of databases in a SOX and PHI compliant environment
Installation, configuration, and integration of thirdparty applications and tools
Development of procedures for automated monitoring and proactive intervention, preventing customer impact QualificationsBachelors Degree/ Post Bachelor Degree 8 - 10+ Years
Skills Required
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field
- 9-15 years of experience in data engineering and data warehousing
- Minimum 5 years of hands-on experience with Azure data engineering technologies
- Minimum 4 years of hands-on experience with Azure Databricks and the Spark ecosystem
- Strong understanding of data lake, Lakehouse, and cloud-native architecture patterns
- Experience handling large-scale structured and unstructured datasets
- Strong analytical, problem-solving, and troubleshooting skills
- Microsoft Certified Azure Data Engineer Associate DP-203
- Databricks Certified Data Engineer Associate or Professional certification
- Experience with Snowflake, Power BI, or Microsoft Fabric
- Experience with real-time streaming solutions using Kafka or Azure Event Hubs
- Exposure to data governance and master data management initiatives
- Experience with machine learning data pipelines
- Experience with data governance tools such as Microsoft Purview
- Strong stakeholder management and communication skills
- Ability to lead technical initiatives and drive architecture discussions
- Experience working in Agile or Scrum environments
- Excellent documentation and presentation skills
- Strong mentoring and team leadership abilities
What We Do
Choosing a digital partner is about more than capabilities — it’s about collaboration and character. Unrealistic overhauls and off-the-shelf products ignore what matters most — your unique needs, culture, goals, and your legacy data and technology environments. At EXL, our collaboration is built on ongoing listening and learning to adapt our methodologies. We’re your business evolution partner—tailoring solutions that make the most of data to make better business decisions and drive more intelligence into your increasingly digital operations. Whether your goals are scaling the use of AI and digital, redesign operating models, or driving better and faster decisions, we’re here to partner with you to help you gain—and maintain—competitive advantage with efficient, sustainable models at scale. Our expertise in transformation, data science, and change management helps make your business more efficient and effective, improve customer relationships and enhance revenue growth. Instead of focusing on multi-year, resource- and time-intensive platform designs or migrations, we look deeper at your entire value chain to integrate strategies with impact. We use our specialization in analytics, digital interventions, and operations management—alongside deep industry expertise — to deliver solutions that help you outperform the competition. At EXL, it’s all about outcomes—your outcomes—and delivering success on your terms. Share your goals with us and together, we’ll optimize how you leverage data to drive your business forward. For more information, visit www.exlservice.com.
