The Role
Designs and builds scalable data platforms, Lakehouse architectures, ETL/ELT pipelines, data models, and cloud-native solutions using Databricks, PySpark, Python, SQL, and AWS. Responsibilities include Spark optimization, governance, security, data quality, observability, CI/CD, testing, DataOps, production issue resolution, architectural recommendations, and technical leadership across globally distributed teams.
Summary Generated by Built In
Job Role: Senior Data Engineer
About Anblicks
Role Summary
Key Responsibilities
Location: Hyderabad, India
Experience: 6–8 Years
Work Mode: Hybrid
About Anblicks
Anblicks is a Great Place to Work® Certified Data & AI company helping enterprises build modern, intelligent, and scalable data platforms. As a Snowflake Elite Services Partner, Databricks Consulting Partner, AWS Advanced Consulting Partner, and Microsoft Azure Gold Partner, we work with leading organizations to solve complex data, cloud, analytics, and AI challenges.
We are looking for an experienced Senior Data Engineer to design and build scalable data platforms and high-performance data solutions using Databricks, PySpark, Python, SQL, and cloud technologies.
Role Summary
As a Senior Data Engineer, you will be responsible for designing, developing, and optimizing modern data engineering solutions and Lakehouse platforms. You will work on large-scale data processing, data pipelines, data modeling, performance optimization, governance, and cloud-native technologies while contributing to technical design and engineering best practices.
Key Responsibilities
- Design and develop scalable, reliable, and high-performance data pipelines using Databricks, Spark, PySpark, and cloud technologies.
- Design and implement complex ETL/ELT workflows using Databricks, PySpark, SQL, and cloud services.
- Establish engineering standards, design patterns, and best practices for data engineering solutions.
- Make technical and architectural recommendations focused on scalability, performance, maintainability, security, and cost optimization.
- Develop reusable frameworks, libraries, and accelerators to improve engineering productivity.
- Design and implement optimized data models for analytics, operational, and AI/ML use cases.
- Identify and resolve performance bottlenecks through Spark optimization, query tuning, partitioning, and resource management.
- Implement and support data quality, monitoring, observability, governance, and security capabilities.
- Follow software engineering best practices including code reviews, automated testing, CI/CD, Infrastructure as Code, and DevSecOps.
- Lead root-cause analysis and resolution of complex production issues.
- Drive improvements in platform reliability, resiliency, scalability, and operational efficiency.
- Contribute to modern DataOps and Data Product practices.
- Collaborate with architects, developers, data scientists, business stakeholders, and cross-functional teams.
- Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related field.
- 6–8 years of overall Data Engineering experience, including 5+ years of hands-on Databricks experience.
- Strong experience building large-scale data platforms on public cloud environments.
- Expert-level proficiency in Python, PySpark, SQL, and distributed data processing.
- Strong understanding of Lakehouse architecture, data warehousing, and modern data engineering concepts.
- Hands-on experience with Spark optimization, performance tuning, partitioning strategies, and workload management.
- Strong experience with AWS services such as S3, EMR, Glue, Lambda, ECS/EKS, Redshift, and IAM.
- Experience designing scalable data models and data architectures for analytical and operational workloads.
- Strong understanding of CI/CD, automated testing, and software engineering best practices.
- Hands-on experience with Databricks Asset Bundles (DABs).
- Experience implementing security, governance, privacy, and compliance controls within data platforms.
- Ability to drive technical initiatives and contribute to architectural decisions.
- Strong problem-solving, communication, and stakeholder management skills.
- Experience working in Agile, cross-functional, and globally distributed teams.
- Experience with Infrastructure as Code tools such as Terraform or CloudFormation.
- Experience implementing data quality, observability, and metadata management solutions.
- Experience with streaming technologies such as Kafka, Azure Event Hubs, or Amazon Kinesis.
- Experience deploying and operationalizing Machine Learning and Generative AI solutions.
- Strong knowledge of Delta Lake optimization, Unity Catalog, and Databricks governance.
- Experience leading technical design reviews and mentoring engineering teams.
- Work on large-scale Data, Cloud, AI & Analytics projects.
- Opportunity to work with leading cloud and data technologies including Databricks, Snowflake, AWS, Azure, and AI/ML.
- Exposure to enterprise-scale transformation and modernization initiatives.
- Collaborative, technology-driven engineering environment.
Skills Required
- Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related field
- 6-8 years of overall Data Engineering experience
- 5+ years of hands-on Databricks experience
- Experience building large-scale data platforms on public cloud environments
- Expert-level proficiency in Python, PySpark, SQL, and distributed data processing
- Strong understanding of Lakehouse architecture, data warehousing, and modern data engineering concepts
- Experience with Spark optimization, performance tuning, partitioning strategies, and workload management
- Experience with AWS services including S3, EMR, Glue, Lambda, ECS/EKS, Redshift, and IAM
- Experience designing scalable data models and data architectures for analytical and operational workloads
- Understanding of CI/CD, automated testing, and software engineering best practices
- Hands-on experience with Databricks Asset Bundles
- Experience implementing security, governance, privacy, and compliance controls within data platforms
- Ability to drive technical initiatives and contribute to architectural decisions
- Strong problem-solving, communication, and stakeholder management skills
- Experience working in Agile, cross-functional, and globally distributed teams
- Experience with Infrastructure as Code tools such as Terraform or CloudFormation
- Experience implementing data quality, observability, and metadata management solutions
- Experience with streaming technologies such as Kafka, Azure Event Hubs, or Amazon Kinesis
- Experience deploying and operationalizing Machine Learning and Generative AI solutions
- Knowledge of Delta Lake optimization, Unity Catalog, and Databricks governance
- Experience leading technical design reviews and mentoring engineering teams
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Anblicks is a Cloud Data Analytics Company based out of Dallas, TX, with offices in USA, India, and Australia. Since 2004, Anblicks has been helping customers by bringing value to their data and implementing modern data architecture and advanced analytics solutions in the cloud.
