The Role
Designs and implements scalable Databricks Lakehouse data architectures, ETL/ELT pipelines, and batch or real-time processing solutions. Uses Spark, Delta Lake, SQL, cloud platforms, orchestration tools, and governance frameworks to optimize performance, cost, security, and scalability. Collaborates with stakeholders to translate business needs into technical solutions, integrates enterprise systems, supports migrations, and provides technical leadership and mentoring to data engineering teams.
Summary Generated by Built In
Databricks ArchitectJob Summary
We are looking for an experienced Databricks Architect to lead the design, development, and implementation of scalable data platforms on Databricks. The ideal candidate will have strong expertise in Spark, Delta Lake, data engineering, and cloud platforms (Azure/AWS/GCP), along with proven experience in architecting modern data solutions.
Key Responsibilities- Design and implement end-to-end data architecture using Databricks Lakehouse platform
- Define data engineering best practices and build scalable ETL/ELT pipelines
- Architect solutions leveraging Apache Spark (PySpark/Scala) and Databricks workflows
- Implement Delta Lake for data reliability, performance, and governance
- Collaborate with stakeholders to understand business requirements and translate them into technical solutions
- Optimize performance, cost, and scalability of data pipelines
- Provide technical leadership, mentoring data engineers and developers
- Ensure data security, governance, and compliance standards are met
- Integrate Databricks with cloud services (Azure, AWS, or GCP) and other enterprise systems
- Support real-time and batch data processing use cases
- Strong hands-on experience with Databricks Platform
- Expertise in Apache Spark (PySpark / Scala / SQL)
- Experience with Delta Lake, Unity Catalog, and Lakehouse architecture
- Proficiency in data pipelines, ETL/ELT frameworks
- Strong experience in SQL and data modeling
- Hands-on experience with cloud platforms:
- Azure (ADF, ADLS, Synapse) OR
- AWS (S3, Glue, EMR) OR
- Data warehousing and data lake solutions
- Batch & real-time data processing
- Data governance and security frameworks
- Performance tuning and optimization
- Databricks Workflows / Jobs
- Airflow / ADF / Prefect (any orchestration tool)
- CI/CD pipelines (Azure DevOps, GitHub Actions)
- Streaming tools like Kafka (preferred)
- Databricks Certifications (e.g., Databricks Certified Professional / Associate)
- Experience in data migration to Databricks
Skills Required
- Strong hands-on experience with the Databricks Platform
- Expertise in Apache Spark, PySpark, Scala, and SQL
- Experience with Delta Lake, Unity Catalog, and Lakehouse architecture
- Proficiency in data pipelines and ETL/ELT frameworks
- Strong experience in SQL and data modeling
- Hands-on experience with Azure, AWS, or GCP cloud platforms
- Knowledge of data warehousing and data lake solutions
- Experience with batch and real-time data processing
- Knowledge of data governance and security frameworks
- Experience with performance tuning and optimization
- Experience with Databricks Workflows or Jobs
- Experience with an orchestration tool such as Airflow, ADF, or Prefect
- Experience with CI/CD pipelines
- Kafka or other streaming tool experience
- Databricks certification
- Experience migrating data to Databricks
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Anblicks is a Cloud Data Analytics Company based out of Dallas, TX, with offices in USA, India, and Australia. Since 2004, Anblicks has been helping customers by bringing value to their data and implementing modern data architecture and advanced analytics solutions in the cloud.
