Data Architect - Databricks

Posted 10 Days Ago
Be an Early Applicant
Bengaluru North, Yelahanka, Bengaluru Urban, Karnataka, IND
In-Office
Expert/Leader
Artificial Intelligence • Software • Consulting • Generative AI
The Role
Lead enterprise data modernization across Databricks, Snowflake, and AWS. Define target-state architectures, Medallion data models, governance frameworks, CI/CD pipelines, and scalable batch, streaming, and CDC ingestion. Build secure, high-performance data foundations for analytics, machine learning, and Generative AI, including vector storage and feature layers. Establish architecture standards, optimize platform performance, collaborate across technical teams, and advise clients on data strategy and roadmaps.
Summary Generated by Built In

We are hiring a Data Architect to lead a strategic enterprise data modernization initiative for platform using Medallion architecture AWS Databricks. The role will define target-state architecture, establish enterprise data governance frameworks, implement CI/CD best practices for data platforms, and build GenAI-ready data foundations that enable advanced analytics, machine learning, and secure enterprise data access. 

You will be responsible for architecting scalable cloud-native data ecosystems, designing governed data models, and enabling real-time and batch data processing capabilities. This role requires deep ownership of architecture standards, platform performance, and cross-functional alignment across engineering, analytics, and data science teams to ensure consistent, secure, and high-performance data consumption across enterprise use cases.


Location - Mumbai/Bangalore/Hyderabad/Gurgaon (Hybrid - 3 Days a week in Office)


Responsibilities -

  • Lead enterprise-scale implementation of data warehouse data platforms on Databricks and Snowflake environments.
  • Design and implement Medallion (Bronze/Silver/Gold) architecture and scalable enterprise data models.
  • Establish data modeling standards (dimensional, data vault, lakehouse patterns) and ensure best practices across projects 
  • Establish enterprise data governance frameworks including cataloging, lineage, stewardship, and compliance using Atlan. 
  • Define and implement CI/CD pipelines for infrastructure and data platform deployments 
  • Design data architectures that support AI/ML and Generative AI workloads including vector storage, feature layers, and secure access patterns. 
  • Build scalable ingestion frameworks supporting batch, streaming, and CDC pipelines. 
  • Architect secure, high-performance data integration layers for analytics, BI, and AI consumption. 
  • Develop target-state architecture blueprints and enforce data standards, governance, and best practices across teams. 
  • Collaborate with engineering, analytics, and data science teams to ensure platform alignment and scalability. 
  • Engage with clients as a trusted advisor, driving data strategy, roadmap definition, and identifying opportunities for expansion. 


Requirements
  • Minimum 10 years of experience in Data & Analytics Architecture with proven leadership in large-scale enterprise data modernization initiatives
  • Strong understanding of batch, streaming, real-time, and near real-time data architectures
  • Proven experience implementing enterprise Data Governance frameworks and tool
  • Hands-on experience enabling AI/ML and Generative AI data pipelines.
  • Deep expertise in data domains including Data Warehousing, Data Modeling (Dimensional, Data Vault), MDM, Data Quality, Metadata Management, and Data Catalog implementation
  • Advanced Databricks experience including architecture design, optimization, and security; including hands-on experience with Delta Lake, Databricks SQL, Unity Catalog, Delta Live Tables, MLflow and integration with cloud services (AWS/Azure)
  • Strong hands-on experience with AWS data ecosystem (S3, Glue, EMR, Lambda, Redshift, Lake Formation, Athena, DMS, etc.)
  • Experience with real-time/streaming technologies (Kafka, Kinesis, or similar)
  • Familiarity with orchestration tools such as Apache Airflow or MWAA
  • Experience implementing CI/CD pipelines for data platforms
  • AWS, Databricks and/or Snowflake certifications preferred

Note: Given the urgency of the role, we are currently prioritizing candidates who can join immediately or within 2 weeks.


Skills Required

  • Minimum 10 years of experience in Data and Analytics Architecture
  • Leadership experience in large-scale enterprise data modernization initiatives
  • Strong understanding of batch, streaming, real-time, and near-real-time data architectures
  • Experience implementing enterprise data governance frameworks and tools
  • Hands-on experience enabling AI, machine learning, and Generative AI data pipelines
  • Expertise in data warehousing, dimensional and Data Vault modeling, MDM, data quality, metadata management, and data catalogs
  • Advanced Databricks experience, including architecture, optimization, security, Delta Lake, Databricks SQL, Unity Catalog, Delta Live Tables, and MLflow
  • Experience integrating Databricks with AWS or Azure cloud services
  • Strong hands-on experience with the AWS data ecosystem, including S3, Glue, EMR, Lambda, Redshift, Lake Formation, Athena, and DMS
  • Experience with real-time and streaming technologies such as Kafka or Kinesis
  • Familiarity with Apache Airflow or MWAA
  • Experience implementing CI/CD pipelines for data platforms
  • AWS, Databricks, and/or Snowflake certifications
  • Ability to join immediately or within two weeks
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: San Francisco, CA
Year Founded: 2022

What We Do

AuxoAI partners with enterprise leaders to build AI systems, enabling the creation of AI-first enterprises by moving from AI strategy to production-grade deployed systems in weeks.

Similar Jobs

Eli Lilly and Company Logo Eli Lilly and Company

Data Architect

Healthtech • Biotech • Pharmaceutical
In-Office
Bengaluru, Bengaluru Urban, Karnataka, IND
39451 Employees

Datamatics Technologies Logo Datamatics Technologies

Data Architect

Artificial Intelligence • Information Technology • Software • Analytics
In-Office
Bengaluru, Bengaluru Urban, Karnataka, IND
65 Employees

Optum Logo Optum

Software Engineer

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office
Bengaluru, Bengaluru Urban, Karnataka, IND
160000 Employees

Micron Technology Logo Micron Technology

Staff Engineer

Artificial Intelligence • Hardware • Information Technology • Machine Learning
In-Office
Bengaluru, Bengaluru Urban, Karnataka, IND
45000 Employees

Similar Companies Hiring

LTX Thumbnail
Robotics • Conversational AI • Generative AI
Jerusalem, Israel
200 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees
Revel.io Thumbnail
Aerospace • Hardware • Robotics • Software
US
50 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account