The Role
Designs, builds, and maintains scalable cloud data platforms and production pipelines across Azure, Databricks, Snowflake, and SAP. Responsibilities include ingestion, ETL/ELT, Lakehouse and dimensional modeling, CI/CD with Databricks Asset Bundles and Azure DevOps, data quality and observability, performance optimization, documentation, and stakeholder collaboration. The role also includes technical architecture, client advisory, engineering leadership, mentoring, and delivery of production Generative AI systems.
Summary Generated by Built In
This is a remote position.
Job Description We are seeking a Technical Architect with 10+ years of overall technology experience, including proven experience architecting and delivering production Generative AI systems. The ideal candidate will combine deep hands-on command of large language models, retrieval-augmented generation, and agentic architectures with the judgement to design AI solutions that hold up under real enterprise constraints of cost, latency, security, and compliance. This role owns the technical architecture of AI engagements end-to-end: shaping the solution during discovery and pre-sales, defining reference architectures and integration patterns, selecting models and platforms, and guiding delivery teams through implementation. You will work directly with client stakeholders, product managers and engineering leads, and will help set the technical standards of our AI practice. This is a senior, hands-on architecture role rather than a purely advisory one. This role owns multiple concurrent AI engagements from discovery through production. Beyond technical architecture, the Technical Architect is expected to act as a trusted advisor to clients, lead engineering teams, mentor future technical leaders, and drive engineering excellence through hands-on involvement
Requirements
Job Description
We are looking for a well-rounded Senior Data Engineer to proactively design, build and maintain modern cloud data platforms end to end. The role spans the full data engineering lifecycle - ingestion, transformation, storage, modeling and delivery - with strong hands-on expertise across Azure, Databricks and Snowflake, fully automated CI/CD, and integration of enterprise source systems including SAP. It requires cross-functional collaboration and ensures the highest standards of data quality, reliability and performance.
Responsibilities
- Understand the values and vision of the organization
- Protect the Intellectual Property
- Adhere to all the policies and procedures
- Design, develop, and maintain scalable data pipelines for data ingestion, processing and storage.
- Build and optimize data architectures and data models (Lakehouse / medallion, dimensional) for efficient data storage and retrieval.
- Develop ETL/ELT processes to transform and load data from various sources into data warehouses and data lakes.
- Build and orchestrate pipelines on Azure Databricks using PySpark, Spark SQL and Delta Lake, orchestrated with Databricks Workflows.
- Integrate data from enterprise source systems including SAP (ABAP/CDS extracts, RPA/CSV or connectors) and load into Snowflake and Databricks.
- Own end-to-end CI/CD for data pipelines using Databricks Asset Bundles (DAB) and Azure DevOps (Git repositories, YAML build and release pipelines), promoting code across dev, QA and production.
- Implement data quality, validation, freshness and reconciliation checks with pipeline observability.
- Ensure data integrity, quality, and security across all data systems.
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.
- Monitor and troubleshoot data pipelines and workflows to ensure high availability and performance.
- Document data processes, architectures, and data flow diagrams.
Job
- 7 - 8 years of hands-on data engineering experience building and running production data pipelines at scale.
- Strong expertise in Azure and Azure data services (ADLS Gen2, Azure Databricks, Azure DevOps).Deep hands-on experience with
- Databricks: PySpark, Spark SQL, Delta Lake, Lakehouse / medallion architecture and Databricks Workflows.
- CI/CD for data engineering using Databricks Asset Bundles (DAB) and Azure DevOps (Git, YAML build/release pipelines, multi-environment promotion).
(Must-have)
- Strong Snowflake experience (data modeling, performance tuning, loading and optimization).
- Proficiency in SQL and Python.
- Experience integrating data from SAP and other enterprise ERP / source systems into a data lake or warehouse.
- Solid data modeling (dimensional, star/snowflake, Lakehouse) and ETL/ELT design.
- Building data-quality, validation, reconciliation and pipeline monitoring / observability.
- Excellent communication and interpersonal skills, with the ability to engage with all levels of employees and management.
- Collaborative approach to effectively present and advocate for quick design solutions.
- Stay updated on the latest design trends, tools, and technologies, bringing innovative ideas to enhance the product experience.
- A proactive approach to problem solving, with a focus on delivering exceptional customer satisfaction.
At least one current certification is required; multiple is a strong plus:
- Databricks Certified Data Engineer Associate or Professional. (Required)
- Microsoft Certified: Azure Data Engineer Associate (DP-203), or Azure Fundamentals (DP-900 / AZ-900).
- SnowPro Core or SnowPro Advanced: Data Engineer (Snowflake).
Job
- Familiarity with SAP finance / ERP data domains (Accounts Receivable, invoice-to-pay, bank statements).
- Streaming and event-driven pipelines (Apache Kafka / Azure Event Hubs).
- Workflow orchestration (Apache Airflow, Databricks Workflows).
- Databricks Unity Catalog and Delta Live Tables.
- Infrastructure as Code (Terraform) and containerization (Docker).
- Data governance, lineage and cost/performance optimization on Databricks and Snowflake.
- Exposure to BI / visualization tools (Power BI, Tableau).
- Demonstrate proactive thinking
- Strong communication and collaboration skills.
- Should have strong interpersonal relations, expert business acumen and mentoring skills
- Strong problem-solving skills with attention to detail.
- Have the ability to work under stringent deadlines and demanding client conditions.
- Strong analytical and problem-solving skills.
- Ability to work independently and as part of a team.
- Bachelor's degree in Computer Science, Information Technology, or a related field.
- 7 - 8 years of experience in data engineering & architecture, including hands-on Databricks and Azure DevOps CI/CD.
Benefits
- This role offers the flexibility of working remotely in India.
Skills Required
- 7–8 years of hands-on data engineering experience building and operating production data pipelines at scale
- Strong expertise in Azure data services, including ADLS Gen2, Azure Databricks, and Azure DevOps
- Hands-on Databricks experience with PySpark, Spark SQL, Delta Lake, Lakehouse or medallion architecture, and Databricks Workflows
- Experience implementing CI/CD for data engineering with Databricks Asset Bundles and Azure DevOps Git and YAML pipelines
- Strong Snowflake experience in data modeling, performance tuning, loading, and optimization
- Proficiency in SQL and Python
- Experience integrating SAP and other enterprise ERP or source systems into data lakes or warehouses
- Experience with dimensional, star, snowflake, and Lakehouse data modeling and ETL/ELT design
- Experience building data-quality, validation, reconciliation, monitoring, and observability processes
- At least one current certification, including Databricks Certified Data Engineer Associate or Professional
- Bachelor’s degree in Computer Science, Information Technology, or a related field
- 10+ years of overall technology experience and production Generative AI architecture experience
- Familiarity with SAP finance or ERP data domains
- Experience with streaming and event-driven pipelines using Apache Kafka or Azure Event Hubs
- Experience with Apache Airflow, Unity Catalog, Delta Live Tables, Terraform, Docker, Power BI, or Tableau
- Azure Data Engineer, Azure Fundamentals, or SnowPro certification
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Headquartered at San Francisco and founded in 2007, LeewayHertz is one of the first few companies to build and launch a commercial app on Apple's App Store. Our team of certified designers and developers has designed and developed more than 100 digital platforms on Mobile, Cloud, AI, IoT and Blockchain. At LeewayHertz, we have developed digital solutions for Fortune 500 companies and startups to ease their business functions with the latest technologies. Some of our reputed clients include ESPN, NASCAR, Hershey's, McKinsey, P&G, Siemens, 3M, Pearson and more. Being an award-winning software development company, we have also proven our expertise in blockchain development and worked on more than 20+ blockchain projects. We have created a workforce of blockchain developers who can build blockchain apps on different blockchain platforms such as Ethereum, Hyperledger Fabric, Hyperledger Sawtooth, Hyperledger Iroha, Hyperledger Indy, EOS, Stellar, Tron and Corda. We design, develop, deploy and maintain technology products. Uber and Twitter are using our inventions and patents. We work with tech geeks and passionate technologists who are trained by the experts at Apple and Google and always remains at the cutting edge of technology. If you meet this criterion, join us at www.leewayhertz.com
.png)
