The Role
Build and optimize scalable ELT and data transformation pipelines using Snowflake, dbt, Azure Databricks, PySpark, Spark SQL, Delta Lake, and Matillion. Responsibilities include data ingestion, modeling, orchestration, performance tuning, testing, documentation, CI/CD, monitoring, and production support. Collaborate with analysts, data scientists, architects, and client teams in a consulting environment while contributing to knowledge sharing and Agile delivery.
Summary Generated by Built In
Role Overview
We are looking for an experienced and results-driven Data Engineer to join our growing Data Engineering practice. The ideal candidate will be proficient in building scalable, high-performance data transformation pipelines using Snowflake, Azure Databricks, and dbt, and will thrive in Techdome's fast-paced, client-facing consulting environment. In this role, you will be instrumental in ingesting, transforming, and delivering high-quality data to enable data-driven decision-making across our clients' organizations — owning your deliverables like a true Tech Doctor.
Key Responsibilities- Design, configure, and optimize ingestion, transformation, and orchestration workflows using Matillion DPC where applicable.
- Design and implement scalable ELT pipelines using dbt on Snowflake, following industry-accepted best practices.
- Build and maintain data processing workloads on Azure Databricks (PySpark/Spark SQL), including notebooks, jobs, Delta Lake tables, and Lakehouse/medallion architectures.
- Build ingestion pipelines from various sources including relational databases, APIs, cloud storage, and flat files into Snowflake and Databricks (Delta Lake).
- Implement data modelling and transformation logic to support layered architecture (staging, intermediate, and mart layers, or medallion architecture) to enable reliable and reusable data assets.
- Leverage orchestration tools (e.g., Airflow, dbt Cloud, Azure Data Factory, or Databricks Workflows) to schedule and monitor data pipelines.
- Apply dbt best practices: modular SQL development, testing, documentation, and version control.
- Perform performance optimizations in dbt/Snowflake/Databricks through clustering, query profiling, materialization, partitioning, caching, and efficient SQL/Spark design.
- Apply CI/CD and Git-based workflows for version-controlled deployments.
- Take complete ownership of assigned pipelines and deliverables — from design through production support — in line with Techdome's ownership culture.
- Contribute to Techdome's internal knowledge base of dbt macros, Databricks patterns, conventions, and testing frameworks; participate in internal tech talks and knowledge-sharing sessions.
- Collaborate with data analysts, data scientists, and data architects across onshore and offshore teams to understand requirements and deliver clean, validated datasets.
- Write well-documented, maintainable code using Git for version control and CI/CD processes.
- Participate in Agile ceremonies including sprint planning, stand-ups, and retrospectives.
- Support consulting engagements through clear documentation, demos, and delivery of client-ready solutions that reflect Techdome's commitment to excellence.
- 1 to 4 years of experience in data engineering roles, with 6+ months of hands-on experience in Snowflake and dbt.
- Mandatory: Hands-on experience with Azure Databricks — developing and deploying pipelines using PySpark/Spark SQL, Delta Lake, notebooks, clusters, and Databricks Workflows/Jobs in a production environment.
- Hands-on experience with Matillion Data Productivity Cloud (Matillion DPC) for data ingestion, transformation, or orchestration.
- Experience building and deploying dbt models in a production environment.
- Expert-level SQL and strong understanding of ELT principles; strong understanding of ELT patterns and data modelling (Kimball/Dimensional preferred).
- Familiarity with data quality and validation techniques: dbt tests, dbt docs, etc.
- Experience with Git, CI/CD, and deployment workflows in a team setting.
- Familiarity with orchestrating workflows using tools like dbt Cloud, Airflow, Azure Data Factory, or Databricks Workflows.
- Building robust and modular data pipelines using dbt and Databricks.
- Writing efficient SQL and PySpark for data transformation and performance tuning in Snowflake and Databricks.
- Managing environments, sources, and deployment pipelines in dbt.
- Strong proficiency with Snowflake: warehouse sizing, query profiling, data loading, and performance optimization.
- Strong proficiency with Azure Databricks: cluster configuration and sizing, Delta Lake optimization (OPTIMIZE, Z-ORDER, vacuum), Unity Catalog basics, and Spark job tuning.
- Experience working with cloud storage (Azure Data Lake, AWS S3, or GCS) for ingestion and external stages.
- Python / PySpark: For data transformation, notebook development, and automation on Databricks.
- SQL: Strong grasp of SQL for querying and performance tuning.
- Knowledge of modern data architecture concepts including layered architecture (staging → intermediate → marts) and Lakehouse/Medallion architecture.
- Familiarity with data quality, unit testing (dbt tests), and documentation (dbt docs).
- Understanding of access control within Snowflake (RBAC) and Databricks (Unity Catalog/workspace permissions), role hierarchies, and secure data handling.
- Familiarity with data privacy policies (GDPR basics) and encryption at rest/in transit.
- Version control using Git; experience with CI/CD practices in a data context.
- Monitoring and logging of pipeline executions, alerting on failures.
- Ownership mindset: take full responsibility for your work, from problem diagnosis to production delivery.
- Client-first communication: ability to present solutions confidently and handle client demos and discussions.
- Collaboration: work closely with onshore and offshore teams of analysts, data scientists, and architects — every voice is heard and respected.
- Ability to document pipelines and transformations clearly.
- Basic SSIS and Matillion.
- Comfort with ambiguity, competing priorities, and fast-changing client environments — we move fast and adapt faster.
- Passion for continuous learning and knowledge sharing (tech talks, meetups, internal upskilling).
- Experience in client-facing roles or consulting engagements.
- Exposure to AI/ML data pipelines and feature stores.
- Exposure to MLflow for basic ML model tracking (native to Databricks).
- Experience/exposure using data quality tooling.
- Certifications such as Snowflake SnowPro, Databricks Certified Data Engineer Associate, or dbt Certified Developer are a plus.
- Work on high-impact projects for global clients — from startups to Fortune 500 enterprises — across diverse industries.
- Be part of an inclusive culture that promotes open communication, teamwork, and respect for every voice.
- Grow with a company that invests in your professional development through mentorship, tech events, and hands-on exposure to cutting-edge data and AI technologies.
- Join a journey driven by insight and innovation — where we don't just solve problems, we push the boundaries of technology.
Skills Required
- 1 to 4 years of experience in data engineering roles
- At least 6 months of hands-on Snowflake and dbt experience
- Production experience developing and deploying Azure Databricks pipelines using PySpark or Spark SQL, Delta Lake, notebooks, clusters, and Databricks Workflows or Jobs
- Hands-on experience with Matillion Data Productivity Cloud for ingestion, transformation, or orchestration
- Experience building and deploying dbt models in production
- Expert-level SQL and strong understanding of ELT principles and data modeling
- Familiarity with data quality and validation techniques, including dbt tests and dbt docs
- Experience with Git, CI/CD, and team-based deployment workflows
- Familiarity with workflow orchestration using dbt Cloud, Airflow, Azure Data Factory, or Databricks Workflows
- Strong proficiency with Snowflake warehouse sizing, query profiling, data loading, and performance optimization
- Strong proficiency with Azure Databricks cluster configuration, Delta Lake optimization, Unity Catalog basics, and Spark job tuning
- Experience with cloud storage such as Azure Data Lake, AWS S3, or GCS
- Understanding of Snowflake RBAC, Databricks Unity Catalog or workspace permissions, secure data handling, and basic GDPR concepts
- Ability to monitor and log pipeline executions and alert on failures
- Basic SSIS and Matillion knowledge
- Client-facing or consulting engagement experience
- Exposure to AI/ML data pipelines and feature stores
- Exposure to MLflow for basic model tracking
- Experience with data quality tooling
- Snowflake SnowPro, Databricks Certified Data Engineer Associate, or dbt Certified Developer certification
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Techdome is a technology consultancy and IT solutions provider founded in 2020. It helps businesses modernize operations and navigate digital transformation through scalable, end-to-end services spanning planning, building, designing, developing, and launching technology solutions, alongside AI, cloud computing, automation, data engineering, machine learning, custom large language models, and cybersecurity. Its mission is to make complex technology accessible and support business growth across diverse industries and markets.
