The Role
Design, build, and maintain Airflow DAGs and dbt transforms within a Snowflake medallion architecture. Migrate scheduled ETL into orchestrated pipelines, implement automated testing, alerting, and failure recovery across Paradime and Airflow, audit dbt for performance and cost, maintain integrations, and ensure zero downtime for downstream BI while evolving the data stack for future AI needs.
Summary Generated by Built In
AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
WHY JOIN US
If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!
ABOUT THE ROLE
We are looking for a Senior Data Engineer with deep Airflow and dbt expertise to support Medallion Architecture implementation, harden pipeline reliability through automated testing and failure alerting, and migrate scheduled ETL jobs into orchestrated Airflow DAGs within a modern Snowflake data stack. You will build complex DAG architectures using Python, audit dbt codebases for performance and cost optimization, maintain integration points between Paradime and Airflow, and ensure zero downtime for downstream BI tools during ongoing infrastructure changes. Eastern Time Zone overlap preferred for daily team standups.
WHAT YOU WILL DO
- Support implementation of bronze/silver/gold layering across the Snowflake warehouse, working with the Lead Data Engineer on architecture and modeling decisions.
- Build automated testing and failure alerting across Paradime (ETL) and Airflow (orchestration), including reworking select existing jobs so failures are clearly attributable.
- Migrate select schedule-based Paradime jobs into orchestrated Airflow DAGs.
- Work within established data governance and access controls when building or modifying pipelines touching PII data.
- Translate dbt transform schedules and dependencies into scalable, dynamic Airflow DAGs using modern Python design patterns (@task decorators, dynamic task mapping).
- Maintain and evolve integration points between Paradime and Airflow as both tools continue to be used in production.
- Audit dbt codebases and pipeline execution paths to identify bottlenecks, reducing compute runtime and cost.
- Partner with the Lead Data Engineer to adjust the modern data stack architecture to support future AI semantic layers and downstream consumption.
- Establish and maintain automated error handling, alerting, and failure recovery routines across the pipeline stack.
- Ensure no downtime or data loss for downstream BI (Sigma) and analytics tools during ongoing infrastructure changes.
- Take on additional data engineering priorities as assigned, based on evolving business needs.
MUST HAVES
- 4+ years of experience as a Data Engineer.
- Apache Airflow / Astronomer: Substantial hands-on production experience building, debugging, and managing complex DAG architectures using Python.
- dbt (Data Build Tool) expertise: Strong proficiency with dbt Core architecture, macros, packages, custom materializations, CI/CD patterns (state deferral), and environment configurations.
- Advanced Git & Data CI/CD: Track record building deployment pipelines and automated regression testing for data teams using GitHub Actions or similar.
- Production Data Engineering: Solid general software/data engineering background writing clean, maintainable, modular Python and SQL.
- Upper-intermediate English level.
NICE TO HAVES
- Snowflake expertise: tuning compute (warehouse sizing, query profiling, clustering), RBAC/security architecture.
- Infrastructure as Code: Terraform, Docker, or CloudFormation.
- AWS S3 experience as part of a data pipeline (e.g., staging/raw ingestion layers).
- Enterprise retail, supply chain, or inventory systems data background.
PERKS AND BENEFITS
- Growth without limits: build your skills through mentorship, internal TechTalks, challenging projects, and a dedicated annual learning budget
- Competitive compensation: get recognition that reflects your skills and impact, with regular performance and compensation reviews
- Flexibility: work 100% remotely with flexible hours that support focus, autonomy, and a healthy work rhythm
- Meaningful, modern projects: build impactful products using modern technologies alongside global teams and leading brands
- Collaborative culture: join a supportive environment with zero micromanagement where ideas are welcomed and contributions are recognized
- Well-being & support: access local well-being programs and people-focused support tailored to your location
Skills Required
- 4+ years of experience as a Data Engineer
- Production experience building, debugging, and managing complex Apache Airflow (Astronomer) DAG architectures using Python
- dbt Core expertise including macros, packages, custom materializations, CI/CD patterns, and environment configurations
- Advanced Git and Data CI/CD experience (GitHub Actions or similar) for deployment pipelines and automated regression testing
- Production data engineering skills writing clean, maintainable, modular Python and SQL
- Experience maintaining and integrating Paradime ETL with orchestration tooling (Airflow)
- Upper-intermediate English level
- Snowflake expertise (warehouse sizing, query profiling, clustering, RBAC/security)
- Infrastructure as Code: Terraform, Docker, or CloudFormation
- AWS S3 experience as part of a data pipeline
- Enterprise retail, supply chain, or inventory systems data background
- Eastern Time Zone overlap for daily standups (preferred)
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
AgileEngine is a privately held company established in 2010 that builds dedicated teams of designers and developers. We turn good ideas into awesome software that people actually want to use. Some of the biggest names and the hottest startups around the world chose us to build their tech.
