The Role
Support a data platform migration from Amazon Redshift to Databricks by migrating tables, views, and stored procedures; build and maintain Databricks pipelines/jobs; work with AWS services (S3, Glue, Athena); enforce data governance, lineage, cataloging, and RBAC; collaborate with stakeholders during a 5-6 month engagement.
Summary Generated by Built In
AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
WHY JOIN US
If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!
ABOUT THE ROLE
We are looking for a Data Engineer to support a strategic data platform migration, moving tables, views, and stored procedures from Amazon Redshift to Databricks. You will build and support Databricks pipelines and jobs, work with AWS services including S3, Glue, and Athena, and apply data governance practices including data lineage, cataloging, and role-based access controls throughout the migration. The role requires 3–5 years of data engineering experience. This is a 5–6 month project engagement.
WHAT YOU WILL DO
- Migrate tables, views, and stored procedures from Amazon Redshift to Databricks.
- Build and support Databricks pipelines and jobs.
- Work with AWS services, including Amazon S3, AWS Glue, and Amazon Athena.
- Support data migration activities within a modern data lake architecture.
- Apply data governance and data management practices throughout the migration.
- Collaborate with project stakeholders to support migration activities.
MUST HAVES
- 3–5 years of experience as a Data Engineer.
- Hands-on experience with Databricks.
- Experience with Amazon Redshift.
- Strong SQL skills.
- Experience with AWS services, including Amazon S3, AWS Glue, and Amazon Athena.
- Experience building and supporting Databricks pipelines and jobs.
- Understanding of data lake architectures.
- Familiarity with data governance and data management practices, including data lineage, data catalogs, and role-based access controls (RBAC).
- Upper-intermediate English level
NICE TO HAVES
- Experience with Databricks Lakebridge transpiling capabilities.
PERKS AND BENEFITS
- Growth without limits: build your skills through mentorship, internal TechTalks, challenging projects, and a dedicated annual learning budget
- Competitive compensation: get recognition that reflects your skills and impact, with regular performance and compensation reviews
- Flexibility: work 100% remotely with flexible hours that support focus, autonomy, and a healthy work rhythm
- Meaningful, modern projects: build impactful products using modern technologies alongside global teams and leading brands
- Collaborative culture: join a supportive environment with zero micromanagement where ideas are welcomed and contributions are recognized
- Well-being & support: access local well-being programs and people-focused support tailored to your location
Skills Required
- 3-5 years of experience as a Data Engineer
- Hands-on experience with Databricks
- Experience with Amazon Redshift
- Strong SQL skills
- Experience with AWS services, including Amazon S3, AWS Glue, and Amazon Athena
- Experience building and supporting Databricks pipelines and jobs
- Understanding of data lake architectures
- Familiarity with data governance and data management practices, including data lineage, data catalogs, and role-based access controls (RBAC)
- Upper-intermediate English level
- Experience with Databricks Lakebridge transpiling capabilities
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
AgileEngine is a privately held company established in 2010 that builds dedicated teams of designers and developers. We turn good ideas into awesome software that people actually want to use. Some of the biggest names and the hottest startups around the world chose us to build their tech.
