The Role
Architect and lead the modernization of legacy IBM DataStage ETL/ELT pipelines to scalable Databricks and AWS data platforms. Responsibilities include migration strategy, architecture standards, governance, CI/CD, testing, data quality, observability, lineage, cutover, hypercare, legacy decommissioning, documentation, and technical direction for engineering teams.
Summary Generated by Built In
This is a remote position.
Onsite/Hybrid/Remote: Remote
Duration: 12 months
Rate Range: $75 in W2
Work Authorization: GC and US Citizens Only
Must Have:
- ETL/ELT architecture and modernization
- IBM DataStage
- Databricks and Apache Spark
- Delta Lake, Unity Catalog, and Photon
- AWS Glue, Redshift, and Lambda
- Python and Unix/Linux scripting
- Terraform and CI/CD
- Large-scale database and data warehouse migrations
- Data governance, quality, and observability
Responsibilities:
- Define the architecture and migration strategy for modernizing legacy ETL and ELT pipelines.
- Assess IBM DataStage jobs, databases, and data warehouses for migration readiness.
- Design scalable data solutions using Databricks Spark on AWS.
- Establish architecture standards, reusable frameworks, and governance controls.
- Design CI/CD pipelines for automated builds, testing, and deployment.
- Lead technical design reviews, migration planning, and artifact validation.
- Define parity, functional, UAT, regression, and performance testing strategies.
- Ensure schema validation, data quality, lineage, security, and production readiness.
- Guide cutover, go-live, hypercare, and operational stabilization activities.
- Oversee the decommissioning of legacy DataStage jobs and related components.
- Create operational documentation and conduct knowledge-transfer sessions.
- Provide technical direction to developers and engineering teams.
Qualifications:
- Extensive experience designing enterprise ETL/ELT architectures.
- Hands-on experience with IBM DataStage and Databricks modernization projects.
- Strong experience with Databricks, Delta Lake, Unity Catalog, Photon, and Spark.
- Strong knowledge of AWS data services, including Glue, Redshift, and Lambda.
- Experience designing large-scale database and data warehouse migration programs.
- Proficiency in Python and Unix/Linux scripting.
- Experience with Terraform and automated deployment pipelines.
- Knowledge of data governance, observability, lineage, and operational readiness.
- Experience defining testing and production-readiness standards.
- Experience working in Agile/Scrum environments and PI planning.
Nice to Have:
- GitLab or Azure DevOps experience
- JIRA experience
- Automated data-testing framework experience
- Experience leading enterprise cutovers and hypercare activities
- Experience mentoring data engineering teams
Skills Required
- Extensive experience designing enterprise ETL/ELT architectures
- Hands-on experience with IBM DataStage and Databricks modernization projects
- Strong experience with Databricks, Delta Lake, Unity Catalog, Photon, and Apache Spark
- Strong knowledge of AWS Glue, Redshift, and Lambda
- Experience designing large-scale database and data warehouse migration programs
- Proficiency in Python and Unix/Linux scripting
- Experience with Terraform and automated deployment pipelines
- Knowledge of data governance, observability, lineage, and operational readiness
- Experience defining testing and production-readiness standards
- Experience working in Agile/Scrum environments and PI planning
- GitLab or Azure DevOps experience
- JIRA experience
- Automated data-testing framework experience
- Experience leading enterprise cutovers and hypercare activities
- Experience mentoring data engineering teams
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Often, the biggest barrier between setting business objectives and achieving them is talent. Finding technically qualified people when you need them is hard enough. Finding technically qualified people who are the best fit for your organization is tougher. You need a staffing partner with the right expertise who can find the right talent in the right time frame, because your project can’t wait. That’s where we come in. aKube Inc is committed to leveraging its corporate values and operating model to achieve the highest level of performance and respect within the industry. We have developed a highly efficient delivery model for supporting a wide array of clients with expertise in supporting high-volume contingent worker programs.
