The Role
Modernize legacy IBM DataStage ETL/ELT workloads by re-engineering them as scalable Databricks and Spark pipelines on AWS. Responsibilities include migration assessment, pipeline development, testing, data validation, CI/CD automation, production cutover, stabilization, legacy decommissioning, documentation, and technical training. The role performs parity, unit, functional, UAT, regression, and performance testing while supporting application redeployment and operational readiness.
Summary Generated by Built In
This is a remote position.
Onsite/Hybrid/Remote: Remote
Duration: 12 months
Rate Range: $68 in W2
Work Authorization: GC and US Citizens Only
Must Have:
- IBM DataStage development
- ETL/ELT migration and modernization
- Databricks and Apache Spark on AWS
- PySpark, Python, SQL, and Shell scripting
- Delta Lake, Unity Catalog, and Photon
- AWS Glue, Lambda, and Redshift
- CI/CD and automated ETL testing
- PyTest and XML/JSON parsing
Responsibilities:
- Assess existing ETL/ELT jobs, databases, and data warehouses for migration readiness.
- Re-engineer legacy IBM DataStage jobs using Databricks Spark on AWS.
- Develop, test, and deploy scalable cloud-based data pipelines.
- Perform parity checks, unit testing, functional testing, UAT, regression testing, and performance testing.
- Validate schemas, data quality, pipeline performance, and migration accuracy.
- Build CI/CD processes for automated development, testing, and deployment.
- Support migration cutover, production deployment, and hypercare.
- Coordinate application redeployment and end-to-end testing.
- Decommission legacy DataStage jobs and related artifacts.
- Prepare technical documentation and operational readiness materials.
- Conduct knowledge-transfer and training sessions for engineering and business teams.
Qualifications:
- Strong hands-on experience developing and maintaining IBM DataStage jobs.
- Proven experience modernizing large-scale ETL/ELT and data warehouse workloads.
- Experience building Databricks and Spark pipelines in an AWS environment.
- Advanced programming skills in PySpark, Python, SQL, and Shell scripting.
- Experience with Delta Lake, Unity Catalog, and Databricks Photon.
- Experience with GitLab or Azure DevOps for CI/CD.
- Knowledge of automated testing frameworks and ETL validation methods.
- Experience supporting production cutovers, stabilization, and application teams.
Nice to Have:
- Experience with AWS Glue, Lambda, and Redshift.
- Experience with JIRA and Agile delivery.
- Knowledge of data governance and cloud security standards.
- Experience creating operational documentation and delivering technical training.
Skills Required
- Hands-on experience developing and maintaining IBM DataStage jobs
- Experience modernizing large-scale ETL/ELT and data warehouse workloads
- Experience building Databricks and Apache Spark pipelines in AWS
- Advanced programming skills in PySpark, Python, SQL, and Shell scripting
- Experience with Delta Lake, Unity Catalog, and Databricks Photon
- Experience with GitLab or Azure DevOps for CI/CD
- Knowledge of automated testing frameworks and ETL validation methods
- Experience supporting production cutovers, stabilization, and application teams
- Experience with AWS Glue, Lambda, and Redshift
- Experience with JIRA and Agile delivery
- Knowledge of data governance and cloud security standards
- Experience creating operational documentation and delivering technical training
- United States citizenship or green card work authorization
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Often, the biggest barrier between setting business objectives and achieving them is talent. Finding technically qualified people when you need them is hard enough. Finding technically qualified people who are the best fit for your organization is tougher. You need a staffing partner with the right expertise who can find the right talent in the right time frame, because your project can’t wait. That’s where we come in. aKube Inc is committed to leveraging its corporate values and operating model to achieve the highest level of performance and respect within the industry. We have developed a highly efficient delivery model for supporting a wide array of clients with expertise in supporting high-volume contingent worker programs.
