The Role
Design and architect robust ETL/data pipelines on cloud platforms. Build and automate data ingestion and preprocessing using Spark/Databricks, AWS Glue, Airflow. Implement warehousing architectures (EDW, DM, ODS, MOLAP/ROLAP), ensure SQL-based data quality, use version control, and deliver tested production pipelines while communicating with stakeholders.
Summary Generated by Built In
This is a remote position.
Roles & Responsibilities
- Strong verbal/written communication & facilitation skills. Ability to take a requirement document, work through any ambiguity and follow through to implementation independently
- Strong analytical and problem-solving abilities
- Strong work ethic, sense of ownership and a team player
- Architect and design robust ETL processes using tools like Spark, Airflow, AWS Glue etc. to desired client specifications
- Ability to analyze the data to identify the necessary pre-processing steps for the automated ETL processes
- Solid understanding of SQL using databases like MS SQL Server, Redshift, etc.
- Experience in Cloud Infrastructure (AWS, Azure, GCP)
- Delivering the data pipeline with Quality and automated testing
- Bachelor’s degree in computer science, information technology, or a related field.
- Extensive knowledge of coding languages used to build data pipelines, such as Java, Scala, and/or Python
- Experience with Databricks or Spark
- Hands on experience building data pipelines in AWS using AWS Glue, Redshift, S3 buckets and other relevant technologies.
- Proficiency in warehousing architecture techniques, including MOLAP, ROLAP, ODS, DM, and EDW.
- Proven work experience as an ETL developer.
- Experience using Source Code and Version Control systems like Git etc.
- Strong project management skills.
- Ability to analyze a company’s big-picture data needs.
- Clear communication skills.
- Ability to troubleshoot and solve complex technical problems.
- Desire to continually keep up with advancements in data best practices
Skills Required
- Bachelor's degree in computer science, information technology, or related field.
- Extensive knowledge of coding languages used to build data pipelines (Java, Scala, and/or Python).
- Experience with Databricks or Spark.
- Hands-on experience building data pipelines in AWS using AWS Glue, Redshift, and S3.
- Solid understanding of SQL using databases like MS SQL Server and Redshift.
- Proficiency in warehousing architecture techniques (MOLAP, ROLAP, ODS, DM, EDW).
- Proven work experience as an ETL developer.
- Experience using source code and version control systems like Git.
- Experience with Cloud Infrastructure (AWS, Azure, GCP).
- Experience architecting and designing ETL processes using tools like Spark, Airflow, AWS Glue.
- Delivering data pipelines with quality and automated testing.
- Strong verbal and written communication and facilitation skills.
- Strong project management skills.
- Ability to analyze big-picture data needs and troubleshoot complex technical problems.
- Desire to continually keep up with advancements in data best practices.
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Pivotal Solutions is a national staffing and consulting firm founded in 2002 that specializes in matching job seekers and companies for roles in IT & Technology and Accounting & Finance. They offer a variety of staffing solutions, including Direct Hire and Contract-to-Hire placements, serving a diverse client base that ranges from innovative startups to Fortune 500 companies across the United States.
