Job Description:
- Responsible for designing and building data pipelines.
- Designing and implementing solutions and developing applications utilizing airflow, dataflow, Apache beam, snowflake or GCP components.
- Reading, extracting, transforming, staging, and loading data to multiple targets, using Hadoop, Hive, Snowflake, BigQuery, and Cloud Storage.
- Migrating existing data processing from standalone or legacy technology scripts to distributed framework processing.
- Working with gigabytes/terabytes of data and dealing with the challenges of transforming and enriching such large datasets.
- Utilizing Google Cloud Platform and strong big data technology foundation
- Designing and Architecting data model to modernize legacy data warehouses.
- Writing ELT jobs using DBT to maintain data lineage and data auditability.
- Designing architecture of CI/CD pipelines to perform continuous build and isolate multiple environments.
Experience/Qualifications
- Master’s or foreign academic equivalent in Business Analytics, Computer Science, Statistics, Data Science or any related field plus one (1) year of experience. Location: Northridge, CA, telecommuting permitted, HQ in Naperville, IL. Approx. 25% domestic travel required. Travel Expenses to be paid by employer. One year of experience with Hadoop, Spark, Pig, Hive, Snowflake, BigQuery, Kubernetes, and Dataflow required.
Similar Jobs
What We Do
Egen is a data engineering and cloud modernization firm partnering with leading Chicagoland companies to launch, scale, and modernize industry-changing technologies. We are catalysts for change who create digital breakthroughs at warp speed. Our team of cloud and data engineering experts are trusted by top clients in pursuit of the extraordinary.
Our mission is to be an enabler of amazing possibilities for companies looking to use the power of cloud and data. We want to stand shoulder to shoulder with clients, as true technology partners, and make sure they succeed at what they have set out to do. We want to be disruptors, game-changers, and innovators who have played an important part in moving the world forward.
