Senior Data Engineer
Job Requisition ID #
22WD62671
Position Overview
Autodesk seeks a senior engineer for an agile team within the Enterprise Data Management (EDM) organization, designated with designing new robust and resilient data pipelines for core datasets in customer domain. These datasets form the basis building blocks for all data scientists and data analysts throughout the organization. This role collaborates with Product Managers, Program/Project managers, Architects, Operations, and the Engineering Team to ensure data pipelines and delivery uses modern, scalable, reliable methods. Communication skills are critical, and you must be comfortable distilling complex technical information to a non-technical audience.
Responsibilities
- Solutioning and implementing data pipelines for enterprise consumption
- Mapping and automating data flows, from multiple sources
- Develop workflows to run Snowflake applications using Airflow and dbt; Apache Spark applications using Airflow and EMR
- Deploy solutions via automated CI/CD processes to execute data pipelines running in batch
- Developing, executing, and documenting software solutions to address data collection, processing, transformation, and reporting issues
- Interpreting and translating business needs into technical requirements
- Troubleshooting code level problems quickly and efficiently as needed
- Participating in code and document reviews
Minimum Qualifications
- Strong experience in several of the following: Snowflake / Snowflake SQL, dbt, Fivetran, Apache Airflow/Astronomer, Amazon EMR, Lambda, S3, Python / PySpark, Spark / Spark SQL, Hive, Hadoop
- Experience across layers of data engineering and pipelines
- Familiarity with distributed and cloud architecture patterns
- General understanding of programming paradigms, their usage, as well as data structures
- Ability to create high quality documentation and engineering artifacts (ERDs, etc)
- Effective communication and collaboration skills, understanding how to tailor your message to your audience
- General experience implementing CI/CD and familiarity with TDD/BDD
- Strong experience using data lakes and data warehouses
- Strong desire to learn and work independently after receiving general guidance and instruction
- Ability to work collaboratively across an organization
- Desire to investigate and follow best practices
- Familiarity with the roles/responsibilities of Scrum
- Eagerness to learn and share knowledge with a good attitude
At Autodesk, we're building a diverse workplace and an inclusive culture to give more people the chance to imagine, design, and make a better world. Autodesk is proud to be an equal opportunity employer and considers all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender, gender identity, national origin, disability, veteran status or any other legally protected characteristic. We also consider for employment all qualified applicants regardless of criminal histories, consistent with applicable law.
Are you an existing contractor or consultant with Autodesk? Please search for open jobs and apply internally (not on this external site). If you have any questions or require support, contact Autodesk Careers.