Senior Data Engineer

As part of the overall analytics capability for the company, the Senior Data Engineer develops and manages data pipelines, ELT (extract/load/transform) processes, and data storage solutions that ensure data quality, consistency, and availability. The impact that a Senior Data Engineer has on the company is to ensure that the organization is able to leverage its data to its full potential, making it a competitive advantage. They should have a deep understanding of the organization's business objectives and data needs, and work closely with other teams to ensure that data is being used effectively to drive business outcomes.  

Performance Outcomes 

Data Pipelines; Data-focused Development of Applications/Systems   

The Senior Data Engineer is well-versed in the process of developing quality data feature, data pipeline, data warehouse, and data science and will have clear areas of expertise within their area of focus. They may begin to develop secondary areas of expertise and are a regular contributor to architectural and design decisions of their team, extending to the entire company. They can work on complex systems independently and be trusted. The scope of work for the Senior Data Engineer continues to be strongly rooted at the product level. They consistently look to provide better design solutions to simplify, optimize existing, or build/implement new systems. Their work is focused beyond the individual task or ticket, solving broader problems in anticipation of requirements. They contribute increasingly more to defining the requirements, architecture, and design of what they are building.  

Individual tasks include:  

• Build and Maintain Data Pipelines: assist to develop data pipelines in a consumer and producer pattern to move data in and out of the data lake and warehouse using a variety of technologies that include Apache Airflow, Python, SQL, Postgres, Snowflake, and API endpoints

• Structure Data and Queries for Performance: work with senior members of the Data Engineering team to partner with data analysts, data scientists, and business intelligence developers to model and sanitize data for algorithm development, reporting, performance, and cost management 

• Build and Maintain the Data Dictionary: learn and help to document, extend and maintain the data dictionary and grow your domain expertise using tools such as Atlan, Alation and DataHub

• Engage in ongoing personal study of modern technologies and industry-related knowledge and skills 

Engagement and Collaboration 

As a senior member of data engineering, the Senior Data Engineer is often consulted by other members of the team in approaches and will begin to contribute to common patterns of engineering at the company. They provide their opinions and judgment on tooling and frameworks and have strong rationale for it. They typically take on leadership or ownership of certain projects for the team and will help train and mentor other members in the team. They may also conduct cross-team presentations and work closely with business teams.  

Engagement responsibilities include:  

• Partner effectively with teammates, which may include fellow data engineers, software engineers, program/product managers, or others 

• Contribute to the implementation of dynamic requirements/initiatives with teammates, cross-team collaborators  

• Actively participate in and help lead team meetings and activities  

• Help train and mentor new and more junior teammates  

Education, Knowledge, and Experience 

• 5-10 years of industry experience in data engineering  

• Bachelor’s degree in Data Science, Computer Science, or related field, or equivalent industry experience  

• 5+ years of experience in the data warehouse space, and in writing and analyzing SQL statements 

• 3+ years of experience in schema design and dimensional data modeling 

• 3+ years of experience in ELT/ETL design, implementation, testing, and maintenance 

• Experience with large-scale cloud-based data warehouses like Snowflake, Google’s BigQuery, or AWS Redshift 

• Experience with Python or other modern scripting languages 

• Experience managing real-time and streaming data producers 

• Experience leveraging APIs, databases, and streams as data producers

• Experience with orchestration tools such as Apache Airflow, Dagster and Prefect

• Willingness to learn new approaches and technologies 

• Ability to work well with other engineers in a collaborative environment 

• Attention to detail and quality 

• Open-minded approach to new ideas and approaches 

• Willingness to challenge others and be challenged 

• Share in the goal of creating the best design and architecture possible 

FLSA Status 

Exempt  

Physical Requirements/ Work Environment  

Employee must be able to sit or stand for long periods of time, with the physical ability to work at the computer or other sedentary tasks for long periods of time. Employee can conduct duties discreetly and impartially. If working remotely, employee is able to work in a space that allows them to effectively complete their job tasks, including having reliable internet connectivity and the ability to participate effectively while on phone and video calls. Employee has regular and predictable attendance and punctuality.