Data Engineer

Duration: 12 months Contract

Evaluation: Performance will be reviewed after 8 months to assess the potential for a permanent role.

As BPM continues to build a data-driven culture, the Data Engineer will be supporting the continued success of that effort. For BPM, the culture of data represents the firm’s approach to data management, stewardship, lineage, architecture, collection, storage, and utilization for delivering analytic results. It also includes tools, applications and approaches which lead the firm to have access to timely and accurate data. This role will play an integral part in the delivery, maintenance, building of business relationships and success to the culture of data.

As member of the Business Insights and Analytics team in the Business Transformation Office, this Data Engineer role is continuing the build, maintenance, and governance of data pipelines with best of breed technology in an Azure environment. This position is responsible for developing the pipelines to the data Lakehouse while ensuring confidence with the business and executive stakeholders by developing a reliable, scalable data solution that is high quality, secure, current, and usable to generate insights and support decision making. The Data Engineer stays up to date with the latest technology and methodologies with a pragmatic mindset. Participates in technology road maps and maintains the data pipeline and tool documentation.

· Undergraduate degree in the field of data or computer science, information technology, statistics, or mathematics preferred.

· Minimum of 2 years work experience as a Data Engineer working in a Databricks environment with specific expertise in Databricks Delta Lake, notebooks, and clusters.

· Data Vault Modeling required.

Experienced in developing, deploying, and supporting high-quality, fault-tolerant data pipelines.

Builds the infrastructure required for optimal extraction, loading, and transformation of data from a wide variety of data sources using best practices.

Supports the architecture for observing, cataloging, and governing data.

Knowledge of big data technologies and tools, such as Hadoop, Spark, and Kafka.

Strong understanding of relational data structures, theories, principles, and practices.

Hands-on experience in ETL functionality by way of coding (Python, DbT, SQL).

Build and optimize ETL.

Monitor and troubleshoot ELT processes to ensure data accuracy and reliability.

Monitors and analyzes performance metrics and recommends improvements.

Proficiency in programming languages such as Python and SQL.

Strong understanding of data modeling, algorithms, and data transformation strategies and techniques for data science consumption.

Ensures security and integrity of data.

Implement data governance and access controls to ensure data security and compliance.

Collaborate with the security team to implement encryption, authentication, and authorization mechanisms.

Monitor and audit data access to maintain data privacy and integrity.

Proven experience with business and technical requirements analysis, elicitation, modeling, verification, and methodology development.

Ability to create systematic and consistent requirements specifications in both technical and user-friendly language.

Collaborates with the team as well as key stakeholders across the organization.

Understands the need to profile the data source to effectively send the data to the Lakehouse.

Utilizes development best practices including technical design reviews, implementing test plans, monitoring/alerting, peer code reviews, and documentation.

Ability to communicate with both technical and business end-user colleagues to deliver meaningful outcomes.

Excellent critical thinking skills and the ability to understand the relationships between data and business intelligence.

Ability to work with the Information Technology Department in partnership.

Fully remote position

Sitting, and typing (repetitive motion) are strong components of the role.

Ability to work flexible schedule (am, pm & weekends as needed)

Minimal travel might be required.