Lead Data Engineer

This role can be based at any of the following Shift4 locations: Center Valley, PA / Tampa, FL / Morrisville, NC / Las Vegas, NV / Atlanta, GA   Relocation assistance may be available. 

Responsibilities:

Shift4 is expanding globally and building out our data engineering team, seeking an experienced Lead Data Engineer with a minimum of 7 years of overall experience to include 1 year of leadership to spearhead our data architecture, pipeline development, and data lake / data warehousing efforts. This role involves not just technical expertise but also project leadership in guiding data strategies that empower our business decisions and initiatives.

• Data Architecture: Design, implement, and manage our data infrastructure, ensuring scalability, efficiency, and security. This includes defining data schemas, data pipelines, and data storage solutions.
• Lead data engineers, setting standards for data handling, integration, and warehousing.
• Mentor junior team members, fostering a culture of continuous learning and improvement.
• Develop robust ETL (Extract, Transform, Load) processes to ingest, process, and transform data from various sources.
• Optimize data flow and storage for speed, scalability, and reliability.
• Establish and enforce data quality standards and governance policies.
• Implement data validation processes to ensure data integrity.
• Collaborate with data scientists and analysts to tailor data solutions that meet analytical requirements.
• Support the creation of BI tools, dashboards, and reports by providing clean, structured data.
• Research and recommend new technologies and tools to improve data management and processing.
• Drive the adoption of best practices in data engineering, like data lake architectures, streaming data, and real-time analytics.
• Oversee data-related projects from conception to completion, including planning, execution, and delivery.
• Ensure project timelines, quality, and objectives are met.
• Work closely with business stakeholders, product managers, and other engineering teams to align data strategy with business goals.

Qualifications:

• 7+ years of experience in data engineering or a similar discipline.
• 1+ years of experience in project leadership and leading data engineers.
• Proficiency in SQL and experience with big data technologies (e.g., Hadoop, Spark).
• Strong coding skills in languages like Python, Java, Scala, Node JS, Golang, etc…
• Experience with cloud services (AWS, Azure, Google Cloud) for data solutions.
• Knowledge of data warehousing solutions (e.g., Snowflake, Redshift, BigQuery).
• Familiarity with data modeling, ETL tools, and data pipeline orchestration tools (e.g., Airflow, Luigi).
• Understanding of machine learning pipelines and feature engineering is a plus.
• Excellent analytical and problem-solving skills.
• Strong communication skills to effectively articulate complex data concepts to non-technical stakeholders.
• Certifications: AWS Certified Big Data - Specialty, Google Professional Data Engineer, or similar would be a plus to have.
• BS or MS in Computer Science, Data Science, Information Technology or related field - experience in lieu of a degree may be considered. 

#LI-BN1