Data Engineer III

JOB SUMMARY

The Senior Data Engineer is responsible for leading the design, development, and maintenance of scalable data infrastructure and pipelines to meet business needs. The Senior Data Engineer will collaborate with cross-functional teams to gather and analyze requirements, architect data solutions, and implement best practices to ensure high-quality data engineering. Produces high-quality, efficient, and well-documented pipelines and data models. Implements and maintains automated testing and CI/CD practices. Leads by example in code reviews and data architecture decisions.

KEY RESPONSIBILITIES

• Lead the design, development, and implementation of scalable data pipelines and infrastructure using Azure Databricks, Azure Data Factory, SQL Server, Salesforce integrations, and related technologies
• Develop and maintain both OLTP and dimensional data models based on business requirements, applying Kimball and Inmon methodologies as appropriate
• Optimize data storage and retrieval performance across SQL Server and cloud data platforms
• Integrate data from Salesforce and other SaaS applications via APIs and extraction workflows
• Implement and maintain batch and real-time data processing architectures
• Participate in code and architecture reviews to ensure adherence to data engineering standards and best practices
• Analyze and troubleshoot data pipeline and performance issues, providing timely and effective solutions
• Contribute to the continuous improvement of data engineering practices, tools, and processes
• Stay current with emerging trends in cloud data platforms and identify opportunities for innovation
• Champion comprehensive documentation of data architecture, lineage, and modeling decisions
• Facilitate knowledge sharing and mentorship across the team; engage cross-functionally to align on architectural vision
• Drive adoption of best practices including data governance, quality, security, and compliance standards
• Lead initiatives to reduce technical debt and improve pipeline performance and cost efficiency

ROLE QUALIFICATIONS

EDUCATION & EXPERIENCE

REQUIRED

• Bachelor's degree in Computer Science, Information Systems, Engineering, or related field
• At least five (5) years of experience in data engineering or a related field
• Three (3) or more years of hands-on experience with Azure cloud services including Azure Databricks, Azure Data Factory, and Azure Data Lake
• Two (2) or more years of specific experience with the Databricks platform, including ETL/ELT pipeline development, cluster management, and data workflow implementation
• Advanced proficiency with SQL Server including database administration, performance tuning, and stored procedures
• Strong proficiency in Python and/or Scala for data processing and automation
• Demonstrated expertise in conceptual, logical, and physical data modeling including star schema, snowflake schema, SCD Types 1/2/3, and medallion architecture patterns
• Experience with Salesforce data architecture and API-based data extraction
• Proficiency with Git workflows and CI/CD pipeline development
• Strong understanding of data engineering principles, data quality, and best practices
• Excellent written and verbal communication and interpersonal skills

PREFERRED

• Experience with streaming data platforms such as Azure Event Hubs or Apache Kafka
• Familiarity with data governance tools and data privacy/compliance regulations
• Knowledge of MLOps and data science workflows
• Experience with additional cloud platforms such as AWS (Redshift, Glue) or GCP (BigQuery, Dataflow)
• Experience with Sigma Computing for cloud-native business intelligence and data exploration on top of cloud data warehouses

KEY COMPETENCIES

• Results-Oriented: ability to plan, schedule, and organize work to achieve strategic goals within or ahead of established time frames
• Adaptability to Change: ability to be flexible and supportive, react swiftly to, and positively assimilate change in a rapid growth environment
• Interpersonal Communication: ability to choose communication behaviors that are both appropriate and effective for a given situation; ability to understand and manage emotions and influence outcomes
• Team Orientation and Collaboration: ability to build and maintain collaborative relationships through shared responsibility, respect, and empathy to achieve common goals
• Accountability: ability to act with a clear sense of ownership; takes personal responsibility for decisions, actions, deliverables, and failures; embraces experimentation, creativity, and positive change
• Cultural Competence: ability to understand and respect values, attitudes, beliefs, and perspectives that differ across cultures, and to respond appropriately in all professional interactions