Data Engineer II

Key Responsibilities (Level 2 Operational Support)

• Resolve data pipeline issues and perform proactive monitoring for sensitive and critical batch processes. 
• Conduct Root Cause Analysis (RCA) and retrospectives for incidents and operational issues. 
• Create, update, and maintain operational runbooks and manage entitlements as required. 
• Implement configuration changes to batch processes and underlying cloud infrastructure. 
• Manually control and manage pipeline executions during scheduled maintenance windows. 
• Perform on-demand operational changes based on instructions provided by the Data Engineering (L3) team. 
• Handle user service requests; for non-runbook-based requests, coordinate effectively with the L3 team. 
• Participate in Disaster Recovery (DR) testing activities (twice a year). 
• Support development and production data synchronization activities. 
• Represent the team in weekly change management board meetings. 

Knowledge, Skills, and Abilities

• Strong hands-on experience with: 
• Azure Data Lake Storage 
• Azure Databricks 
• Azure Data Factory 
• Azure Synapse Analytics 
• Snowflake 
• Azure Event Hub 
• Tidal Scheduler 
• Pyramid, Collibra 
• Power BI, Tableau, MicroStrategy (MSTR), Alteryx 
• Ability to read, write, and troubleshoot SQL queries and stored procedures. 
• Proven experience in configuring, managing, and troubleshooting cloud data platforms. 
• Strong analytical and problem-solving skills. 
• Excellent written and verbal communication skills, with the ability to clearly explain technical concepts to non-technical stakeholders. 
• Ability to work effectively in small teams while promoting collaboration and inclusiveness. 

Job Complexity

• Requires extensive problem-solving capabilities, including independent investigation of issues and identification of root causes. 
• Ability to research and utilize multiple information sources, including Microsoft and vendor documentation, to resolve complex issues. 
• Must be capable of prioritizing multiple incidents and tasks while managing competing deadlines. 

Experience and Education

• Bachelor’s degree in Computer Science or a related field. 
• 3–6 years of hands-on experience in configuring and managing Azure-based data analytics solutions. 
• Prior experience working in Azure cloud environments is strongly preferred. 

Physical Demands and Work Conditions

• General office work environment; no special physical demands required. 
• Schedule flexibility required, including regular weekend and holiday coverage to support 24×7 operations. 
• Occasional travel may be required (less than 10%).