Data Engineering Lead

Role: Lead Data Engineer 

Responsibilities: 

• Design and build scalable, reliable data pipelines and ETL/ELT workflows across batch and near-real-time processing scenarios 
  

• Develop and maintain data models across relational (SQL Server) and NoSQL (MongoDB, Atlas) systems to support analytical and operational use cases 
  

• Build and manage data solutions on Microsoft Fabric — including Lakehouses, Warehouses, Dataflows, and Pipelines — to deliver a unified analytics platform 
  

• Drive AI-enabled development practices within the team — leveraging AI coding assistants, LLM-based tools, and intelligent automation to accelerate pipeline development, improve code quality, and reduce manual effort 
  

• Lead, mentor, and grow a team of data engineers; conduct code reviews, provide technical guidance, and support career development 
  

• Collaborate with stakeholders, product owners, data scientists, and analysts to understand data needs and translate them into engineering solutions 
  

• Build and maintain Power BI data models, semantic layers, and datasets to support self-service analytics and business reporting 
  

• Continuously identify opportunities to optimize pipeline performance, reduce costs, and improve data reliability and quality 
  

• Adhere to and enforce data engineering standards, data security, and governance practices across the platform 
  

Required Skills and Experience: 

• 8 to 10 years of experience in data engineering with a strong track record of delivering production-grade data solutions 
  

• Strong proficiency in Python (PySpark, Pandas) and SQL for data transformation, pipeline development, and performance tuning 
  

• Proficient with Microsoft Fabric including Lakehouses, Warehouses, Dataflows Gen2, and Data Pipelines 
  

• Strong experience with SQL Server including schema design, stored procedures, indexing, and query optimization 
  

• Experience with MongoDB and MongoDB Atlas for NoSQL data modeling, indexing, aggregation pipelines, and Atlas Search 
  

• Solid experience designing and operating ETL/ELT pipelines in production, including error handling, monitoring, and SLA management 
  

• Experience with Power BI including dataset design, DAX, semantic modeling, and enabling self-service reporting 
  

• Strong exposure to AI-enabled development — using AI coding assistants, prompt-driven development, or LLM-integrated tooling to build and accelerate data engineering workflows 
  

• Experience leading or managing a small team of engineers — task allocation, mentoring, and performance support 
  

• Good understanding of data modeling concepts — dimensional modeling, star/snowflake schemas, data vault 
  

• Ability to communicate technical ideas clearly to both technical and non-technical audiences 
  

Nice to Have Qualities & Skills 

• Hands-on experience with Databricks including Delta Lake, notebooks, jobs, clusters, and Unity Catalog 
  

• Exposure to cloud data services on Azure (preferred), GCP, or AWS 
  

• Experience with streaming and event-driven architectures using Apache Kafka, Azure Event Hubs, Azure Service Bus, or similar queue/messaging technologies 
  

• Exposure to .NET for building data-adjacent services or APIs 
  

• Familiarity with data governance, data cataloging, and data lineage tooling 
  

• Exposure to MLOps or supporting ML pipeline infrastructure 
  

• Exposure to Mortgage or Real Estate domain