Overview of The Role
The Data Engineer is responsible for designing, building, and supporting scalable, secure, and high-performance analytics solutions on the Microsoft Fabric platform. This role combines strong technical depth with business acumen to translate evolving business needs into trusted data products.The individual will play a key role in shaping and executing our clients modern BI and analytics strategy.
How You Will Make Contributions That Matter:
· Partner closely with stakeholders to translate business requirements into Fabric-based data models, pipelines, and analytical solutions
· Contribute to the end-to-end design and evolution of a scalable enterprise analytics architecture using Microsoft Fabric
· Collaborate with Business SMEs and technical teams (Infrastructure, Security, Dynamics, external) to design and deliver sustainable, governed analytics solutions
· Assist with preparation and facilitation of design workshops with documented outcomes (requirements, design, data models)
· Ensure solutions are scalable, secure, cost-effective, and aligned with data & AI strategy
· Design and develop high-performance data engineering components capable of processing large volumes of structured and semi-structured data
· Build reusable, enterprise-grade data models (Lake-house and semantic models) aligned with architectural standards and best practices
· Contribute to administration, governance, and operational health of the Microsoft Fabric platform
· Define and execute test strategies, including data validation, performance tuning, and data profiling
· Monitor Fabric pipelines, data refreshes, and workloads, proactively troubleshoot and resolve issues
· Own incident resolution and day-to-day sustainment of Fabric analytics solutions
· Utilize Microsoft Analytics tools for seamless integration and efficient user interface development to enhance overall user experience
· Optimize SQL, Spark, and DAX queries for performance and scalability
· Mentor analytics, functional and business teams on Microsoft Fabric capabilities, patterns, and best practices
· Contributes to work effort estimation for new and enhanced project opportunities
Supervision
· Acts as an individual contributor and operates independently while ensuring alignment with enterprise BI and analytics strategy
· Can oversee project and support activities such as troubleshooting, enhancements, capacity planning, upgrades, and migrations
· Communicates clearly and collaborates effectively with cross-functional and geographically distributed teams
Qualifications
· Bachelor’s degree in information systems, Computer Science, Software Engineering, or equivalent experience
· 5+ years of hands-on experience delivering enterprise analytics and data engineering solutions on Microsoft platforms
· Strong experience engineering solutions on Microsoft Fabric and modern Azure analytics services, with understanding and working knowledge of other industry-standard analytics and data engineering tools.
· Microsoft Azure / Fabric certifications
· Strong foundation in:
o Data ingestion and integration
o Data transformation and orchestration
o Data storage, data lakes, and lake-house architecture
o Data warehousing and dimensional modelling
· Experience with CI/CD, DevOps, and source control (Azure DevOps, GitHub)
· Understanding of administration, security, and governance for Microsoft analytics platforms
· Experience with enterprise delivery methodologies (Agile/ Waterfall) and estimation practices
· Robust understanding of Optimization techniques to improve ingestion process
· Experience in building real time solutions end to end
· Experience in ingesting data from SaaS, on-prem, PaaS, SharePoint and different file formats
· Understanding of advanced AI technologies like ML, Agentic AI.
· Strong written and verbal communication skills, especially for technical design documentation
· Ability to operate effectively across complex, interdependent work-streams, teams, and geographies
· Up-to-date knowledge of industry trends and best practices in data engineering and analytics
Technical qualifications
· Strong hands-on experience as a Data Engineer on Microsoft Fabric and Azure platforms, including:
o Fabric Lake-house/Warehouse/Event House/ Materialized Lake view
o Fabric Ingestion (Data Factory, Pipelines, Dataflow Gen1, Dataflow Gen2, Copy Job, Notebooks, EventStream, Spark Job Definition, Spark structure streaming)
o Fabric Orchestration (Apache Airflow)
o Data Engineering Languages (Py-Spark/Spark SQL /KQL)
· Experience with Azure analytics services, including:
o Azure Data Factory
o Azure Synapse Analytics
o Azure Data Lake Storage
o Azure Data-bricks (where applicable)
· Development using Python and Spark notebooks
· Understanding of data modelling skills, including dimensional modelling (Kimball / star schema)
· Advanced SQL skills for analytical workloads
· Good understanding of Power BI
· Workspace, security, and capacity management
· Solid understanding of DevOps / GitHub for version control and CI/CD
· Practical understanding of Microsoft Fabric governance, security (One Lake Security, SQL security), and workload management
Skills Required
- Bachelor's degree in information systems, computer science, software engineering, or equivalent experience
- 5+ years of hands-on experience delivering enterprise analytics and data engineering solutions on Microsoft platforms
- Strong experience engineering solutions on Microsoft Fabric and modern Azure analytics services
- Microsoft Azure or Fabric certifications
- Experience with data ingestion and integration
- Experience with data transformation and orchestration
- Experience with data storage, data lakes, and lakehouse architecture
- Experience with data warehousing and dimensional modeling
- Experience with CI/CD, DevOps, and source control using Azure DevOps or GitHub
- Understanding of administration, security, and governance for Microsoft analytics platforms
- Experience with Agile or Waterfall delivery methodologies and estimation practices
- Understanding of optimization techniques for improving ingestion processes
- Experience building end-to-end real-time solutions
- Experience ingesting data from SaaS, on-premises, PaaS, SharePoint, and various file formats
- Understanding of advanced AI technologies such as machine learning and agentic AI
- Strong written and verbal communication skills for technical design documentation
- Ability to work across complex, interdependent workstreams, teams, and geographies
- Strong hands-on experience with Microsoft Fabric Lakehouse, Warehouse, Eventhouse, Materialized Lake Views, ingestion, orchestration, and Spark capabilities
- Experience with Fabric Data Factory, Pipelines, Dataflows, Copy Job, Notebooks, Eventstream, Spark Job Definition, and Structured Streaming
- Experience with Apache Airflow orchestration
- Proficiency in PySpark, Spark SQL, and KQL
- Experience with Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage, and Azure Databricks
- Development experience using Python and Spark notebooks
- Advanced SQL skills for analytical workloads
- Good understanding of Power BI
- Experience with workspace, security, and capacity management
- Practical understanding of Microsoft Fabric governance, OneLake Security, SQL security, and workload management
- Up-to-date knowledge of data engineering and analytics industry trends and best practices
What We Do
Blackbuck Insights is a global data and AI consultancy providing data engineering services, cloud modernization, AI activation, and platform operations. The company helps enterprises modernize legacy environments, migrate and scale data platforms, build AI-enabled business applications, and establish trusted data foundations for analytics and intelligent decision-making. Its services support organizations seeking reliable, performant, and scalable technology solutions.








