Data Engineer – Microsoft Fabric Developer (Spark Notebooks)
Location: Remote
Department: Digital Solutions / Data Engineering
Reports To: Data Engineering Lead / Director
Job Type: Full-time
About AIM
Architecture in Motion Inc. (AIM) is a Canadian technology consulting firm that supports organizations across public and private sectors with cloud platforms, data engineering, analytics, and enterprise technology solutions.
We partner with clients to deliver scalable, secure, and data-driven systems, with a strong focus on modern cloud platforms, including Microsoft Azure and Microsoft Fabric.
Role Overview
AIM is seeking a Data Engineer – Microsoft Fabric Developer with strong expertise in Spark Notebooks and modern data platforms.
This role is a hands-on individual contributor position focused on building, optimizing, and managing data pipelines, transformations, and analytics solutions within Microsoft Fabric. The ideal candidate will work closely with data analysts, Power BI developers, and solution architects to deliver high-quality, scalable data solutions.
Key Responsibilities
Data Engineering & Development
- Design, develop, and maintain data pipelines using Microsoft Fabric
- Develop and optimize Spark Notebooks (PySpark / Spark SQL) for data transformation and processing
- Build scalable ETL/ELT workflows for structured and unstructured data
Data Platform & Architecture
- Work within Microsoft Fabric ecosystem (Data Factory, Lakehouse, Warehousing)
- Design and manage data models and storage layers (Bronze, Silver, Gold architecture)
- Ensure performance optimization and efficient data processing
Integration & Data Management
- Integrate data from multiple sources, including:
- APIs
- Databases
- Cloud storage systems
- Ensure data quality, validation, and governance best practices
Analytics & Collaboration
- Collaborate with Power BI developers and analysts to support reporting requirements
- Enable clean, structured datasets for reporting and visualization
- Support business teams with data insights and troubleshooting
Optimization & Support
- Monitor and optimize data pipelines and Spark workloads
- Troubleshoot data issues and ensure reliability and performance
- Support deployment and CI/CD processes for data solutions
Documentation & Process Improvement
- Maintain technical documentation for data pipelines and workflows
- Support standardization of data engineering processes and best practices
Required Qualifications
Education
- Bachelor’s degree in Computer Science, Data Engineering, IT, or a related field
Experience
- 4–6+ years of experience in data engineering or data platform development
- Hands-on experience with Microsoft Fabric or Azure Data Platform
- Strong experience working with Spark Notebooks (PySpark / Spark SQL)
Technical Skills
- Expertise in:
- Microsoft Fabric
- Spark / PySpark / Spark SQL
- Data pipelines (ETL/ELT)
- Experience with:
- Azure Data Factory / Synapse / Fabric pipelines
- Data Lake / Lakehouse architecture
- SQL and relational databases
- Understanding of:
- Data modeling concepts
- Data governance and quality frameworks
Core Competencies
- Strong problem-solving and analytical skills
- Ability to work in a cross-functional, collaborative environment
- Strong communication and documentation skills
- Ability to manage tasks independently in a remote setup
Nice to Have
- Experience with Power BI and data visualization
- Familiarity with CI/CD pipelines and DevOps practices
- Experience with large-scale enterprise data platforms
What We Offer
- 100% remote work environment
- Opportunity to work on modern cloud and data platforms (Microsoft Fabric)
- Exposure to enterprise-scale data and analytics projects
- Collaborative and growth-focused culture
Application Instructions
Please share examples of data engineering projects, GitHub repositories (if available), or relevant certifications in Microsoft/Azure data technologies.
Skills Required
- Bachelor's degree in Computer Science, Data Engineering, IT, or related field
- 4-6+ years of experience in data engineering or data platform development
- Hands-on experience with Microsoft Fabric or Azure Data Platform
- Strong experience working with Spark Notebooks (PySpark / Spark SQL)
- Expertise in data pipelines (ETL/ELT)
What We Do
AIM is a specialized company providing a wide range of API management solutions and services to ensure optimal performance and security for businesses. Key offerings include API Health Check, which addresses vulnerabilities and potential improvements; Managed Support Services for active API management software maintenance; and Dedicated API Expert Support for personalized assistance. AIM also offers Certified Trainers for API strategy, design, and management, as well as Custom Approach to Training tailored to specific business needs. To maximize ROI, AIM provides Upgrade Service, assisting teams in transitioning to the latest product versions with training and knowledge transfer. With a proactive approach, AIM offers 24/7 infrastructure monitoring and operational support during business hours. Other services include Native Monitoring with a log analyzer for security threats, Threat and Risk Assessments (TRA) for enterprise-wide and system-specific evaluations, Technical Vulnerability Assessment and Penetration Testing for network infrastructure, computing layer, and application layer assessments, and Information Security Health Check for evaluating critical security elements and providing improvement recommendations. By offering a comprehensive suite of services tailored to the unique needs of businesses relying on API management solutions, AIM helps organizations stay competitive and secure in the fast-paced digital economy.
