Aprio Colombia - Data Engineer, IT Data Services

Position Responsibilities

• Data Architecture and Design: 
• Design and implement data solutions that are scalable, reliable, and secure. 
• Develop data models, data flows, and data integration architectures. 
• Ensure data solutions meet business requirements and align with best practices. 
• Data Ingestion and Integration: 
• Develop and manage data pipelines to ingest data from various sources into Azure, including sourcing from third party APIs. 
• Utilize Azure Data Factory, and other Azure services to integrate, transform, and load data. 
• Ensure efficient and effective data integration processes. 
• Integrating master data management solutions into data pipelines, ensuring seamless data synchronization across various systems 
• Data Storage and Management:  
• Design and implement data storage solutions using Microsoft Fabric and Azure SQL Database 
• Master data management: defining match, merge, and survivorship rules, and ensuring the platform is integrated with other systems 
• Data Transformation and Processing: 
• Use Azure Data Factory to process and transform data. 
• Implement data transformation logic to prepare data for analysis and reporting. 
• Optimize data processing workflows for efficiency and scalability. 
• Data Quality:  
• Implementing data quality mechanisms, monitoring data quality, and ensuring compliance with data governance policies.   
• Implement data quality checks and validation processes to ensure data accuracy and consistency. 
• Monitoring and Optimization: 
• Monitor data solutions for performance, reliability, and scalability. 
• Troubleshoot and resolve data-related issues. 
• Optimize data solutions to improve performance and reduce costs. 
• Collaboration and Communication: 
• Work closely with analysts and business stakeholders to understand data requirements. 
• Collaborate with cross-functional teams to integrate data solutions with other systems and applications. 
• Documentation and Training: 
• Document data architectures, processes, and procedures. 
• Provide training and support to other team members and end-users. 
•  Design and Development: 
• Design and develop OLAP cubes using SSAS to support complex analytical queries and reporting requirements. 
• Create tabular models for faster query performance and simpler data analysis. 
• Define data sources and data source views to connect and integrate data from various databases and systems. 
• Design dimensions, hierarchies, and measures to meet business requirements. 
• Data Integration: Develop ETL processes to extract data from various sources, transform it into a suitable format, and load it into the data warehouse or SSAS databases. 
•   Performance Optimization: 
• Query Performance: Optimize the performance of OLAP cubes and tabular models to ensure fast query responses. 
• Indexing and Aggregations: Implement indexing, aggregations, and partitioning strategies to enhance performance. 
• Monitoring:
• Monitor the performance of SSAS solutions and make necessary adjustments to maintain optimal performance. 
•   Data Analysis and Reporting: 
• Write complex MDX (Multidimensional Expressions) and DAX (Data Analysis Expressions) queries to retrieve and manipulate data for analysis. 
• Integrate SSAS cubes and models with reporting tools such as Power BI, Excel, or SQL Server Reporting Services (SSRS) to deliver insightful reports and dashboards. 
•  Security and Data Governance: 
• Implement security measures to control access to data and ensure that users have the appropriate permissions. 
• Ensure data quality and consistency by implementing data validation and cleansing processes. 
• Configuring and managing Microsoft Purview, including configuration and maintenance of the Purview platform and ensuring the platform is integrated with other systems to derive insights and manage the data estate. 
• Other duties as assigned 

Qualifications

• Demonstrated experience in analyzing complex business and technical needs to develop data architecture & solutions 
• Excellent communication skills to articulate complex data concepts to diverse stakeholders 
• Strategic thinker with strong problem-solving and analytical skills. 
• Expertise & experience in: 
• Data solutions development using SQL, NoSQL, and lakehouse concepts; Microsoft Fabric in particular is desirable 
• Data integration, processing and transformation (ETL, ELT, data pipeline orchestration, real-time data processing, ingesting streaming data); ingesting streaming data, and integrating data from RESTful APIs 
• Data modeling, optimization & tuning 
• Data security and data privacy compliance such as role-based security, access control & identity management, auditing and encryption practices; also API security best practices, including authentication, authorization, and encryption. 
• Experience with API management tools such as Azure API Management, Postman, or Swagger 
• Master data management using Profisee software 
• Managment of data estate using Purview software 
• Data virtualization 
• Related Azure services such as Microsoft Fabric, Azure Data Lake Storage, Azure Synapse Analytics, Azure Databricks, ADF, etc 
• Containerization tools, orchestration, deployment 
• Data quality management and monitoring  
• Full data lifecycle management 
• Nice-to-have knowledge & experience: 
• AirByte connectors & DBT  
• Familiarity with accounting principles and the financial services industry 
• Basic AI/ML concepts, algorithms, and fundamentals  
• Knowledge and experience with training, validation, and testing datasets 
• Certifications in Azure, Power BI, or related areas are desirable 
• Project management experience  
• Minimum 5 years of experience in database development, data architecture, data solutioning 
• Bachelor's or master's degree in computer science, data science, information technology, or related fields (or equivalent experience)