AWS Data Engineer

About AllCloud

AllCloud is a leader in amplifying organizations’ cloud potential through AI. With a track record of hundreds of successful migrations and implementations across AWS and Salesforce, AllCloud has developed strategies and solutions that enable businesses of all sizes to remain at the forefront of innovation.

AllCloud is a leader in AI-led professional and managed services. As an AWS Premier and audited managed services Partner, and Salesforce Consulting partner, AllCloud provides comprehensive AI-led cloud journey support, from initial migration to ongoing management through our Engage Managed Services. Our expertise ensures that clients remain aligned with ecosystem best practices while focusing on their core business growth.

AllCloud serves clients across the globe with offices in EMEA and North America.

Job Summary

AllCloud is looking for a savvy Data Engineer to join our growing team of data experts. The hire will be responsible for migrating to the cloud, and optimizing our customers’ databases and data flows and enrich our operational and functional data flows with AI/ML algorithms.

The ideal candidate is not afraid of data in any form or scale, and is experienced with cloud services to ingest, stream, store and manipulate data. The Data Engineer will support new systems designs and migration of existing ones, working closely with solutions architects, project managers and data scientists. The candidate must be self-directed, fast-learners and comfortable supporting the data needs of multiple teams, systems and products. The right candidate will be excited by the prospect of optimizing or re-designing our customers’ data architecture to support their next generation of products and data initiatives and machine learning systems.

Summary of Key Responsibilities

• Building scalable Cloud data solutions using MPP Data Warehouses (Snowflake, Redshift, or Azure Data Warehouse/Synapse), data storage (S3, Azure Blob Storage, Delta Lakes, or AWS Lake Formation) and analytics platforms (i.e. Spark, Databricks, etc.)
• Creation of data pipelines and transformations
• Knowledge of ETL tools such as– Matillion, FiveTran, Informatica, Talend, SSIS, etc.
• Transformations – dbt
• Experience with elastic/open search
• Load historical data to a data warehouse
• Scripting in Python, Java Script or Shell
• Workflow Orchestrations using Apache Airflow, AWS Step Functions, etc.
• Familiarity with automated promotions, SCM tools, and CICD best practices
• Modeling and curation of data for visualization and predictive modeling users
• Design and implementation of AWS and/or Azure services such as Lambda, SNS, etc.
• Creating data integrations with scripting languages such as Python
• Writing complex SQL queries, stored procedures, etc.

Must-Have Skills & Experience

• Bachelor’s degree, or equivalent experience, in Computer Science, Engineering, Mathematics or a related field. Commensurate work experience will be considered in lieu of degree
• Experience building scalable Cloud data solutions using MPP Data Warehouses (Snowflake, Redshift, or Azure Data Warehouse/Synapse), data storage (S3, Azure Blob Storage, Delta Lakes, or AWS Lake Formation) and analytics platforms (i.e. Spark, Databricks, etc.)
• Complex SQL queries and scripting
• Experience building data pipelines via Python, Spark, or GUI Based tools
• Experience loading historical data to data warehouses
• Experience with AWS 
• Developing, and deploying scalable enterprise data solutions (Enterprise Data Warehouses, Data Marts, ETL/ELT workloads, etc.)
• Supporting business intelligence and analytic projects
• DevOps experience (Networking, IaC - Terraform, CDK)
• Experience working in an environment with automated promotions to production
• Familiar with ETL/ELT patterns and methodologies.
• Good understanding of code repositories such as GIT
• Excellent written and oral communication skills

Advantages:

• Ability to integrate and leverage Large Language Models (LLMs).
• Knowledge of RAG (Retrieval-Augmented Generation) frameworks.
• Experience with ML models for classification, scoring, and predictive analytics.
• Experience with Deep Learning Neural Networks, including Convolutional and NLP models.
• Experience with a Business Intelligence tool such as Tableau, PowerBI, Sigma, etc.
• Experience working at a consulting company
• Experience with Data Vault architecture
• Any speciality certifications

Why work for us? 

Our team inspires progress in each other and in our customers through our relentless pursuit of excellence; you will work with leaders who promote learning and personal development. We offer competitive salaries, bonus incentives, benefits, flexible hours, and mentoring. Apply now to become part of the team. 

AllCloud is an Equal Opportunity Employer and considers applicants for employment without regard to race, color, religion, sex, orientation, national origin, age, disability, genetics or any other basis forbidden under federal, provincial, or local law.