The Role
The Senior Data Engineer will architect and develop robust data pipelines, optimizing for performance and data quality, using Snowflake and cloud platforms like Azure and AWS.
Summary Generated by Built In
Allata is a global consulting and technology services firm with offices in the US, India, and Argentina. We help organizations accelerate growth, drive innovation, and solve complex challenges by combining strategy, design, and advanced technology. Our expertise covers defining business vision, optimizing processes, and creating engaging digital experiences. We architect and modernize secure, scalable solutions using cloud platforms and top engineering practices.
Allata also empowers clients to unlock data value through analytics and visualization and leverages artificial intelligence to automate processes and enhance decision-making. Our agile, cross-functional teams work closely with clients, either integrating with their teams or providing independent guidance—to deliver measurable results and build lasting partnerships.
If you are a smart & passionate team player - then this Senior/Data Engineer [Snowflake] opportunity is for you!
We at IMRIEL (An Allata Company) are looking for a Senior/Data Engineer to implement methods to improve data reliability and quality. You will combine raw information from different sources to create consistent and machine-readable formats. You will also develop and test architectures that enable data extraction and transformation for predictive or prescriptive modeling. Resourcefulness is a necessary skill in this role. If you truly love gaining new technical knowledge and can add more awesomeness to the team, you are eligible!
What you'll be doing:
• Architecting, developing, and maintaining scalable, efficient, and fault-tolerant data pipelines to process, clean, and integrate data from diverse sources using Python and PySpark etc.
• Designing and implementing modern Data Warehouse and Data Lake solutions on cloud platforms like Azure or AWS to support complex analytical and operational workloads.
• Building and automating ETL/ELT workflows using advanced tools such as Snowflake, Azure Data Factory, or similar platforms, ensuring optimal performance and scalability.
• Leveraging DBT (Data Build Tool) to define, document, and execute data transformation and modeling workflows.
• Writing optimized SQL queries for data retrieval, aggregation, and transformation to support downstream analytics applications.
What you need:
Basic Skills:
• Advanced skills in Python and PySpark for high-performance distributed data processing.
• Proficient in creating data pipelines with orchestration frameworks like Apache Airflow or Azure Data Factory.
• Strong experience with Snowflake, SQL Data Warehouse, and Data Lake architectures.
• Ability to write, optimize, and troubleshoot complex SQL queries and stored procedures.
• Deep understanding of building and managing ETL/ELT workflows using tools such as DBT, Snowflake, or Azure Data Factory.
• Hands-on experience with cloud platforms such as Azure or AWS, including services like S3, Lambda, Glue, or Azure Blob Storage.
• Proficient in designing and implementing data models, including star and snowflake schemas.
• Familiarity with distributed processing systems and concepts such as Spark, Hadoop, or Databricks.
Responsibilities:
• Develop robust, efficient, and reusable pipelines to process and transform large-scale datasets using Python and PySpark.
• Design pipeline workflows for batch and real-time data processing using orchestration tools like Apache Airflow or Azure Data Factory.
• Implement automated data ingestion frameworks to extract data from structured, semi-structured, and unstructured sources such as APIs, FTP, and data streams.
• Architect and optimize scalable Data Warehouse and Data Lake solutions using Snowflake, Azure Data Lake, or AWS S3.
• Implement partitioning, bucketing, and indexing strategies for efficient querying and data storage management.
• Develop ETL/ELT pipelines using tools like Azure Data Factory or Snowflake to handle complex data transformations and business logic.
• Integrate DBT (Data Build Tool) to automate data transformations, ensuring modularity and testability.
• Ensure pipelines are optimized for cost-efficiency and high performance, leveraging features such as pushdown optimization and parallel processing.
• Write, optimize, and troubleshoot complex SQL queries for data manipulation, aggregation, and reporting.
• Design and implement dimensional and normalized data models (e.g., star and snowflake schemas) for analytics use cases.
• Deploy and manage data workflows on cloud platforms (Azure or AWS) using services like AWS Glue, Azure Synapse Analytics, or Databricks.
• Monitor resource usage and costs, implementing cost-saving measures such as data lifecycle management and auto-scaling.
• Implement data quality frameworks to validate, clean, and enrich datasets.
• Build self-healing mechanisms to minimize downtime and ensure the reliability of critical pipelines.
• Optimize distributed data processing workflows for Spark by tuning configurations such as executor memory and partitioning.
• Conduct profiling and debugging of data workflows to identify and resolve bottlenecks.
• Collaborate with data analysts, scientists, and stakeholders to define requirements and deliver usable datasets.
• Maintain clear and comprehensive documentation for pipelines, workflows, and architectural decisions.
• Conduct code reviews to ensure best practices in coding and performance optimization.
Good To Have:
• Experience with real-time data processing frameworks such as Kafka or Kinesis.
• Certifications in Snowflake.
• Cloud Certifications. (Azure, AWS, GCP) Knowledge
• Knowledge of data visualization platforms such as Power BI, Tableau, or Looker for integration purposes.
Personal Attributes:
• Ability to identify, troubleshoot, and resolve complex data issues effectively.
• Strong teamwork, communication skills and intellectual curiosity to work collaboratively and effectively with cross-functional teams.
• Commitment to delivering high-quality, accurate, and reliable data products solutions.
• Willingness to embrace new tools, technologies, and methodologies.
• Innovative thinker with a proactive approach to overcoming challenges.
At Allata, we value differences.
Allata is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.
Allata makes employment decisions without regard to race, color, creed, religion, age, ancestry, national origin, veteran status, sex, sexual orientation, gender, gender identity, gender expression, marital status, disability or any other legally protected category.
This policy applies to all terms and conditions of employment, including but not limited to, recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation, and training.
Top Skills
Apache Airflow
Aws S3
Azure Data Factory
Azure Data Lake
Dbt
Pyspark
Python
Snowflake
SQL
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Allata (pronounced a-ley-ta) is a strategy, architecture and enterprise-level application development company focused on helping clients enhance or scale business opportunities, create efficiencies and automate processes through custom technologies.
We are building a different kind of firm – focused on doing exciting, transformational work for great clients and bringing caring and dedicated people to make our clients goals a reality. Our vision is to build an energized group of talented professionals that can stand strong on their own but work better as a networked team.
We enable business agility at the intersection of people, process, and technology. We provide solutions and expert services to assist businesses to become more nimble, transformative, and disruptive in their respective industries. We define vision, strategy, and value creation models for shaping strategic product designs, managing, and transforming enterprise delivery.
Just as strongly as we care about our clients, we feel that it is important to give back to the community and non-profits that we are passionate about. Every month, Allata donates 2% of our net income to a charitable cause our team believes in.
We live by our mantra:
Family comes first, clients are king, we take great care of our people.
