Big Data Lead

Posted 6 Days Ago
Be an Early Applicant
Hiring Remotely in México
Remote
Expert/Leader
Information Technology • Consulting
The Role
Leads hands-on data engineering work, developing reusable Python packages, automated ETL pipelines, Airflow workflows, and cloud-based data integrations. Responsibilities include advanced SQL and Snowflake optimization, pipeline orchestration and troubleshooting, data quality testing with Python and dbt, configuration-driven code generation, GitHub-based CI/CD, and secure AWS data storage and processing. The role requires strong communication, client-facing capability, attention to detail, and effective individual and team collaboration.
Summary Generated by Built In
Data Engineer Must have below: 10+ Years Experience Great Communicator/Client Facing/ Attention to detail Individual Contributor and ability to work as a team. 100% Hands on in the mentioned skills Programming Skills: Python and Data Pipelines: Advanced Proficiency in Python concepts like Code Structures, Modules, Packages, Class, SubClass, Inheritance, Multi-Threading and Functional Programming. Experience in developing reusable Python packages for internal or public usage Ability to write automating ETL processes and scheduling jobs like Airflow DAG. Ability to track job pipeline runs to reprocess error records Ability to orchestration different pipelines to run in sequence or parallel. Troubleshoot data pipeline errors and fix issues Export or Import data to/from various formats like CSV, JSON, XML etc preferably from S3 or other cloud storage. Experience in using AI IDE tool SQL: Advanced SQL skills, including complex joins, CTE's and subqueries Experience in optimizing SQL queries for performance and optimization in data warehouse technologies preferably Snowflake Testing and documentation: Proficiency in Python unit, integration and system test. Proficiency in implementing DBT tests for data validation and quality checks Code Generation: Experience in generating code using configurations using python and jinja templates Version control: Experience in GitHub, including implementing CI/CD process from scratch AWS Expertise: Data Storage solutions: In depth understanding of AWS S3 for data storage, ECS, IAM including best practices for organization and security Cloud Security: Knowledge of AWS security best practices, including IAM roles, encryption standard, secure coding guidelines, DBT profiles access configurations and more. Data Integration (nice to have): Experience with AWS lambda for serverless data processing tasks

Skills Required

  • 10+ years of experience
  • Strong communication and client-facing skills
  • Attention to detail
  • Ability to work independently as an individual contributor and collaboratively as part of a team
  • Hands-on expertise with Python, including modules, packages, classes, inheritance, multithreading, and functional programming
  • Experience developing reusable Python packages
  • Experience automating ETL processes and scheduling jobs with Airflow DAGs
  • Experience tracking pipeline runs, reprocessing error records, orchestrating sequential or parallel pipelines, and troubleshooting pipeline errors
  • Experience importing and exporting CSV, JSON, XML, and other data formats, preferably using S3 or cloud storage
  • Experience using AI IDE tools
  • Advanced SQL skills, including complex joins, CTEs, and subqueries
  • Experience optimizing SQL queries and data warehouse workloads, preferably with Snowflake
  • Proficiency in Python unit, integration, and system testing
  • Proficiency implementing dbt tests for data validation and quality checks
  • Experience generating configuration-driven code using Python and Jinja templates
  • Experience with GitHub and implementing CI/CD processes from scratch
  • In-depth knowledge of AWS S3, ECS, and IAM, including organizational and security best practices
  • Knowledge of AWS security practices, IAM roles, encryption standards, secure coding guidelines, and dbt profile access configurations
  • Experience with AWS Lambda for serverless data processing

Hexaware Technologies Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Hexaware Technologies and has not been reviewed or approved by Hexaware Technologies.

  • Healthcare Strength — Feedback suggests health insurance is a standout element, often described as strong and comparable or better than many employers. Medical coverage and related protections are repeatedly highlighted as reliable pillars of the package.
  • Parental & Family Support — Parental leave options, including extended paid leave in some locations, are offered and seen as valuable for family needs. Feedback suggests these policies complement core medical coverage for a more complete family safety net.
  • Wellbeing & Lifestyle Benefits — Wellbeing resources such as an Employee Assistance Program, wellness initiatives, on-site gyms, and engagement activities provide lifestyle support. These offerings are reinforced by flexible work options and shift-related extras where applicable.

Hexaware Technologies Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Navi Mumbai, Maharashtra
27,000 Employees
Year Founded: 1990

What We Do

At Hexaware, we're not just a global technology and business process services company; we're a community of 27,000 Hexawarians dedicated to one singular purpose: creating smiles through the power of great people and technology. With a presence in 40+ offices across 19 countries, we empower enterprises worldwide to embark on their digital transformation journey with unparalleled scale and speed. As an employer, we're more than just a workplace. We put our people first, foster diversity and inclusivity, and prioritize their growth through robust learning and development programs. Our culture is a canvas for innovation as we work toward our shared vision of becoming the world’s most loved digital transformation partner. Exciting, isn’t it? Visit www.hexaware.com to join us in embracing the magic of technology, as we passionately advocate for its potential to transform lives today and shape a brighter future. Together, we'll make the digital world a better place.

Similar Jobs

VALCE Talent Solutions Logo VALCE Talent Solutions

Big Data Lead/MDM/PYSPARK

Information Technology • Logistics • Professional Services • Consulting
Remote
MEX
21 Employees

Hexaware Technologies Logo Hexaware Technologies

Big Data Lead

Information Technology • Consulting
Remote
México
27000 Employees

Sequoia Connect Logo Sequoia Connect

Big Data Lead/MDM/PYSPARK

Information Technology • Consulting
Remote
México
30 Employees

Hexaware Technologies Logo Hexaware Technologies

Big Data Lead

Information Technology • Consulting
Remote
México
27000 Employees

Similar Companies Hiring

NODA AI Thumbnail
Artificial Intelligence • Information Technology • Software • Cybersecurity
Austin, TX
54 Employees
Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
567 Employees
RapidScale Thumbnail
Cloud • Information Technology • Consulting
Raleigh, NC
600 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account