Big Data Lead

Posted Yesterday
Be an Early Applicant
Hiring Remotely in United States
Remote
Senior level
Information Technology • Consulting
The Role
Design, build, and optimize end-to-end ETL/ELT pipelines using Python and PySpark for large structured and unstructured datasets. Ensure data quality through validation and reconciliation, troubleshoot performance issues, implement CI/CD and code review practices, and collaborate with IT and business stakeholders to translate requirements into scalable data solutions.
Summary Generated by Built In
Responsibilities: • Development and Maintain Data Pipelines: Design, implement, and optimize end-to-end ETL/ELT pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data. • Utilize Python and Pyspark: Write efficient, scalable and maintainable code in Python and leverage Pyspark for large-scale data processing in distributed computing environments. Also be able to review existing code and identify areas of improvement. • Ensure Data Quality and Integrity: Implement data validation, cleansing, transformation and reconciliation processes to ensure data accuracy and consistency throughout the data lifecycle. • Collaborate with Stakeholders: Work closely with IT teams and business stakeholders to gather data requirements and translate them to technical solutions. • Troubleshoot and Optimize: Monitor job performance, troubleshoot complex data issues and fine-tune for performance and scalability. • Adhere to Best Practices: Participate in code reviews, establish coding standards, and implement CI/CD pipelines for automated testing and deployment. Skills: • Strong hands-on coding proficiency in Python, Pyspark and SQL (Microsoft SQL Server preferred) • Experience with big data frameworks (Hadoop, Spark). • Experience with cloud platforms ( AWS, Azure or GCP) • Experience with Code versioning tools ( Bitbucket, Github ) • Experience with CI/CD and setting up pipelines. • Solid understanding of database design principles, data modelling, schemas and data warehousing solutions. • Excellent problem-solving and analytical skills to troubleshoot complex data issues independently.

Skills Required

  • Hands-on coding proficiency in Python
  • Hands-on coding proficiency in PySpark
  • Proficiency in SQL
  • Experience with Microsoft SQL Server
  • Experience with big data frameworks (Hadoop, Spark)
  • Experience with cloud platforms (AWS, Azure or GCP)
  • Experience with code versioning tools (Bitbucket, GitHub)
  • Experience designing and implementing CI/CD pipelines
  • Ability to design, implement, and optimize ETL/ELT pipelines
  • Strong understanding of database design, data modeling, schemas and data warehousing
  • Implement data validation, cleansing, transformation and reconciliation
  • Excellent problem-solving and analytical skills

Hexaware Technologies Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Hexaware Technologies and has not been reviewed or approved by Hexaware Technologies.

  • Healthcare Strength Feedback suggests health insurance is a standout element, often described as strong and comparable or better than many employers. Medical coverage and related protections are repeatedly highlighted as reliable pillars of the package.
  • Parental & Family Support Parental leave options, including extended paid leave in some locations, are offered and seen as valuable for family needs. Feedback suggests these policies complement core medical coverage for a more complete family safety net.
  • Wellbeing & Lifestyle Benefits Wellbeing resources such as an Employee Assistance Program, wellness initiatives, on-site gyms, and engagement activities provide lifestyle support. These offerings are reinforced by flexible work options and shift-related extras where applicable.

Hexaware Technologies Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Navi Mumbai, Maharashtra
27,000 Employees
Year Founded: 1990

What We Do

At Hexaware, we're not just a global technology and business process services company; we're a community of 27,000 Hexawarians dedicated to one singular purpose: creating smiles through the power of great people and technology. With a presence in 40+ offices across 19 countries, we empower enterprises worldwide to embark on their digital transformation journey with unparalleled scale and speed. As an employer, we're more than just a workplace. We put our people first, foster diversity and inclusivity, and prioritize their growth through robust learning and development programs. Our culture is a canvas for innovation as we work toward our shared vision of becoming the world’s most loved digital transformation partner. Exciting, isn’t it? Visit www.hexaware.com to join us in embracing the magic of technology, as we passionately advocate for its potential to transform lives today and shape a brighter future. Together, we'll make the digital world a better place.

Similar Jobs

In-Office or Remote
San Francisco, CA, USA
210 Employees

Hexaware Technologies Logo Hexaware Technologies

Big Data Lead

Information Technology • Consulting
Remote
United States
27000 Employees

Hexaware Technologies Logo Hexaware Technologies

Big Data Lead

Information Technology • Consulting
Remote
United States
27000 Employees

Similar Companies Hiring

Amplify Platform Thumbnail
Fintech • Financial Services • Consulting • Cloud • Business Intelligence • Big Data Analytics
Scottsdale, AZ
62 Employees
Standard Template Labs Thumbnail
Artificial Intelligence • Information Technology • Software
New York, NY
25 Employees
Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
178 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account