[Job-32192] Senior Data Engineer - Databricks

Posted 3 Days Ago
Be an Early Applicant
Hiring Remotely in Brazil
Remote
Senior level
Information Technology • Consulting
The Role
Designs, develops, and maintains large-scale Databricks data pipelines using Python, SQL, PySpark, and Delta Lake. Integrates APIs, databases, and files; applies ETL/ELT, DataOps, CI/CD, testing, governance, and data quality practices. Collaborates with business, analytics, data science, and platform teams while mentoring peers and troubleshooting performance, scalability, and reliability issues.
Summary Generated by Built In

At CI&T, we help large enterprises transform the potential of AI into real business impact with AI Deployment, AI-native execution, and tech-integrated business solutions.

With 30 years of experience in technological transformation, we accelerate innovation with expertise in Agentic SDLC, Application modernization, Data & AI, Martech and Business strategy.

We are 8,000 CI&Ters across more than 25 countries, collaborating to build solutions with real impact. AI is already part of how we work, evolve, and innovate every day.


We are looking for a Senior Data Developer with strong knowledge in developing and maintaining data pipelines in Databricks, Medallion Architecture, integrating and transforming data from various sources, such as APIs, relational databases, files, and many others!

As a senior member of the team, you will also provide technical guidance, mentor peers, and collaborate with cross-functional teams including data scientists, analysts, and platform engineers.

As a Senior Data Developer, you will lead the design and development of robust data pipelines, integrating and transforming data from diverse data sources such as APIs, relational databases, and files. Collaborating closely with business and analytics teams, you will ensure high-quality deliverables that meet the strategic needs of our organization.

Your expertise will be pivotal in maintaining the quality, reliability, security and governance of the ingested data, therefore driving our mission of Collaboration, Innovation, & Transformation.


Key Responsibilities
  • Develop and maintain data pipelines.

  • Integrate data from various sources (APIs, relational databases, files, etc.).

  • Collaborate with business and analytics teams to understand data requirements.

  • Ensure quality, reliability, security and governance of the ingested data.

  • Follow modern DataOps practices such as Code Versioning, Data Tests and CI/CD.

  • Document processes and best practices in data engineering.

Required Skills and QualificationsMust-have Skills
  • Proven experience in building and managing large-scale data pipelines in Databricks (PySpark, Delta Lake, SQL).

  • Strong programming skills in Python and SQL for data processing and transformation.

  • Deep understanding of ETL/ELT frameworks, data warehousing, and distributed data processing.

  • Hands-on experience with modern DataOps practices: version control (Git), CI/CD pipelines, automated testing, infrastructure-as-code.

  • Familiarity with cloud platforms (AWS, Azure, or GCP) and related data services.

  • Strong problem-solving skills with the ability to troubleshoot performance, scalability, and reliability issues.

  • Proficiency in Git.

  • Advanced English is essential.

Nice-to-have Skills
  • Experience with data contracts, schema evolution, and ensuring compatibility across services.

  • Expertise in data quality frameworks (e.g., Great Expectations, Soda, dbt tests, or custom-built solutions).

  • Integration with Power BI.

#MidSenior
#Li-Mh04

Our benefits:
 
-Health and dental insurance
-Meal and food allowance
-Childcare assistance
-Extended paternity leave
-Partnership with gyms and health and wellness professionals via Wellhub (Gympass) TotalPass;
-Profit Sharing and Results Participation (PLR);
-Life insurance
-Continuous learning platform (CI&T University);
-Discount club
-Free online platform dedicated to physical, mental, and overall well-being
-Pregnancy and responsible parenting course
-Partnerships with online learning platforms
-Language learning platform
And many more!
 
More details about our benefits here: https://ciandt.com/br/pt-br/carreiras
 
At CI&T, inclusion starts at the first contact. If you are a person with a disability, it is important to present your assessment during the selection process. See which data needs to be included in the report by clicking here.This way, we can ensure the support and accommodations that you deserve. If you do not yet have the assessment, don't worry: we can support you in obtaining it.
 
We have a dedicated Health and Well-being team, inclusion specialists, and affinity groups who will be with you at every stage. Count on us to make this journey side by side.

Skills Required

  • Proven experience building and managing large-scale data pipelines in Databricks using PySpark, Delta Lake, and SQL
  • Strong programming skills in Python and SQL for data processing and transformation
  • Deep understanding of ETL/ELT frameworks, data warehousing, and distributed data processing
  • Hands-on experience with DataOps practices, including Git, CI/CD pipelines, automated testing, and infrastructure-as-code
  • Familiarity with AWS, Azure, or GCP and related data services
  • Strong problem-solving skills for performance, scalability, and reliability issues
  • Proficiency in Git
  • Advanced English proficiency
  • Experience with data contracts, schema evolution, and service compatibility
  • Expertise with data quality frameworks such as Great Expectations, Soda, dbt tests, or custom solutions
  • Experience integrating with Power BI
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Campinas
6,474 Employees
Year Founded: 1995

What We Do

CI&T is your end-to-end digital transformation partner. As a digital native, we bring a 27-year track record of accelerating business impact through complete and scalable digital solutions. With a global presence of 6,000+ professionals in strategy, research, data science, design and engineering, we unlock top-line growth, improve customer experience and drive operational efficiency.

Similar Jobs

Circle (circle.so) Logo Circle (circle.so)

Lead Product Designer

Artificial Intelligence • Consumer Web • Digital Media • Information Technology • Social Impact • Software
Remote
32 Locations
250 Employees
140K-170K Annually

Circle (circle.so) Logo Circle (circle.so)

Creative Director

Artificial Intelligence • Consumer Web • Digital Media • Information Technology • Social Impact • Software
Remote
32 Locations
250 Employees

Circle (circle.so) Logo Circle (circle.so)

Growth Lead, Discover

Artificial Intelligence • Consumer Web • Digital Media • Information Technology • Social Impact • Software
Remote
32 Locations
250 Employees
140K-160K Annually

Circle (circle.so) Logo Circle (circle.so)

Designer

Artificial Intelligence • Consumer Web • Digital Media • Information Technology • Social Impact • Software
Remote
32 Locations
250 Employees
100K-120K Annually

Similar Companies Hiring

Axle Health Thumbnail
Artificial Intelligence • Healthtech • Information Technology • Logistics
Santa Monica, CA
25 Employees
NODA AI Thumbnail
Artificial Intelligence • Information Technology • Software • Cybersecurity
Sydney, AU
54 Employees
Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
178 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account