Senior Data Engineer (Databricks) — Data & Analytics Platform

Posted 2 Days Ago
Latham, NY
In-Office
Senior level
Information Technology • Software
The Role
The Senior Data Engineer will design and optimize scalable ETL pipelines for data processing using PySpark and Databricks, focusing on performance and cost efficiency.
Summary Generated by Built In
About Truelogic

At Truelogic we are a leading provider of nearshore staff augmentation services headquartered in New York. For over two decades, we’ve been delivering top-tier technology solutions to companies of all sizes, from innovative startups to industry leaders, helping them achieve their digital transformation goals.

Our team of 600+ highly skilled tech professionals, based in Latin America, drives digital disruption by partnering with U.S. companies on their most impactful projects. Whether collaborating with Fortune 500 giants or scaling startups, we deliver results that make a difference.

By applying for this position, you’re taking the first step in joining a dynamic team that values your expertise and aspirations. We aim to align your skills with opportunities that foster exceptional career growth and success while contributing to transformative projects that shape the future.

Our Client

A technology-driven organization operating in the marketing and analytics space, focused on building advanced data platforms that power media mix modeling, scenario planning, and large-scale analytical decision-making.


Job Summary

The Data Engineer will play a key role in designing, building, and optimizing scalable data pipelines that support complex analytics and modeling use cases. This role is ideal for a hands-on engineer with strong experience in PySpark and Databricks who enjoys working on performance-critical data workflows and transforming large datasets efficiently.

The ideal candidate brings deep expertise in ETL development, data transformations, and pipeline optimization, along with the ability to collaborate effectively across technical and non-technical teams. You will contribute directly to the reliability, scalability, and cost-efficiency of a modern data platform.

Responsibilities
  • Design, build, and maintain scalable ETL pipelines for large-scale data processing.

  • Implement data transformations and workflows using PySpark at an intermediate to advanced level.

  • Work extensively with Databricks to develop, manage, and optimize data pipelines.

  • Optimize pipelines for performance, scalability, and cost efficiency across environments.

  • Troubleshoot, debug, and resolve data processing and pipeline issues.

  • Collaborate with cross-functional teams to gather requirements and deliver high-quality data solutions.

  • Apply best practices for data quality, governance, and performance tuning.

  • Document data workflows, design decisions, and operational processes to ensure maintainability.

Qualifications and Job Requirements
  • 5–7 years of professional experience in data engineering.

  • Strong hands-on proficiency with PySpark (intermediate to advanced level).

  • Solid experience working with Databricks, including Autoloader, Python-based workflows, and platform best practices.

  • Proven experience optimizing data pipelines for performance and cost efficiency.

  • Strong understanding of ETL processes and large-scale data transformations.

  • Excellent problem-solving skills with the ability to diagnose and resolve complex data issues.

  • Strong communication skills and the ability to collaborate effectively with both technical and non-technical stakeholders.

What We Offer
  • 100% Remote Work: Enjoy the freedom to work from the location that helps you thrive. All it takes is a laptop and a reliable internet connection.

  • Highly Competitive USD Pay: Earn an excellent, market-leading compensation in USD, that goes beyond typical market offerings.

  • Paid Time Off: We value your well-being. Our paid time off policies ensure you have the chance to unwind and recharge when needed.

  • Work with Autonomy: Enjoy the freedom to manage your time as long as the work gets done. Focus on results, not the clock.

  • Work with Top American Companies: Grow your expertise working on innovative, high-impact projects with Industry-Leading U.S. Companies.

Why You’ll Like Working Here
  • A Culture That Values You: We prioritize well-being and work-life balance, offering engagement activities and fostering dynamic teams to ensure you thrive both personally and professionally.

  • Diverse, Global Network: Connect with over 600 professionals in 25+ countries, expand your network, and collaborate with a multicultural team from Latin America.

  • Team Up with Skilled Professionals: Join forces with senior talent. All of our team members are seasoned experts, ensuring you're working with the best in your field.

Apply now!

Top Skills

Databricks
Pyspark
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
266 Employees
Year Founded: 2003

What We Do

Truelogic Software is a Nearshore tech firm specializing in Staff Augmentation Services and Innovation Projects. We build innovative digital products by extending the engineering teams of US companies with our elite group of 500 Latin American highly experienced tech talent. Our SERVICES Staff Augmentation Services Dedicated Agile Teams Innovation Projects Our EXPERTISE Mobile & Web Development Data Engineering DevOps QA Automation & Testing UX/UI Designing Project Management Email us [email protected]

Similar Jobs

SmartAsset Logo SmartAsset

Senior Data Engineer

Consumer Web • Digital Media • eCommerce • Fintech • Financial Services
Easy Apply
In-Office or Remote
New York, NY, USA
300 Employees
136K-200K Annually

Zocdoc Logo Zocdoc

Associate, Influencer Marketing

Healthtech • Information Technology • Software • Telehealth
Easy Apply
Hybrid
New York, NY, USA
900 Employees
80K-105K Annually

ZS Logo ZS

Consultant

Artificial Intelligence • Healthtech • Professional Services • Analytics • Consulting
Hybrid
3 Locations
13000 Employees
120K-137K Annually

ZS Logo ZS

Evidence Generation Manager

Artificial Intelligence • Healthtech • Professional Services • Analytics • Consulting
Hybrid
2 Locations
13000 Employees
190K-217K Annually

Similar Companies Hiring

Scotch Thumbnail
Software • Retail • Payments • Fintech • eCommerce • Artificial Intelligence • Analytics
US
25 Employees
Milestone Systems Thumbnail
Software • Security • Other • Big Data Analytics • Artificial Intelligence • Analytics
Lake Oswego, OR
1500 Employees
Fairly Even Thumbnail
Software • Sales • Robotics • Other • Hospitality • Hardware
New York, NY

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account