Lead Engineer, Data Processing (Databricks)

Posted 6 Days Ago
Hiring Remotely in United States
Remote
Senior level
Logistics • Software • Transportation
The Role
Lead the Data Processing team, owning system design and implementation, driving modernization, and ensuring operational reliability and data quality through effective pipeline management.
Summary Generated by Built In

Empowering the people that make global commerce happen.


About CargoSprint

CargoSprint is made up of a world-class team of highly motivated individuals who are passionate about transforming the cargo industry. We have developed cutting-edge digital solutions that streamline cargo operations, enhance efficiency, and improve the overall experience for everyone involved. Our workplace fosters innovation, collaboration, and the drive to solve industry challenges.


CargoSprint is dedicated to delivering game-changing solutions that connect the cargo industry like never before, and we are looking for driven, enthusiastic people who share our vision of innovation and excellence.


If you think we are a great mutual fit, we want hear from you!

About You

You're a hands-on leader who can run a team and still ship. You care about operational reliability, clear ownership boundaries, and measurable throughput. You modernize systems safely, reduce toil, and build habits that make pipelines dependable. 
About the role

This role leads our Data Processing team. You'll own the pipelines and processing jobs that transform data for product and business use cases, with an emphasis on modernization, reliability, and data quality. Strong experience with Databricks (or similar Spark-based platforms) is required. 


What you'll do

  • Lead the Data Processing team: technical direction, delivery cadence, coaching, and hiring support (as needed). 
  • Own core processing systems end-to-end: design, implementation, quality, observability, and on-call readiness. 
  • Drive modernization of legacy processing: simplify flows, reduce failure modes, improve performance and cost. 
  • Build and operate pipelines (batch and/or streaming) with strong data quality, lineage, and backfill strategies. 
  • Establish SLAs/SLOs for key pipelines and improve monitoring, alerting, and incident response. 
  • Partner with application engineering, finance, and analytics to prioritize work and deliver dependable outputs. 
  • Raise the engineering bar via standards, PR reviews, runbooks, and pragmatic architecture decisions. 


Qualifications

  • 8+ years building production systems; 2+ years leading engineers or acting as a team tech lead. 
  • Strong experience with data processing systems (ETL/ELT, streaming, batch processing, orchestration). 
  • Hands-on experience with Databricks and Spark (PySpark/Scala) and strong SQL. 
  • Experience modernizing legacy systems safely and incrementally. 
  • Operational discipline: monitoring, alerting, incident response, and root cause analysis for pipelines. 
  • Ability to drive cross-team alignment and translate requirements into reliable delivery. 

Nice to have 

  • Lakehouse patterns (Delta Lake, Iceberg, or Hudi) and data catalog/governance tools. 
  • .Net or Java exprertise 
    Event-driven architectures (Kafka/Kinesis/PubSub) and streaming pipelines. 
  • FinTech, payments, billing, invoicing, or other high-volume transactional domains. 
  • Experience enabling self-serve analytics for business users. 
  • Logistics, cargo, or supply chain experience.  
  • Spanish language proficiency. 

 

Compensation and benefits 

We offer competitive pay and benefits designed to fuel our team's success. 

  • Health and Wellness: Medical, dental, and vision plans for you and your family 
  • Future-Ready: 401(k) with company match 
  • Work Life Balance: Generous flexible PTO program and paid holidays 
  • Grow With Us: Professional development opportunities 


#LI-Remote

Does this role sound like the next step in your career?

We’d love to hear from you! If you don’t meet all of the requirements exactly, we encourage you to use your cover letter to tell us about your unique experience—talent comes from many places, and skills are transferable.


Our Commitment to an Extraordinary Work Environment

At CargoSprint, we value diversity and inclusivity. We strive to create a welcoming and supportive community for employees from all backgrounds. Regardless of your gender, sexual orientation, physical ability, religion, ethnicity, race, or age, you will find a place where you can thrive and be your authentic self.

Our CargoSprint Recruitment Team personally reviews every application.

Top Skills

Databricks
Delta Lake
Hudi
Iceberg
Kafka
Kinesis
Pubsub
Pyspark
Scala
Spark
SQL
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Atlanta, Georgia
175 Employees
Year Founded: 2012

What We Do

CargoSprint's supply chain orchestration SaaS solutions reduce operational friction for freight forwarders, third-party logistics providers, cargo facilities, and transportation providers across air, ocean, road, and rail. By combining industry-leading tools for payments, appointments, and visibility, we ensure faster and more efficient cargo movement. In a year, we enable 4 million payments worth $2 billion on behalf of 30,000 logistics providers and orchestrate 9 million cargo appointments.

Similar Jobs

DailyPay Logo DailyPay

Enterprise Account Executive

Fintech • HR Tech • Payments • Social Impact • Financial Services
Remote
US
900 Employees

Sailor Health Logo Sailor Health

Credentialing Specialist

Healthtech • Social Impact • Telehealth
Remote
United States
20 Employees
40K-55K Annually

Deepgram Logo Deepgram

Customer Success Engineer (Americas)

Artificial Intelligence • Machine Learning • Natural Language Processing • Software • Conversational AI
Remote
USA
150 Employees
150K-195K Annually

Deepgram Logo Deepgram

Account Executive

Artificial Intelligence • Machine Learning • Natural Language Processing • Software • Conversational AI
Remote
USA
150 Employees
120K-130K Annually

Similar Companies Hiring

Scotch Thumbnail
Software • Retail • Payments • Fintech • eCommerce • Artificial Intelligence • Analytics
US
25 Employees
Milestone Systems Thumbnail
Software • Security • Other • Big Data Analytics • Artificial Intelligence • Analytics
Lake Oswego, OR
1500 Employees
Fairly Even Thumbnail
Software • Sales • Robotics • Other • Hospitality • Hardware
New York, NY

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account