Senior Data Engineer

Posted An Hour Ago
Be an Early Applicant
Atlanta, GA, USA
In-Office
Senior level
Financial Services
The Role
Design and implement an on-premises data platform, scalable self-service pipelines, data quality frameworks, and batch and real-time ML/AI workflows. Establish engineering standards for version control, CI/CD, testing, governance, and reusable integrations. Optimize Kubernetes-based infrastructure, collaborate with stakeholders and ML teams, and mentor junior engineers while improving data reliability and platform performance.
Summary Generated by Built In
Job Description

About You

The ideal candidate is a problem-solver who enjoys working on complex data systems and is passionate about data quality. You thrive in collaborative environments but can also work independently to deliver solutions. You're comfortable working directly with technical and non-technical stakeholders and can communicate complex technical concepts clearly. Most importantly, you're excited about creating systems that empower others to work with data efficiently and confidently.


About the Role

We're seeking a talented Senior Data Engineer to join our Enterprise Architecture team in a cross-cutting role that will help define and implement our next-generation data platform. In this pivotal position, you'll lead the design and implementation of scalable, self-service data pipelines with a strong emphasis on data quality and governance. This is an opportunity to shape our data engineering practice from the ground up, working directly with key stakeholders to build mission-critical ML and AI data workflows.

Key Responsibilities

  • Design, build, and maintain our on-premises data orchestration platform using the best-in-breed open source tools
  • Create self-service capabilities that empower teams across the organization to build and deploy data pipelines without extensive engineering support
  • Implement robust data quality testing frameworks that ensure data integrity throughout the entire data lifecycle
  • Establish data engineering best practices, including version control, CI/CD for data pipelines, and automated testing
  • Collaborate with ML/AI teams to build scalable feature engineering pipelines that support both batch and real-time data processing
  • Develop reusable patterns for common data integration scenarios that can be leveraged across the organization
  • Work closely with infrastructure teams to optimize our Kubernetes-based data platform for performance and reliability
  • Mentor junior engineers and advocate for engineering excellence in data practices

Qualifications

  • 5+ years of professional experience in data engineering, with at least 2 years working on enterprise-scale data platforms
  • Bachelor's Degree in CS or equivalent
  • Deep expertise with orchestrating workflows, performance optimization, and operational management
  • Strong understanding of data transformation techniques, including experience with testing frameworks and deployment strategies
  • Experience with stream processing frameworks and technologies
  • Proficiency with SQL and Python for data transformation and pipeline development
  • Familiarity with containerized application deployment
  • Experience implementing data quality frameworks and automated testing for data pipelines
  • Ability to work cross-functionally with data scientists, ML engineers, and business stakeholders

Preferred Qualifications

  • Experience with self-hosted data orchestration platforms (rather than managed services)
  • Background in implementing data contracts or schema governance
  • Knowledge of ML/AI data pipeline requirements and feature engineering
  • Experience with real-time data processing and streaming architectures
  • Familiarity with data modeling and warehouse design principles
  • Prior experience in a technical leadership role

We emphasize building systems that are maintainable, scalable, and focused on enabling self-service data access while maintaining high standards for data quality and governance.


#LI-HR1 #LI-ONSITE

----------Intercontinental Exchange, Inc. is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to legally protected characteristics.

Skills Required

  • 5+ years of professional data engineering experience
  • At least 2 years working on enterprise-scale data platforms
  • Bachelor’s degree in Computer Science or equivalent
  • Expertise in workflow orchestration, performance optimization, and operational management
  • Strong understanding of data transformation, testing frameworks, and deployment strategies
  • Experience with stream processing frameworks and technologies
  • Proficiency with SQL and Python
  • Familiarity with containerized application deployment
  • Experience implementing data quality frameworks and automated pipeline testing
  • Ability to collaborate with data scientists, ML engineers, and business stakeholders
  • Experience with self-hosted data orchestration platforms
  • Experience implementing data contracts or schema governance
  • Knowledge of ML/AI data pipeline requirements and feature engineering
  • Experience with real-time data processing and streaming architectures
  • Familiarity with data modeling and warehouse design principles
  • Prior technical leadership experience
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Atlanta, GA
4,631 Employees
Year Founded: 2001

What We Do

ICE (NYSE: ICE) connects people to data, technology and expertise that create opportunity and inspire innovation.

Similar Jobs

SoFi Logo SoFi

Senior Data Engineer

Fintech • Mobile • Software • Financial Services
Easy Apply
Remote or Hybrid
United States
4500 Employees
125K-234K Annually

Cox Enterprises Logo Cox Enterprises

Senior Data Engineer

Artificial Intelligence • Automotive • Greentech • Information Technology • Machine Learning • Software • Cybersecurity
Hybrid
Atlanta, GA, USA
30000 Employees
102K-169K Annually

Cargill Logo Cargill

Senior Data Engineer

Food • Greentech • Logistics • Sharing Economy • Transportation • Agriculture • Industrial
In-Office
Atlanta, GA, USA
155000 Employees
107K-160K Annually

Cargill Logo Cargill

Senior Data Engineer

Food • Greentech • Logistics • Sharing Economy • Transportation • Agriculture • Industrial
In-Office
2 Locations
155000 Employees

Similar Companies Hiring

Granted Thumbnail
Artificial Intelligence • Healthtech • Insurance • Mobile • Financial Services
New York, New York
23 Employees
Hanover Park Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
42 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
85 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account