Senior Data Engineer

Posted Yesterday
Be an Early Applicant
Atlanta, GA, USA
In-Office
Senior level
Financial Services
The Role
Design and implement an on-premises enterprise data platform, including scalable self-service pipelines, data quality frameworks, workflow orchestration, CI/CD, and automated testing. Collaborate with ML and AI teams on batch and real-time feature engineering workflows, optimize Kubernetes-based infrastructure, establish governance and reusable integration patterns, and mentor junior engineers.
Summary Generated by Built In
Job Description

Job Purpose

We're seeking a talented Senior Data Engineer to join our Enterprise Architecture team in a cross-cutting role that will help define and implement our next-generation data platform. In this pivotal position, you'll lead the design and implementation of scalable, self-service data pipelines with a strong emphasis on data quality and governance. This is an opportunity to shape our data engineering practice from the ground up, working directly with key stakeholders to build mission-critical ML and AI data workflows.


Responsibilities

  • Design, build, and maintain our on-premises data orchestration platform using the best-in-breed open source tools
  • Create self-service capabilities that empower teams across the organization to build and deploy data pipelines without extensive engineering support
  • Implement robust data quality testing frameworks that ensure data integrity throughout the entire data lifecycle
  • Establish data engineering best practices, including version control, CI/CD for data pipelines, and automated testing
  • Collaborate with ML/AI teams to build scalable feature engineering pipelines that support both batch and real-time data processing
  • Develop reusable patterns for common data integration scenarios that can be leveraged across the organization
  • Work closely with infrastructure teams to optimize our Kubernetes-based data platform for performance and reliability
  • Mentor junior engineers and advocate for engineering excellence in data practices

Knowledge and Experience

  • At least a Bachelor’s degree in Computer Science, Computer Engineering, Data Science, Mathematics, Engineering, or related disciplines.
  • 5+ years of professional experience in data engineering, with at least 2 years working on enterprise-scale data platforms
  • Deep expertise with orchestrating workflows, performance optimization, and operational management
  • Strong understanding of data transformation techniques, including experience with testing frameworks and deployment strategies
  • Experience with stream processing frameworks and technologies
  • Proficiency with SQL and Python for data transformation and pipeline development
  • Familiarity with containerized application deployment
  • Experience implementing data quality frameworks and automated testing for data pipelines
  • Ability to work cross-functionally with data scientists, ML engineers, and business stakeholders

Preferred Qualifications

  • Experience with self-hosted data orchestration platforms (rather than managed services)
  • Background in implementing data contracts or schema governance
  • Knowledge of ML/AI data pipeline requirements and feature engineering
  • Experience with real-time data processing and streaming architectures
  • Familiarity with data modeling and warehouse design principles
  • Prior experience in a technical leadership role

#LI-HR1 #LI-ONSITE

----------Intercontinental Exchange, Inc. is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to legally protected characteristics.

Skills Required

  • Bachelor’s degree in Computer Science, Computer Engineering, Data Science, Mathematics, Engineering, or a related discipline
  • At least 5 years of professional experience in data engineering
  • At least 2 years of experience working on enterprise-scale data platforms
  • Expertise in workflow orchestration, performance optimization, and operational management
  • Strong understanding of data transformation, testing frameworks, and deployment strategies
  • Experience with stream processing frameworks and technologies
  • Proficiency with SQL and Python
  • Familiarity with containerized application deployment
  • Experience implementing data quality frameworks and automated testing for data pipelines
  • Ability to work cross-functionally with data scientists, ML engineers, and business stakeholders
  • Experience with self-hosted data orchestration platforms
  • Experience implementing data contracts or schema governance
  • Knowledge of ML/AI data pipeline requirements and feature engineering
  • Experience with real-time data processing and streaming architectures
  • Familiarity with data modeling and warehouse design principles
  • Prior experience in a technical leadership role
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Atlanta, GA
4,631 Employees
Year Founded: 2001

What We Do

ICE (NYSE: ICE) connects people to data, technology and expertise that create opportunity and inspire innovation.

Similar Jobs

SoFi Logo SoFi

Senior Data Engineer

Fintech • Mobile • Software • Financial Services
Easy Apply
Remote or Hybrid
United States
4500 Employees
125K-234K Annually

Cox Enterprises Logo Cox Enterprises

Senior Data Engineer

Artificial Intelligence • Automotive • Greentech • Information Technology • Machine Learning • Software • Cybersecurity
Hybrid
Atlanta, GA, USA
30000 Employees
102K-169K Annually

Cargill Logo Cargill

Senior Data Engineer

Food • Greentech • Logistics • Sharing Economy • Transportation • Agriculture • Industrial
In-Office
Atlanta, GA, USA
155000 Employees
107K-160K Annually

Cargill Logo Cargill

Senior Data Engineer

Food • Greentech • Logistics • Sharing Economy • Transportation • Agriculture • Industrial
In-Office
2 Locations
155000 Employees
110K-150K Annually

Similar Companies Hiring

Granted Thumbnail
Artificial Intelligence • Healthtech • Insurance • Mobile • Financial Services
New York, New York
23 Employees
Hanover Park Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
42 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
85 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account