Reference Data Engineer

Posted 9 Hours Ago
Be an Early Applicant
New York, NY, USA
In-Office
135K-170K Annually
Mid level
Insurance • Financial Services
The Role
Designs and owns Python backend services, scalable data pipelines, Spark processing workflows, data models, ETL/ELT systems, and data distribution patterns. Manages Terraform-based cloud infrastructure, ensures data quality and observability, leads technical workstreams, performs production root-cause analysis, and mentors junior engineers. Collaborates with product, data, and platform leaders to deliver reliable, scalable solutions.
Summary Generated by Built In

COMPANY OVERVIEW

KKR is a leading global investment firm that offers alternative asset management as well as capital markets and insurance solutions. KKR aims to generate attractive investment returns by following a patient and disciplined investment approach, employing world-class people, and supporting growth in its portfolio companies and communities. KKR sponsors investment funds that invest in private equity, credit and real assets and has strategic partners that manage hedge funds. KKR’s insurance subsidiaries offer retirement, life and reinsurance products under the management of Global Atlantic Financial Group. References to KKR’s investments may include the activities of its sponsored funds and insurance subsidiaries.

TEAM OVERVIEW

KKR's Technology organization is a group of passionate technologists and product managers, unified by a shared mission to deliver exceptional products and solutions that drive value for our stakeholders, clients, and investors. Our passion for technology and innovation fuels our commitment to creating high-quality, impactful solutions that address complex challenges and meet the evolving needs of our sophisticated businesses.

POSITION SUMMARY

We are seeking an Assistant Vice President (AVP), Software Engineering to join our Operations Systems team. This role blends senior individual contribution with emerging technical leadership: you will design and deliver robust backend services and data pipelines while helping guide the technical direction of your team. The ideal candidate is a strong Python developer with hands-on experience in distributed data processing (Spark), infrastructure-as-code (Terraform), and deep data engineering fundamentals spanning data modeling, ETL, and data distribution. You will own significant workstreams end-to-end, raise the engineering bar through code reviews and design input, and mentor junior engineers as you continue to grow toward a lead or vice president level.

KEY RESPONSIBILITIES
  • Design, build, and own scalable backend services and data pipelines primarily in Python.
  • Develop and optimize large-scale data processing workflows using Apache Spark for batch and near-real-time use cases.
  • Lead data modeling and schema design decisions that support reliable analytics and downstream consumption.
  • Architect and maintain robust ETL/ELT workflows with strong data quality, lineage, and observability.
  • Design and implement data distribution patterns to serve data efficiently across internal systems and consumers.
  • Provision and manage cloud infrastructure using Terraform and infrastructure-as-code best practices.
  • Own delivery of significant workstreams end-to-end, from technical design through production and operations.
  • Drive engineering quality through design reviews, code reviews, and clear technical standards.
  • Mentor and support the growth of analysts and junior engineers on the team.
  • Partner with product, data, and platform leaders to shape roadmap and translate requirements into scalable solutions.
  • Lead root-cause analysis for complex production issues and drive continuous improvements in reliability and performance.
QUALIFICATIONS
  • Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.
  • 4–7 years of professional software engineering experience with a strong backend focus.
  • Expert-level proficiency in Python for building production services and data pipelines.
  • Strong hands-on experience with Apache Spark for distributed data processing at scale.
  • Proficiency with Terraform for infrastructure-as-code and cloud resource management.
  • Deep data engineering fundamentals, including data modeling, ETL/ELT design, and data distribution patterns.
  • Solid command of relational and non-relational data stores and SQL.
  • Experience with version control (Git), CI/CD pipelines, and modern software development practices.
  • Demonstrated ability to own complex workstreams and mentor other engineers.
  • Strong communication skills and the ability to collaborate across technical and business teams.
PPREFERRED QUALIFICATIONS
  • Experience with cloud platforms (AWS, Azure, or GCP) and their data services.
  • Familiarity with workflow orchestration tools (e.g., Airflow, Dagster).
  • Experience with streaming technologies (e.g., Kafka, Spark Structured Streaming).
  • Exposure to containerization and orchestration (Docker, Kubernetes).
  • Prior experience in financial services or another data-intensive, regulated industry.
  • Experience with Palantir Foundry or similar data integration and analytics platforms.
WHAT WE OFFER

KKR offers a collaborative, high-performance culture where engineers work on meaningful problems at scale. We provide competitive compensation, comprehensive benefits, and significant opportunities for professional growth, technical leadership, and impact.

This is the expected annual base salary range for this New York-based position. Actual salaries may vary based on factors, such as skill, experience, and qualification for the role. Employees may be eligible for a discretionary bonus, based on factors such as individual and team performance.
Base Salary Range
$135,000—$170,000 USD

KKR is an equal opportunity employer.  Individuals seeking employment are considered without regard to race, color, religion, national origin, age, sex, marital status, ancestry, physical or mental disability, veteran status, sexual orientation, or any other category protected by applicable law.

KKR will provide reasonable accommodations as required by applicable federal, state, and/or local laws. Individuals seeking an accommodation for the application or interview process should email [email protected]. Emails sent for unrelated issues, such as following up on an application, will not receive a response.

If you are a qualified individual with a disability or a disabled veteran, you may request a reasonable accommodation if you are unable or limited in your ability to use or access https://www.kkr.com/careers because of your disability. You can request reasonable accommodations by sending an email to [email protected]. Only emails left for this purpose will be returned.

Massachusetts Applicants: It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability. This notice applies only to applicants and employees who work or will work in Massachusetts, in accordance with applicable state law.

Skills Required

  • Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience
  • 4-7 years of professional software engineering experience with a strong backend focus
  • Expert-level proficiency in Python for production services and data pipelines
  • Strong hands-on experience with Apache Spark for distributed data processing at scale
  • Proficiency with Terraform for infrastructure-as-code and cloud resource management
  • Deep data engineering fundamentals, including data modeling, ETL/ELT design, and data distribution patterns
  • Command of relational and non-relational data stores and SQL
  • Experience with version control, CI/CD pipelines, and modern software development practices
  • Ability to own complex workstreams and mentor other engineers
  • Strong communication skills and ability to collaborate across technical and business teams
  • Experience with AWS, Azure, or GCP and their data services
  • Familiarity with Airflow or Dagster
  • Experience with streaming technologies such as Kafka or Spark Structured Streaming
  • Exposure to Docker and Kubernetes
  • Prior experience in financial services or another data-intensive, regulated industry
  • Experience with Palantir Foundry or similar data integration and analytics platforms

KKR Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about KKR and has not been reviewed or approved by KKR.

  • Fair & Transparent Compensation — Pay is considered strong and often characterized as fair, with a success‑based model intended to tie rewards to realized performance. Feedback suggests this structure improves clarity around how outcomes influence compensation.
  • Equity Value & Accessibility — Broad‑based employee ownership is emphasized, giving team members access to options, RSUs, and portfolio‑company equity programs that can yield meaningful upside. This approach is positioned as additional value without trading off wages or core benefits.
  • Healthcare Strength — Health coverage is described as comprehensive, with standout elements like an on‑site wellness center at the New York headquarters offering primary care, mental health, and preventive services. Wellness programs, nutrition support, and EAP resources further reinforce a robust health offering.

KKR Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: New York, NY
3,705 Employees
Year Founded: 1976

What We Do

KKR is a leading global investment firm that offers alternative asset management and capital markets and insurance solutions. KKR aims to generate attractive investment returns by following a patient and disciplined investment approach, employing world-class people, and supporting growth in its portfolio companies and communities. KKR sponsors investment funds that invest in private equity, credit and real assets and has strategic partners that manage hedge funds. KKR’s insurance subsidiaries offer retirement, life and reinsurance products under the management of The Global Atlantic Financial Group. References to KKR’s investments may include the activities of its sponsored funds and insurance subsidiaries. For additional information about KKR & Co. Inc. (NYSE: KKR), please visit KKR’s website at www.kkr.com and on Twitter @KKR_Co.

Similar Jobs

Clear Street Logo Clear Street

Software Engineer

Fintech • Software • Financial Services
Easy Apply
Hybrid
New York, NY, USA
581 Employees
200K-250K Annually

Wipfli Logo Wipfli

Consultant

Cloud • Fintech • Software • Business Intelligence • Consulting • Financial Services
Remote or Hybrid
United States
2900 Employees
66K-89K Annually
Hybrid
3 Locations
289097 Employees

CrowdStrike Logo CrowdStrike

Platform Professional Services Resident Senior Manager (Remote)

Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Remote or Hybrid
USA
11000 Employees
160K-220K Annually

Similar Companies Hiring

Hanover Park Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
42 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees
Vega Thumbnail
Artificial Intelligence • Automotive • Insurance • Transportation
US
43 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account