Software Engineer, Data Infrastructure

Posted Yesterday
Be an Early Applicant
San Francisco, CA, USA
In-Office
160K-220K Annually
Mid level
Big Data • Information Technology • Software • Analytics
The Role
Own and build the end-to-end data layer: design high-throughput real-time ingestion, architect petabyte-scale open table formats, implement and optimize Spark-based batch/stream pipelines, enable Kafka/Flink integrations, ensure performance/reliability/cost efficiency, define data contracts and schemas, and establish infrastructure best practices on AWS GovCloud, Kubernetes, and Airflow.
Summary Generated by Built In

Backed by leading Silicon Valley investors, Peregrine helps public safety organizations, state and local and governments, federal agencies, and private-sector institutions address society’s challenges with unprecedented speed and accuracy. Our AI-enabled platform turns siloed and disconnected data into operational intelligence — instantly surfacing mission-critical information to empower better, faster decisions that improve outcomes at every touchpoint. Today Peregrine supports hundreds of customers across 30+ states and two countries, serving more than 125 million people — and we’re amplifying our impact as we expand into the enterprise and internationally. 

Team

As an engineering team, we believe strongly that empathy improves our solutions. Seeing how people use the product is a priority and the way we get to the right answer. Engineers will have the opportunity to work closely with our team onsite to understand the variety of use cases that Peregrine serves.

We value both ownership and collaboration—you will take full responsibility for major features and work closely with other engineers to drive them to completion. We believe that humility and empathy are essential for building the right solutions—you will collaborate directly with our deployment team and users as we iterate to solve their problems. Perseverance and creativity are crucial to executing our vision.

Role

We are looking for a Data Infrastructure Engineer to join our growing team, where you will have deep ownership over the data layer that underpins everything Peregrine does. You will join the team who will architect and build the systems that ingest, store, and serve massive volumes of real-time operational data — enabling our customers to make critical decisions with speed and confidence.

This is an individual contributor role for someone who thrives on hard technical problems and brings the experience and judgment to shape foundational infrastructure decisions. You will tackle a wide range of complex challenges, including:

  • Designing and operating a high-throughput, real-time data integration platform across diverse customer environments
  • Architecting a scalable open table format layer for reliable data storage at petabyte scale
  • Building and optimizing distributed data processing pipelines with Apache Spark and adjacent streaming technologies
  • Driving performance, reliability, and cost efficiency across the full data infrastructure stack
  • Collaborating with platform and product engineering teams to define data contracts, schemas, and integration patterns
  • Establishing best practices, tooling, and patterns that raise the quality bar for data infrastructure across the organization

Our stack is constantly evolving but is built on AWS GovCloud, Apache Iceberg, Apache Spark, Apache Kafka, Airflow, Kubernetes, and more.

About You
  • Deep passion for data infrastructure — you care about building systems that are correct, fast, and resilient at scale
  • Thrive on ambiguity and are energized by defining the right solution to hard, open-ended problems
  • Strong technical vision with the ability to translate complex data requirements into clean, durable infrastructure designs
  • Desire to own significant portions of the data stack end-to-end, from ingestion to serving
  • Committed to operational excellence — you build things you’re proud to operate
What We Look For
  • 2-5 years of experience operating large-scale data infrastructure systems in production environments
  • Experience with open table formats, particularly Apache Iceberg — including schema evolution, partitioning strategies, compaction, and time travel
  • Extensive hands-on experience with Apache Spark for batch and streaming data processing at scale
  • Background in real-time data integration and stream processing, leveraging technologies such as Apache Kafka, Apache Flink, or equivalents
  • Experience with data pipeline orchestration using Airflow or similar tools
  • Strong software engineering fundamentals in Python and/or Scala, with a track record of writing production-quality code
  • Experience with AWS or comparable cloud platforms, including S3-based data lake architectures
  • Experience with Kubernetes and containerized deployment of data workloads
  • Degree in Computer Science, Engineering, or a related field, or equivalent practical experience
  • Located in San Francisco and open to working in office

Salary Range: $160,000 - $220,000 Annually + Benefits + Equity (if applicable) + Bonus (if applicable)

Actual compensation is influenced by a wide array of factors including but not limited to skill set, level of experience, certifications or licenses, and specific work location. Information on the benefits offered is here.

Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

Peregrine Technologies is committed to creating an inclusive environment for all employees. We celebrate diversity and are a proud equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status.

Skills Required

  • 2-5 years operating large-scale data infrastructure systems in production environments
  • Experience with open table formats, particularly Apache Iceberg (schema evolution, partitioning, compaction, time travel)
  • Extensive hands-on experience with Apache Spark for batch and streaming data processing at scale
  • Background in real-time data integration and stream processing using Apache Kafka, Apache Flink, or equivalents
  • Experience with data pipeline orchestration using Airflow or similar tools
  • Strong software engineering fundamentals in Python and/or Scala, with production-quality code experience
  • Experience with AWS or comparable cloud platforms, including S3-based data lake architectures
  • Experience with Kubernetes and containerized deployment of data workloads
  • Degree in Computer Science, Engineering, or related field, or equivalent practical experience
  • Located in San Francisco and open to working in office
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
0 Employees

What We Do

Peregrine Technologies provides a real-time public safety and data integration platform that enables state and local government agencies, such as law enforcement, fire-rescue, and EMS, to securely connect and contextualize disconnected data. By transforming fragmented information into actionable insights, the company helps public safety organizations improve situational awareness, collaboration, and response speed when lives are on the line.

Similar Jobs

Peregrine Technologies, Inc. Logo Peregrine Technologies, Inc.

Staff Software Engineer

Big Data • Information Technology • Software • Analytics
In-Office
San Francisco, CA, USA
200K-275K Annually
In-Office
2 Locations
2359 Employees
213K-263K Annually

Decagon Logo Decagon

Senior Software Engineer

Artificial Intelligence • Software
In-Office
San Francisco, CA, USA
49 Employees
200K-400K Annually
In-Office
Santa Clara, CA, USA
471 Employees

Similar Companies Hiring

Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
178 Employees
Kepler  Thumbnail
Fintech • Software
New York, New York
6 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account