Senior Software Engineer, Data Platform

Posted Yesterday
Be an Early Applicant
San Francisco, CA
In-Office
175K-215K Annually
Senior level
Artificial Intelligence • Software
The Role
The Senior Software Engineer will enhance the data platform, manage data pipelines, ensure data quality, optimize storage, and mentor teams in data engineering practices.
Summary Generated by Built In
About Us

At Hayden AI, we are on a mission to harness the power of computer vision to transform the way transit systems and other government agencies address real-world challenges.

From bus lane and bus stop enforcement to transportation optimization technologies and beyond, our innovative mobile perception system empowers our clients to accelerate transit, enhance street safety, and drive toward a sustainable future.

What the job involves

Our Engineering teams work at the cutting edge of technology, leveraging AWS and GCP cloud services and developing our own Linux-based edge devices. As a dynamic startup, we understand the critical importance of cybersecurity in protecting our innovative solutions and ensuring the safety of our digital environment and customer data.

We're looking for a passionate Senior Software Engineer to grow our data platform: enabling reliable, scalable, and high-performance data systems across the organization. You will design and build core data infrastructure, optimize data pipelines, ensure data quality and reliability, and collaborate with product and data-science teams to deliver impactful data solutions that power our product and deliver business insights.

Responsibilities
  • Evolve the data platform leveraging AWS services such as Kinesis/Kafka, Glue, EMR, Lambda, and Redshift to support streaming, batch, and analytical workloads at scale.

  • Design and implement robust data ingestion and transformation pipelines, ensuring reliability, performance, and schema consistency across diverse data sources.

  • Establish and enforce data quality, validation, and observability standards, building automated checks and alerts using CloudWatch, Datadog and custom frameworks.

  • Optimize data storage and lifecycle management through intelligent S3 partitioning, versioning, and lifecycle policies to balance performance, cost, and retention.

  • Conduct regular performance, scalability, and cost-efficiency audits of data pipelines and warehouses, proactively identifying and resolving bottlenecks or inefficiencies.

  • Ensure data reliability and resilience, implementing redundancy, checkpointing, and replay strategies for real-time and near-real-time data systems.

  • Define and maintain data governance and security best practices, including IAM-based access control, encryption policies, and compliance with internal data standards.

  • Collaborate with Data Science, ML, business and product teams to model multi-modal data effectively, enabling self-service analytics and high-impact insights across the organization.

  • Instrument and monitor data infrastructure using CloudWatch, CloudTrail, and AWS Config, ensuring transparency, traceability, and operational health of data workflows.

  • Lead root-cause analysis and incident response for data platform issues, driving long-term reliability improvements and knowledge sharing across teams.

  • Stay current with AWS and data engineering advancements, evaluating new tools and services to continuously improve scalability, developer velocity, and data accessibility.

  • Mentor other engineers and promote best practices in data architecture, pipeline design, and development across the engineering organization.

Qualifications
  • Bachelor’s degree in Computer Science, Engineering, or a related field.

  • 6+ years of experience in software or data engineering, focused on large-scale big-data systems and low latency systems.

  • Strong proficiency in Python, Scala or Java for data processing and automation.

  • Hands-on experience with AWS data services such as Kinesis/Kafka, Glue, EMR, Iceberg, Redshift, S3, and Lambda.

  • Solid understanding of data modeling, ETL/ELT design, and distributed data processing (Spark or Flink).

  • Experience with infrastructure-as-code (Terraform or CloudFormation) and CI/CD automation.

  • Familiarity with data governance, lineage, and observability best practices.

  • Working knowledge of cloud security, IAM, and cost optimization in AWS.

  • Excellent collaboration and communication skills, with experience working cross-functionally with analytics and platform teams.

  • AWS or data engineering certifications are a strong plus.

Top Skills

AWS
CloudFormation
Cloudwatch
Datadog
Emr
Flink
GCP
Glue
Java
Kafka
Kinesis
Lambda
Linux
Python
Redshift
S3
Scala
Spark
Terraform
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Oakland, CA
49 Employees
Year Founded: 2019

What We Do

Artificial intelligence is already impacting almost every aspect of our lives. But cities still rely on outdated technologies that are too expensive to scale and unable to deliver on their promise. As a result, streets are becoming unsafe, and cities are becoming unmanageable. Hayden AI was founded on the belief that by combining mobile sensors with artificial intelligence, we can help governments bridge the innovation gap while making traffic flow less dangerous and more efficient. Led by a team of experts in machine learning, data science, transportation, and government technology, we’ve developed the world’s first autonomous traffic management platform — simultaneously serving citizens and multi-agency missions to help cities become safer and more sustainable.

Similar Jobs

Gusto Logo Gusto

Senior Software Engineer

Fintech • HR Tech
Easy Apply
Hybrid
6 Locations
4405 Employees
163K-247K Annually

Block Logo Block

Senior Software Engineer

Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
In-Office
8 Locations
12000 Employees
185K-327K Annually

Strava Logo Strava

Senior Software Engineer

News + Entertainment • Software • Sports
Easy Apply
In-Office
San Francisco, CA, USA
363 Employees
180K-200K Annually

Block Logo Block

Senior Software Engineer

Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
In-Office
8 Locations
12000 Employees
185K-327K Annually

Similar Companies Hiring

PRIMA Thumbnail
Travel • Software • Marketing Tech • Hospitality • eCommerce
US
15 Employees
Scotch Thumbnail
Software • Retail • Payments • Fintech • eCommerce • Artificial Intelligence • Analytics
US
25 Employees
Idler Thumbnail
Artificial Intelligence
San Francisco, California
6 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account