Staff Software Engineer, OTA Services

Posted Yesterday
Be an Early Applicant
Hiring Remotely in US
Remote
230K-260K Annually
Senior level
Artificial Intelligence • Cybersecurity
The Role
Design and own NodeZero’s over-the-air update infrastructure across thousands of customer deployments. Responsibilities include delivery protocols, version negotiation, rollout orchestration, health verification, rollback, fleet state management, constrained-environment delivery, reliability, incident response, architecture, production coding, and mentoring engineers. The role requires secure, highly reliable distributed systems that operate across air-gapped and customer-managed Kubernetes environments.
Summary Generated by Built In

Get to Know Us

Horizon3 is a fast-growing, remote cybersecurity company dedicated to the mission of enabling organizations to proactively find, fix and verify exploitable attack vectors before criminals exploit them. Our flagship product, the NodeZero™ platform, delivers production-safe autonomous pentests and other key assessment operations that scale across the largest internal, external, cloud, and hybrid cloud environments. NodeZero has been adopted by organizations of all sizes, from small educational institutions to government agencies and Global 100 enterprises. It is used by IT Ops/SecOps teams, consulting pentesters, and MSSPs and MSPs.

We are a fusion of former U.S. Special Operations cyber operators, startup engineers & operators, and formerly frustrated cybersecurity practitioners. We're committed to helping solve our common security problems: ineffective security tools and false positives, resulting in alert fatigue, blind spots, "checkbox" security culture, cybersecurity skills shortage, and the long lead time and expense of hiring outside consultants. Collectively, we are a team of learn-it-alls, committed to a culture of respect, collaboration, ownership, and results.

As a remote first company, we require minimum 25Mbps consumer grade broadband connection.

 

What You'll Do

The Expert Engineer on the OTA Services team is the technical anchor for how NodeZero updates itself across thousands of customer deployments — without customer intervention, without downtime, and without introducing new vulnerabilities in the process. NodeZero runs as a container inside customer infrastructure. When a new image is ready, the OTA system has to find every deployed instance, negotiate the update, deliver it safely, validate it, and roll back gracefully if something goes wrong. Getting that right at scale, across diverse customer environments including air-gapped and restricted networks, is a genuinely hard systems problem.

This is a hands-on staff-level role. You will write production code, produce architecture designs, set the technical direction for update delivery and version management, and mentor P4 engineers on the team. The OTA Services team works in close partnership with the Image Build & Release team — your counterpart on the build side produces the artifacts you deliver. This role reports to the Client Platform Engineering Manager.

Responsibilities

  • OTA Architecture: Design and own the over-the-air update system — the delivery protocol, version negotiation, rollout orchestration, health verification, and automatic rollback

  • Deployment Fleet Management: Own the systems that track the version state of every NodeZero deployment — know what's running where, what's pending, and what's stuck

  • Rollout Strategy: Design safe rollout strategies for updates — phased rollouts, canary deployments, customer pinning, and emergency hold mechanisms

  • Constrained Environment Delivery: Design delivery mechanisms that work reliably across diverse customer environments including air-gapped networks, strict egress policies, and minimal-footprint deployments

  • Reliability & Incident Response: Own OTA system reliability end-to-end — on-call for delivery failures, stuck updates, and version drift incidents

  • Mentorship: Mentor P4 engineers through design review, code review, and technical pairing; be the person the team turns to when delivery semantics get complicated

 

What You'll Bring

Distributed Systems & Delivery

  • 8+ years of professional software engineering experience with deep specialization in distributed systems, remote update systems, or fleet management infrastructure

  • Deep understanding of remote update system design — delivery protocols, version negotiation, delta delivery, health verification, and rollback semantics

  • Experience designing rollout strategies for software deployed across heterogeneous, customer-managed environments — you understand the difference between a lab rollout and rolling something out to thousands of production deployments you don't control

  • Strong proficiency in Go, Python, or Rust for systems-level infrastructure work

Platform & Infrastructure

  • Experience building software that operates in constrained or air-gapped environments — you know how to design delivery systems that don't assume good connectivity

  • Deep understanding of Kubernetes — how deployments are managed, updated, and monitored in running clusters

  • Experience with container image distribution: OCI artifacts, image manifests, layer caching, and registry protocol

  • Strong understanding of mTLS, certificate lifecycle, and secure channel establishment for update delivery

Technical Leadership

  • Track record of setting engineering standards that other engineers adopt — protocol designs, reliability requirements, and testing standards for distributed systems

  • Ability to produce clear architecture documents and drive technical decisions across team boundaries (especially with the Image Build team that produces your input artifacts)

  • Experience mentoring mid-level engineers through design review and code review

 

Nice to Haves

  • Experience with Kubernetes operators or controller patterns — OTA delivery in Kubernetes environments often benefits from operator-style lifecycle management

  • Background in cybersecurity or experience building software for regulated or high-security environments

  • Familiarity with Fleet (or similar open-source fleet management tools) and where custom infrastructure is preferable

  • Experience with signed artifact delivery and chain-of-custody verification end-to-end

 

Travel Required

We are a fully remote company, and this job may require up to 5% of travel to be successful.

 

Compensation and Values

At Horizon3, we believe that our people are our greatest asset, and our compensation philosophy reflects this core value. We are committed to fostering an environment where all employees feel valued, respected, and rewarded for their contributions. Our compensation structure is designed to be fair, competitive, and transparent, ensuring that every team member is recognized and compensated equitably across roles, levels, and locations.

In accordance with various State's transparency regulations, we provide the following salary range information for this position:

  • Base salary range: $229,500 - $260,000 annually. The exact salary will be determined based on the selected candidate's location, qualifications, experience, and relevant skills.

  • Additional compensation: This role may also be eligible for an equity package (in the form of stock options). If any other compensation benefits apply, they will be discussed during the interview process.

 

Perks of Horizon3

  • Inclusive Team: We value diversity and promote an inclusive culture where everyone can thrive.

  • Growth Opportunities: Be part of a dynamic and growing team with numerous career development opportunities.

  • Innovative Culture: Work in a collaborative environment that encourages creativity and out-of-the-box thinking.

  • Remote Work: We are a 100% remote company. Enjoy the flexibility to work in the way that supports you and brings out your best.

  • Competitive Compensation: We offer competitive salary and benefits which includes health, vision & dental care for you and your family, a flexible vacation policy, and generous parental leave.

 

You Belong Here

Horizon3 is not just an equal opportunity employer - we are a community that values diversity, equity, and inclusion as fundamental principles of our culture and success. We are dedicated to fostering a workplace where everyone feels welcome and respected, regardless of race, color, religion, sex, national origin, age, disability, veteran status, sexual orientation, gender identity or expression, genetic information, marital status, or any other legally protected status by law.

Our commitment to diversity and inclusion means we strive to attract, develop, and retain a workforce that reflects the varied communities we serve. We believe that diverse perspectives drive innovation and strengthen our ability to create cutting-edge cybersecurity solutions. At Horizon3, every team member is valued and supported in an environment that encourages personal and professional growth.

We welcome candidates from all backgrounds and experiences, and we encourage all qualified individuals to apply. Come be a part of Horizon3, where your unique contributions are recognized, and your potential is limitless.

 
 

Other Duties

Please note this job description is not designed to cover or contain a comprehensive listing of activities, duties or responsibilities that are required of the employee. Duties, responsibilities, and activities may change at any time with or without notice.

 
 

Application Note

In any materials you submit, you may redact or remove age-identifying information such as age, date of birth, or dates of school attendance or graduation. You will not be penalized for redacting or removing this information.

 

Skills Required

  • 8+ years of professional software engineering experience with deep specialization in distributed systems, remote update systems, or fleet management infrastructure
  • Deep understanding of remote update system design, including delivery protocols, version negotiation, delta delivery, health verification, and rollback semantics
  • Experience designing rollout strategies for software deployed across heterogeneous, customer-managed environments
  • Strong proficiency in Go, Python, or Rust for systems-level infrastructure work
  • Experience building software for constrained or air-gapped environments
  • Deep understanding of Kubernetes deployments, updates, and monitoring
  • Experience with container image distribution, OCI artifacts, image manifests, layer caching, and registry protocols
  • Strong understanding of mTLS, certificate lifecycle, and secure channel establishment
  • Track record of setting engineering standards adopted by other engineers
  • Ability to produce architecture documents and drive technical decisions across team boundaries
  • Experience mentoring mid-level engineers through design and code reviews
  • Experience with Kubernetes operators or controller patterns
  • Background in cybersecurity or software development for regulated or high-security environments
  • Familiarity with Fleet or similar open-source fleet management tools
  • Experience with signed artifact delivery and end-to-end chain-of-custody verification

Horizon3.ai Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Horizon3.ai and has not been reviewed or approved by Horizon3.ai.

  • Fair & Transparent Compensation — Pay is considered competitive across key roles, with employer language and role ranges positioning compensation as market-aligned. Feedback suggests compensation is viewed favorably, particularly in technical positions.
  • Equity Value & Accessibility — Equity is positioned as a core part of total rewards, with stock options broadly available to full-time employees. Feedback suggests equity and related programs enhance overall compensation.
  • Leave & Time Off Breadth — Time off is described as at least four weeks plus generous holidays and vacation packages globally. Feedback suggests this breadth of leave contributes to positive perceptions of total rewards.

Horizon3.ai Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: San Francisco, CA
107 Employees
Year Founded: 2019

What We Do

Horizon3.ai's mission is to help you find and fix attack vectors before attackers can exploit them. NodeZero, our autonomous penetration testing solution, enables organizations to continuously assess the security posture of their enterprise, including external, identity, on-prem, IoT, and cloud attack surfaces. Like APTs, ransomware, and other threat actors, our algorithms discover and fingerprint your attack surface, identifying the ways exploitable vulnerabilities, misconfigurations, harvested credentials, and dangerous product defaults can be chained together to facilitate a compromise. NodeZero is a true self-service SaaS offering that is safe to run in production and requires no persistent or credentialed agents. You will see your enterprise through the eyes of the attacker, identify your ineffective security controls, and ensure your limited resources are spent fixing problems that can actually be exploited. Founded in 2019 by industry, US Special Operations, and US National Security veterans, Horizon3.ai is headquartered in San Francisco, CA, and made in the USA.

Similar Jobs

Optum Logo Optum

Technical Product Manager

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office or Remote
Eden Prairie, MN, USA
160000 Employees
113K-193K Annually

Optum Logo Optum

Director of AI Transformation and Business Modernization - Remote

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office or Remote
Minnetonka, MN, USA
160000 Employees
135K-231K Annually

Optum Logo Optum

Senior Director, Technology Product Management- Health Insurance Platforms & Underwriting - Remote

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office or Remote
Minnetonka, MN, USA
160000 Employees
159K-273K Annually

Optum Logo Optum

Consultant

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office or Remote
Irvine, CA, USA
160000 Employees
92K-164K Annually

Similar Companies Hiring

Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
33 Employees
Vega Thumbnail
Artificial Intelligence • Automotive • Insurance • Transportation
US
65 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account