Senior Software Engineer - Husky Storage

Posted An Hour Ago
Easy Apply
Be an Early Applicant
2 Locations
Hybrid
192K-240K Annually
Senior level
Artificial Intelligence • Cloud • Security • Software • Cybersecurity
We are building the monitoring and security platform for developers, IT ops teams and business users in the cloud age.
The Role
Develop and operate Husky, a petabyte-scale distributed columnar event store. Lead migration to Parquet and DataFusion, improve compaction, compression, indexing, storage tiering, caching, and cost efficiency. Scale event versioning with temporal database techniques, collaborate across ingestion, metadata, platform, and lakehouse teams, and help define service operations. Contribute to open-source software and support analytical workloads through Iceberg-compatible integrations.
Summary Generated by Built In

Husky is what we call the distributed, petabyte-scale columnar event store at the heart of our Event Platform, which powers dozens of Datadog’s most popular products – Logs, RUM, APM, Cloud Network Monitoring, Netflow, and many more. Husky was built from the ground up at Datadog to store and query massive volumes of event data at low cost, with data fully queryable within seconds of arrival.

As a Senior Software Engineer on this team, you'll work hands-on with a distributed storage system at petabyte scale, and help us evolve our platform to support the next level of scale that will power the AI era. We will move from a custom storage format towards open standards (Parquet, Arrow, DataFusion) enabling on prem solutions, increasingly complex analytical/batch workloads, and powering AI observability, sustainably into the future. You'll partner closely with Husky Ingestion, Metadata, and Platform, and increasingly with Lakehouse on the Iceberg-compatible boundary between Husky and analytical/batch workloads. You will also work with the open source community to  build the next generation of reusable software solutions that can be operated at scale.

Datadog has a culture that is increasingly rare in mature technology companies. We are an engineering driven, bottom up, high agency/high responsibility culture–where everyone is empowered and expected to drive the technological and strategic direction of the business over time. We are a blameless culture where we believe supporting each other through challenges produces the best outcomes over time and our track record corroborates this. We are open source champions. We believe in the flexibility of working remotely but also the value of bringing people together across our various offices worldwide–and our offices are nice places to work.

 

What You’ll Do:

  • Drive Husky to the next level of adoption by migrating our storage format to open source Parquet and re-imagine compaction on a format-agnostic, DataFusion-based substrate
  • Drive concrete cost-reduction work: block-level inverted index compression, combining the database with external objects at the row level, and storage tiering/caching
  • Scale our solution to enable customers to manage event versioning at scale, using temporal-database techniques.
  • Play an influential role in operating and deciding how to best operate our service going forward.
  • Work across the write path (with Ingestion) and toward a cleaner Iceberg-based handoff to ADP/Lakehouse for batch/analytical workloads

Who You Are:

  • Enthusiastic and experienced with large-scale distributed systems and/or storage engines
  • Comfortable with improving and operating large distributed systems that customers rely on for critical business functions
  • Strong fundamentals in columnar/file storage formats (Parquet, ORC, Arrow) or comparable systems
  • Bonus: experience with Go, Java, or Rust; with compaction or compression/encoding schemes (RLE, dictionary, delta, gorilla); with object storage (S3/GCS/Azure Blob) at scale, including caching and cost optimization; with Kafka and stream processing; or with Kubernetes

Datadog values people from all walks of life. We understand not everyone will meet all the above qualifications on day one. That's okay. If you’re passionate about technology and want to grow your skills, we encourage you to apply. 


Benefits and Growth: 

  • New hire stock equity (RSUs) and employee stock purchase plan (ESPP)
  • Continuous professional development, product training, and career pathing
  • Intradepartmental mentor and buddy program for in-house networking
  • An inclusive company culture, access to Inclusion Talks
  • Free, global mental health benefits for employees and dependents age 6+
  • Competitive global benefits

Benefits and Growth listed above may vary based on the country of your employment and the nature of your employment with Datadog.

Datadog offers a competitive salary and equity package, and may include variable compensation. Actual compensation is based on factors such as the candidate's skills, qualifications, and experience. In addition, Datadog offers a wide range of best in class, comprehensive and inclusive employee benefits for this role including healthcare, dental, parental planning, and mental health benefits, a 401(k) plan and match, paid time off, fitness reimbursements, and a discounted employee stock purchase plan.

The reasonably estimated yearly salary for this role at Datadog is:
$192,000$240,000 USD

About Datadog: 

Datadog is the leading observability and security platform for the AI era, providing businesses with unified visibility across the technology stack to manage complexity at scale. It brings applications, infrastructure, data, models, and security into one place, using AI to detect and resolve issues before they impact customers. Trusted globally by Fortune 500 companies and high-growth AI leaders, Datadog enables businesses to move faster with clarity and confidence. Learn more about #DatadogLife on Instagram, LinkedIn, and Datadog Learning Center.

Equal Opportunity at Datadog:

Datadog is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and other characteristics protected by law. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. Here are our Candidate Legal Notices for your reference. 

Datadog endeavors to make our Careers Page accessible to all users. If you would like to contact us regarding the accessibility of our website or need assistance completing the application process, please complete this form. This form is for accommodation requests only and cannot be used to inquire about the status of applications. 

Privacy and AI Guidelines:

Any information you submit to Datadog as part of your application will be processed in accordance with Datadog’s Applicant and Candidate Privacy Notice. For information on our AI policy, please visit Interviewing at Datadog AI Guidelines.

Skills Required

  • Experience with large-scale distributed systems and/or storage engines
  • Experience improving and operating large distributed systems supporting critical customer business functions
  • Strong fundamentals in columnar or file storage formats such as Parquet, ORC, or Arrow, or comparable systems
  • Experience with Go, Java, or Rust
  • Experience with compaction or compression and encoding schemes such as RLE, dictionary, delta, or Gorilla
  • Experience with object storage such as S3, GCS, or Azure Blob at scale, including caching and cost optimization
  • Experience with Kafka and stream processing
  • Experience with Kubernetes

What the Team is Saying

Othmane
Angel
Emu
Tay
Norma
Sarah
LJ
Tammy
Olivia

Datadog Compensation & Benefits Highlights

  • Healthcare Strength Health coverage is described as comprehensive, with medical, dental, and vision plans plus added mental-health resources and wellness support. Feedback suggests virtual-care options and fitness reimbursements further strengthen this area.
  • Parental & Family Support Paid parental leave and broad family-forming benefits (e.g., fertility, adoption, surrogacy) are consistently highlighted. Feedback suggests extras like childcare help and pet-related benefits reinforce an inclusive family focus.
  • Equity Value & Accessibility Employees have access to equity via new-hire grants and a discounted Employee Stock Purchase Plan. Feedback suggests these ownership programs materially enhance total rewards for many roles.

Datadog Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: New York, NY
6,500 Employees
Year Founded: 2010

What We Do

Datadog (NASDAQ: DDOG) is a global SaaS business, delivering a rare combination of growth and profitability. We are on a mission to break down silos and solve complexity in the cloud age by enabling digital transformation, cloud migration, and infrastructure monitoring of our customers' entire technology stacks. Built by engineers, for engineers, Datadog is used by organizations of all sizes across a wide range of industries. Together, we champion professional development, diversity of thought, innovation, and work excellence to empower continuous growth. Join the pack and become part of a collaborative, pragmatic, and thoughtful people-first community where we solve tough problems, take smart risks, and celebrate one another.

Why Work With Us

At Datadog, we learn from and celebrate each other daily - each win is a team win. Datadogs solve tough problems, innovate pragmatically, and grow together. We promote from within, provide mentorship and opportunities for career development, and support our colleagues in the process. Best of all? We truly love what we do.

Gallery

Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery

Datadog Offices

Hybrid Workspace

Employees engage in a combination of remote and on-site work.

We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them and their team.

Typical time on-site: 3 days a week
HQNew York, NY
New South Wales
Company Office Image
MX
Amsterdam, NL
Bengaluru, IN
Company Office Image
Boston, MA
Denver, CO
Dublin, IE
Hanyang, KR
Lisbon, PT
United Kingdom
Madrid, ES
Company Office Image
Paris Office
San Francisco, CA
Singapore Office
Tokyo, JP
Learn more

Similar Jobs

Datadog Logo Datadog

Senior Sales Engineer

Artificial Intelligence • Cloud • Security • Software • Cybersecurity
Easy Apply
Remote or Hybrid
8 Locations
6500 Employees
149K-198K Annually

Datadog Logo Datadog

Field Enablement Manager, Sales Enablement

Artificial Intelligence • Cloud • Security • Software • Cybersecurity
Easy Apply
Hybrid
New York, NY, USA
6500 Employees
86K-115K Annually

Datadog Logo Datadog

Operations Associate

Artificial Intelligence • Cloud • Security • Software • Cybersecurity
Easy Apply
Hybrid
New York, NY, USA
6500 Employees
100K-133K Annually

Datadog Logo Datadog

Sales Revenue Analyst - NYC

Artificial Intelligence • Cloud • Security • Software • Cybersecurity
Easy Apply
Hybrid
New York, NY, USA
6500 Employees
79K-105K Annually

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account