Senior Software Engineer - Observability & IRM

Posted 15 Hours Ago
Be an Early Applicant
3 Locations
In-Office
125K-250K Annually
Senior level
AdTech
We help brands and their agencies deliver more relevant ads to audiences everywhere, across devices.
The Role
Build and operate incident management tooling, observability infrastructure, logging systems, developer portal integrations, and alert-quality automation. Responsibilities include automating incident lifecycles, evaluating logging vendors and architectures, extending Backstage with Kubernetes and SLO capabilities, improving alert routing and grouping, and supporting production debugging across distributed systems. The role collaborates with engineers company-wide to improve reliability and on-call effectiveness.
Summary Generated by Built In

The Trade Desk is a global technology company and the world’s leading independent platform for digital advertising, with nearly 4,000 employees across more than 30 offices. Our technology helps advertisers reach the right audiences across the open internet — from streaming TV and podcasts to mobile apps, news, and more. 
Advertising powers the content people love. By making it more transparent, effective, and responsible, we help support trusted journalism, quality entertainment, and creators worldwide. The world’s brands and agencies rely on us to reach their customers and grow their businesses responsibly.
The scale of our platform brings unique technical challenges — from processing massive datasets in real time to building systems that operate reliably on a global scale. When you work here, your impact is worldwide. We welcome diverse perspectives, encourage curiosity, and build teams that learn from one another. If you’re driven to solve meaningful challenges, we’d love to meet you.

About the Team:

The Service Excellence (SE) team owns the tools and infrastructure that help engineers at The Trade Desk understand and operate production systems. The Incident Response Services (IRS) taskforce focuses on the on-call experience.  The team is responsible for making incidents easier to detect, manage, and optimize using historical data points information.

What you will work on:

  • Incident management tooling
  • Build and maintain automation around the incident lifecycle: alerting, escalation, incident channels, retros, and SLA tracking
  • Help evaluate and migrate our logging stack
  • Participate in the re-evaluation of our logging vendor and collection architecture
  • Backstage/Service catalog — Extend our internal developer portal with K8s integrations, maturity models, and SLO  adoption tooling
  • Alert quality tooling — Build the systems that give engineers better signal and less noise — smarter routing, better grouping, tighter feedback loops between alerts and the teams that own them

Who you are: 

  • Experience building and operating production infrastructure or internal developer tooling
  • Comfort working across the stack — this role touches distributed systems, Kubernetes, observability pipelines, and web-based tooling
  • Familiarity with observability concepts: logging, alerting, on-call workflows
  • Strong debugging instincts: You will be expected to be called on when things break
  • Clear communication: The team works closely with engineers across the company; you'll need to explain tradeoffs and advocate for solutions

Plus skills:

  • Experience with Grafana, Prometheus, or similar observability tools
  • Familiarity with Sumo Logic or other log management platforms
  • Prior work on developer portals or service catalog tooling (Backstage, OpsLevel, etc.)
  • Experience with Kubernetes at scale
  • A deep understanding of HunnyPt

Variety of technical opportunity is one of the best things about working at The Trade Desk as a software engineer which is why we do not expect you to know every technology we use when you start. What we care about is that you can learn quickly and find solutions to complex problems using the optimum tools for the job.What you know is less important than how well you learn and innovate. We don't need engineers who know all the answers; we need engineers who can invent the answers no one has thought of yet, to the questions yet to be asked. 

The Trade Desk does not accept unsolicited resumes from search firm recruiters. Fees will not be paid in the event a candidate submitted by a recruiter without an agreement in place is hired; such resumes will be deemed the sole property of The Trade Desk. The Trade Desk is an equal opportunity employer. All aspects of employment will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, veteran status, or any other status protected under federal, state, or local law.

#LI-AD1

In accordance with various US state laws, the range provided is the Trade Desk's reasonable estimate of the base compensation for this role. The actual amount may differ based on non-discriminatory factors such as experience, knowledge, skills, and location.  All employees may be eligible to become The Trade Desk shareholders through eligibility for stock-based compensation grants, which are awarded to employees based on company and individual performance. The Trade Desk also offers other compensation depending on the role such as variable compensation-based incentives and commissions. Plus, expected benefits for this role include comprehensive healthcare (medical, dental, and vision) with premiums paid in full for employees and dependents, retirement benefits such as a 401k plan and company match, short and long-term disability coverage, basic life insurance, well-being benefits, reimbursement for certain tuition expenses, parental leave, sick time of 1 hour per 30 hours worked, vacation time for full-time employees up to 120 hours thru the first year and 160 hours thereafter, and around 13 paid holidays per year.  Employees can also purchase The Trade Desk stock at a discount through The Trade Desk’s Employee Stock Purchase Plan. 

The Trade Desk also offers a competitive benefits package. Click here to learn more.

Note: Interns are not eligible for variable incentive awards such as stock-based compensation, retirement plan, vacation, tuition reimbursement or parental leave

At the Trade Desk, Base Salary is one part of our competitive total compensation and benefits package and is determined using a salary range. The base salary range for this role is
$124,900$250,000 USD

As an Equal Opportunity Employer, The Trade Desk is committed to creating an inclusive hiring experience where everyone has the opportunity to thrive.

Please reach out to us at accommodations@​thetradedesk.​com to request an accommodation or discuss any accessibility needs you may require to access our Company Website or navigate any part of the hiring process. 

When you contact us, please include your preferred contact details and specify the nature of your accommodation request or questions. Any information you share will be handled confidentially and will not impact our hiring decisions.

Skills Required

  • Experience building and operating production infrastructure or internal developer tooling
  • Experience working across distributed systems, Kubernetes, observability pipelines, and web-based tooling
  • Familiarity with observability concepts, including logging, alerting, and on-call workflows
  • Strong debugging skills for production incidents
  • Clear communication skills, including explaining tradeoffs and advocating for solutions
  • Experience with Grafana, Prometheus, or similar observability tools
  • Familiarity with Sumo Logic or other log management platforms
  • Experience with developer portals or service catalog tooling such as Backstage or OpsLevel
  • Experience with Kubernetes at scale
  • Deep understanding of HunnyPt

The Trade Desk Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about The Trade Desk and has not been reviewed or approved by The Trade Desk.

  • Affordable Benefits Healthcare premiums are often paid in full for employees and dependents, reducing out-of-pocket costs. Coverage includes medical, dental, and vision alongside mental health and wellness support.
  • Equity Value & Accessibility Packages commonly include new-hire stock grants, a generous Employee Stock Purchase Plan, and equity refreshers tied to performance. Feedback suggests equity meaningfully boosts total compensation alongside base pay and bonuses.
  • Leave & Time Off Breadth Generous PTO, paid holidays, paid sick time, and sabbaticals are available. Paid parental leave and extensive family-planning support broaden time-off and caregiving coverage.

The Trade Desk Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Ventura, CA
2,700 Employees
Year Founded: 2009

What We Do

The Trade Desk is changing the way global brands and their agencies advertise to audiences around the world. Our mission is to transform media for the benefit of humankind. Our media-buying platform helps brands deliver a more insightful and relevant ad experience for consumers — and sets a new standard for reach, accuracy, and transparency.

Why Work With Us

With up to 11 million queries per second and SLAs in the milliseconds, we’re tackling data and software challenges at massive scale. While we’re growing fast, investing in our value based company culture remains one of our top priorities. If you’re talented, driven, and hungry to bring something entirely new and wildly ambitious, let's talk.

Gallery

Gallery

Similar Jobs

MongoDB Logo MongoDB

Technical Services Engineer - Weekend Shift

Big Data • Cloud • Software • Database
Easy Apply
Remote or Hybrid
United States
5550 Employees
90K-176K Annually

Digicomm International LLC Logo Digicomm International LLC

Inside Sales Representative

Hardware • Internet of Things • Manufacturing
Hybrid
Englewood, CO, USA
105 Employees
60K-70K Annually

Benchling Logo Benchling

Artificial Intelligence Engineer

Cloud • Healthtech • Social Impact • Software • Biotech
Remote or Hybrid
US
605 Employees
176K-265K Annually

Coursera + Udemy  Logo Coursera + Udemy

Director, FP&A Systems and Transformation

Artificial Intelligence • Consumer Web • Edtech • Enterprise Web • HR Tech • Social Impact • Generative AI
Remote or Hybrid
United States
1500 Employees
178K-243K Annually

Similar Companies Hiring

Grocery TV Thumbnail
Software • Retail • Marketing Tech • Hardware • Digital Media • AdTech
Austin, TX
56 Employees
Agentio Thumbnail
AdTech • Artificial Intelligence
New York, New York
65 Employees
ClickMint Thumbnail
AdTech • eCommerce • Marketing Tech • Generative AI
Malibu, CA
9 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account