Staff Platform Engineer

What We’re Building

Honeycomb is the observability platform for teams who manage software that matters. Send any data to our one-of-a-kind data store, solve problems with all the relevant context, and fix issues before your customers find them. Honeycomb is the unified, fast, and collaborative choice for engineering teams who care about customer experience to get the answers they need, quickly. We are passionate about consumer-quality developer tools and excited to build technology that raises our industry’s expectations of what our tools can do for us. We’re working with well known companies like HelloFresh, Slack, LaunchDarkly, and Vanguard and more across a range of industries. This is an exciting time in our trajectory, as we’ve closed Series D funding, scaled past the 200-person mark, and were named to Forbes’ America’s Best Startups of 2022 and 2023!

The Honeycomb product handles enormous amounts of data, and the Core Services team builds and manages all the internals that make the product work. As a Core Services Engineer, you’ll be joining a growing Platform Engineering org, helping usscale and build resilient systems, shared components used by the rest of engineering, and the services that bring everything else together. You’ll shepherd millions of messages per second through our Kafka cluster, manage thousands of scheduled queries across our storage engine, and present the right abstractions around data storage to our application code.

Check out this recent blog post about Investigating Mysterious Kafka Broker I/O When Using Confluent Tiered Storage to see more about what this team has been up to!

Who We Are

How We Work

We are a remote-first company, which means we believe it is not where you sit, but how you deliver that matters most. We invest in our people and care about how you orient to our culture and processes. At the same time we imbue a lot of trust, autonomy, and accountability from Day 1. #LI-Remote

We can hire for this position in the United States and Canada

We are open to hiring this role as a Senior or Staff Engineer

What You'll Do

• Kafka Administration: Manage and optimize Kafka clusters, ensuring high availability, fault tolerance, and scalability. Troubleshoot and resolve issues related to Kafka performance, configuration, and integrations with downstream systems. Mentoring team members to distribute Kafka knowledge. Prior Kafka knowledge is required for this role.
• Scale high-throughput services: We are moving millions of events per second with low latency and growing. This role will let you explore what happens when straightforward individual tasks are suddenly running in between, on top of, and all around each other in interesting and complicated ways. You’ll help us ensure that the product flows smoothly even as it increases in volume and complexity.
• Database Management: Oversee the architecture, performance, and scaling of our databases. You and the team will be responsible for database tuning, query optimization, and ensuring data integrity and availability across different storage systems.
• Building New Services & Maintaining Existing Work: Design and implement new services that meet business and technical requirements. You’ll also take ownership of maintaining and evolving existing services, ensuring they continue to meet the growing demands of our organization while adhering to high standards of reliability and performance.
• Work well with other teams: You'll collaborate cross-functionally with product managers, designers, and other engineering teams to deliver high-impact solutions. Your ability to build relationships and facilitate smooth communication across teams will be crucial in helping the organization achieve its goals efficiently and effectively.

What You'll Bring

• You can debug both automated and human processes. The systems that let our service run are both technical and social. Building processes to facilitate effective responses to service failures from the engineers running these services gives us room to automate responses.
• You can work in both software engineering and operations. Deeply understanding how internet services run and scale is critical to ensuring their reliability. Building systems to help them scale is critical to helping the team move quickly. A Core Services Engineer must live in harmony with both worlds to succeed, pinch hitting across our stack from the storage engine itself, our automation around it, and the configuration of our infrastructure. The ideal candidate would probably enjoy a bit of variety in their work and not find it too distracting to help out with other projects from time to time.
• You can find balance in all things. Distributed systems are complicated creatures and sometimes need complicated tools to support them. Sometimes the added complication is not worth the benefit. This role can help cast a critical eye on balancing complication and simplicity, building and buying technology, spending innovation tokens where needed and choosing boring technology when it’s not.
• You let curiosity run wild. Investigating, learning, experimenting, and trying new ways/things/methods is part of your DNA. You do this because it brings you joy, and you relish bringing others on the ride.
• You see toil as opportunity for change. It is easy for people to patch holes in process or technical implementation for too long. Spotting these opportunities, shortening their feedback loops, and building tools or automation to remove entire classes of problems will help us scale our company and product to the next order of magnitude.

What You Get When You Join the Hive

• Base pay (range) of $190,000 - $235,000 USD (Senior through Staff leveling)
• A stake in our success - generous equity with employee-friendly stock program
• It’s not about how strong of a negotiator you are - our pay is based on transparent levels relative to experience
• Time to Recharge - In addition to our Unlimited PTO policy, we have a company wide break at the end of the year
• A remote-first mindset and culture (really!)
• Home office, co-working, and internet stipend
• 100% employee/75% for dependents coverage for all benefits 
• Up to 16 weeks of paid parental leave, regardless of path to parenthood
• Annual development allowance 
• And much more...

Please note we cannot currently sponsor or do visa transfers at this time.

Diversity & Accommodations:

We're building a diverse and inclusive workplace where we learn from each other, and welcome nontraditional candidates, and people of all backgrounds, experiences, abilities and perspectives. You don't need to be a millennial to join us. All gens are welcome! Further, we (of course) follow federal and state disability laws and are happy to provide reasonable accommodations during the application phase, interview process, and employment. Please email [email protected] to discuss accessible formats or accommodations. As an equal opportunity employer our hiring process is designed to put you at ease and help you show your best work; if we can do better - we want to know!