Software Engineer, AI Gateway

Posted 26 Days Ago
Be an Early Applicant
Milan, ITA
Hybrid
Mid level
Artificial Intelligence • Cloud • Information Technology • Software • Big Data Analytics
The AI Connectivity Company
The Role
Build and maintain Kong’s AI Gateway software, including multi-LLM routing, semantic caching, MCP, proxying, load balancing, database support, and deployment integrations. Troubleshoot bugs, tune performance, optimize low-level systems, and develop scalable, distributed, reliable services. The role requires server-side programming, Linux systems and networking expertise, and familiarity with AI and LLM applications. Engineers also contribute to product documentation while collaborating with a distributed team.
Summary Generated by Built In

Are you ready to unlock intelligence?

If you don’t think you meet all of the criteria below but are still interested in the job, please apply. Nobody checks every box - we’re looking for candidates that are particularly strong in a few areas, and have some interest and capabilities in others.

The Mission

You will join the engineering team that builds the Kong AI gateway, empowering companies and developers around the world to govern every step of their AI & LLM based innovation. As a Software Engineer in the AI Gateway Team, you will be working on core features such as multi-LLM routing and balancing, semantic caching and routing, MCP, and more.

What You’ll Do
  • Partake in the entire lifecycle of core AI Gateway features, from design to production rollout.

  • Build and maintain high-performance components for Kong’s AI Gateway in Rust and Lua.

  • Implement gateway features such as multi-LLM routing and load balancing, caching, guardrails, and observability for AI traffic.

  • Design integrations with LLM providers and AI platforms, filling the gaps between Kong and third-party deployment platforms and tooling.

  • Troubleshoot production issues in the hot path.

  • Work with product management and other engineering teams to turn customer requirements into reliable, documented product features.

What You’ll Bring

The Toolkit
  • 3+ years of programming in server-side applications and components, in a language such as Rust or Go.

  • Experience building and debugging applications that are highly scalable, distributed, reliable and resilient.

  • Knowledge of networking fundamentals and protocols on both L4 and L7 (TCP/IP, HTTP/2, gRPC).

  • Familiarity with cutting-edge AI models concepts, agentic AI protocols, and AI applications development.

  • Solid hands-on expertise in Linux systems.

The Kong DNA
  • Ownership: You have a strong sense of ownership, including the documentation: engineers at Kong write the documentation for their products.

  • Collaboration: You have excellent verbal and written communication skills and are comfortable working with a remotely distributed team.

Bonus Points
  • Proficiency with Lua.

  • Hands-on experience with NGINX and OpenResty.

  • Familiarity with cloud-native environments (Kubernetes, Docker, AWS/GCP/Azure).

#LI-EA1

About Kong:

Kong Inc., the AI Connectivity Company, is building the connectivity layer of AI. Trusted by the Fortune 500® and AI-native startups alike, Kong’s unified API and AI platform enables organizations to secure, manage, accelerate, govern, and monetize the flow of intelligence across APIs and AI traffic — on any model, any cloud. For more information, visit www.konghq.com.

Skills Required

  • 3+ years of programming experience in server-side applications and components
  • Hands-on experience with Golang or Rust
  • Experience building and debugging highly scalable, distributed, reliable, and resilient applications
  • Low-level technical expertise in Linux systems troubleshooting and profiling
  • Experience with distributed systems, the Linux networking stack, and computer systems programming
  • Knowledge of APIs across Layer 7 and Layer 4 transports and protocols
  • Ability to work comfortably with a remotely distributed team
  • Strong ownership and excellent verbal and written communication skills
  • Familiarity with AI, LLM concepts, and application development
  • Hands-on experience with NGINX, OpenResty, Kong, or LuaJIT
  • Hands-on experience with Envoy or Istio

Kong Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Kong and has not been reviewed or approved by Kong.

  • Leave & Time Off Breadth — Flexible PTO, company-wide Unplug Days, and a four‑week paid sabbatical after five years are prominently offered to support rest and work–life balance. These programs are presented as meaningful perks beyond base pay and are highlighted across official materials and third‑party summaries.
  • Healthcare Strength — U.S. medical, dental, and vision coverage is described as strong, with plan quality drawing positive sentiment. Solid healthcare appears to contribute meaningfully to overall satisfaction with the total package.
  • Equity Value & Accessibility — Stock options are offered to most employees, creating ownership upside alongside cash compensation. Equity is positioned as a visible component that helps keep offers competitive for certain technical and go‑to‑market roles.

Kong Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: San Francisco, California
1,000 Employees
Year Founded: 2017

What We Do

Kong Inc., a leading developer of API and AI connectivity technologies, is building the connectivity layer for AI. Trusted by the Fortune 500 and AI-native startups alike, Kong’s unified API and AI platform enables organizations to secure, manage, accelerate, govern, and monetize the flow of intelligence across APIs and AI traffic — on any model, any cloud. No AI without APIs.

Why Work With Us

It starts with how we show up for each other. We’ve created a workplace that’s intentionally flexible, deeply inclusive, and built for meaningful collaboration — whether virtual or in person. We trust our teams to own their work, and we give them the support, tools, and freedom to grow.

Gallery

Gallery

Similar Jobs

Kong Logo Kong

Senior Software Engineer

Artificial Intelligence • Cloud • Information Technology • Software • Big Data Analytics
Hybrid
Milan, ITA
1000 Employees

Kong Logo Kong

Senior Software Engineer

Artificial Intelligence • Cloud • Information Technology • Software • Big Data Analytics
Hybrid
Milan, ITA
1000 Employees

Kong Logo Kong

Software Engineer

Artificial Intelligence • Cloud • Information Technology • Software • Big Data Analytics
Hybrid
Milan, ITA
1000 Employees

Pfizer Logo Pfizer

Director, AI Product Lead

Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Remote or Hybrid
28 Locations
121990 Employees

Similar Companies Hiring

Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
33 Employees
Vega Thumbnail
Artificial Intelligence • Automotive • Insurance • Transportation
US
65 Employees
RapidScale Thumbnail
Cloud • Information Technology • Consulting
Raleigh, NC
600 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account