Jobs at FriendliAI

Customer-facing engineer who deploys, scales, and operates LLM and multimodal inference workloads. Build and manage containerized deployments on Kubernetes, create Terraform/Helm automation, diagnose production performance issues, integrate with customers' CI/CD pipelines, and provide reliability and observability guidance through workshops and hands-on support.

Design, build, and operate a multi-cluster, multi-tenant Kubernetes fleet for GPU inference. Extend Kubernetes with controllers/CRDs, implement GPU scheduling and autoscaling, own the network data plane, design cross-cluster connectivity and service mesh, drive reliability/SLOs, deliver IaC with Terraform/Helm/GitOps, and collaborate with platform, SRE, and security teams.

Design, build, and operate a multi-cluster, multi-tenant Kubernetes fleet for GPU inference. Own cluster architecture, custom controllers and CRDs, GPU scheduling, autoscaling, Kubernetes networking (CNI, IPAM, DNS, load balancing), cross-cluster connectivity, service mesh, and reliability SLOs. Deliver infrastructure-as-code and collaborate with platform, SRE, and security teams to support latency-sensitive, high-traffic inference workloads.

Design, implement, and optimize high-performance GPU kernels (GEMM, attention, routing), develop CUDA/ROCm C++ code including low-level assembly, implement reduced-precision/quantized kernels (FP8/FP4), benchmark and ensure parity across NVIDIA and AMD, contribute to GPU libraries, accelerate multi-modal pipelines, and integrate next-generation GPU features into production inference engine.

Own and evolve core backend microservices for an AI inference platform: build production-grade APIs, multi-tenant SaaS features (auth, RBAC, billing), design OLTP/OLAP data models, collaborate on multi-cloud orchestration, ensure reliability/performance, and drive engineering quality through testing and CI/CD.

Design, build, and maintain agent APIs and production agent applications for document understanding, advanced RAG, and customer support automation. Integrate open-source models, collaborate with backend and infra for deployment and monitoring, and ensure APIs are robust, scalable, and developer-friendly.

Own and evolve core backend microservices for an AI inference platform, building production-grade APIs and multi-tenant SaaS capabilities (authentication, RBAC, billing). Design data models and pipelines across PostgreSQL and ClickHouse, collaborate on multi-cloud orchestration, ensure reliability and performance, and drive engineering quality through testing and CI/CD.

Build and maintain the Python SDK and cross-platform CLI for an AI inference platform. Own packaging/distribution, developer tooling, DevOps automation, documentation, examples, and collaborate across frontend, product, and engineering to deliver ergonomic APIs and top-tier developer experience.

Design and optimize high-performance GPU kernels (GEMM, attention, routing) for AI inference across NVIDIA and AMD GPUs. Implement CUDA/C++ and low-level assembly code, build reduced-precision/quantized (FP8/FP4) kernels, benchmark cross-vendor performance, contribute to internal GPU libraries, accelerate multi-modal pipelines, and integrate next-generation GPU features into production.

Act as technical liaison for customers using FriendliAI's inference platform: onboard developers, provide technical support, create documentation and tutorials, debug production issues with engineering, and run demos and Q&A sessions.

Design, build, and maintain a scalable web platform and APIs for deploying and monitoring multimodal AI models and agent workflows. Collaborate with product, infrastructure, and design teams to optimize performance, ensure reliability, drive CI/CD and testing, and contribute to long-term architecture decisions for a cloud-native, multi-tenant SaaS system.

Lead strategy and roadmap for FriendliAI's inference platform, owning initiatives end-to-end. Mentor junior PMs/designers, drive customer discovery, define product requirements and KPIs, partner with engineering/research, and align with GTM and sales to deliver scalable model APIs, deployment workflows, and developer features.

Lead end-to-end enterprise sales for FriendliAI's AI inference platform: generate pipeline, close high-value deals, run technical POCs, engage AI/ML communities, collaborate with engineering, and inform product roadmap.

Design, implement, and optimize GPU kernels, kernel compiler, memory planner, and runtime for low-latency generative AI inference. Analyze performance bottlenecks across hardware and software, collaborate with infrastructure teams, and maintain production profiling, benchmarking, and validation tooling while supporting new model architectures and multi-GPU strategies.

Design, deploy, and operate large-scale LLM and multimodal inference architectures. Work hands-on with customer engineering teams to containerize, scale, monitor, and troubleshoot GPU-based inference workloads across Kubernetes, CI/CD, and hybrid/on-prem environments. Create Helm charts, Terraform modules, and observability tooling while delivering workshops and platform reliability insights.

Design, build, and maintain agent APIs and production agent applications (document understanding, RAG, automation). Integrate open-source LLMs and multimodal models, collaborate with backend and infra teams for deployment, and ensure APIs are reliable, scalable, and developer-friendly with strong documentation and monitoring.

Build and optimize GPU kernels and core inference engine components (compiler, memory planner, runtime) for latency-critical generative AI workloads. Profile and benchmark performance, collaborate with cloud/infrastructure teams, support new model architectures and multi‑GPU/distributed inference, and maintain production-grade validation tools.