Locations: San Francisco or Remote
About The Role
The NEAR AI team is building decentralized and confidential machine learning infrastructure to enable user-owned AI. Our mission is to build highly scalable and efficient infrastructure for open-source AI at a global scale.
We are specifically seeking an expert in high-performance LLM serving systems and inference optimization. In this role, you will push the boundaries of how large language models are served.
What You'll Be Doing
- Architect and maintain production high-traffic LLM serving systems.
- Optimize throughput, latency, and cost for leading open-source LLMs.
What We're Looking For
- Strong hands-on experience in LLM inference, with expertise debugging and optimizing major inference engines such as SGLang, vLLM, or TensorRT.
- Deep knowledge of state-of-the-art GPU architectures, and effectively exploit them using PyTorch, Triton, CuTe, CUDA, etc.
- Proven track record in designing and maintaining end-to-end high-traffic LLM serving systems.
- Strong problem-solving skills and ability to communicate technical ideas clearly.
We'd Love If You Have
- Experience with Trusted Execution Environments (TEE).
- Active contributor to open-source LLM inference engines.
Please let us know if you require any special requirements for your interview and we'll do our best to accommodate.
Skills Required
- Hands-on experience in LLM inference and debugging/optimizing inference engines such as SGLang, vLLM, or TensorRT.
- Deep knowledge of GPU architectures and experience exploiting them with PyTorch, Triton, CuTe, CUDA.
- Proven track record designing and maintaining end-to-end high-traffic LLM serving systems.
- Strong problem-solving skills and ability to communicate technical ideas clearly.
- Experience with Trusted Execution Environments (TEE).
- Active contribution to open-source LLM inference engines.
What We Do
NEAR AI is an artificial intelligence research, engineering, and product development company committed to building an AI future owned by everyone. Founded by AI pioneer and former Google Deepmind researcher Illia Polosukhin, NEAR AI’s verifiable private inference infrastructure empowers developers and enterprises to deploy AI models with full control over their data. With hardware-backed private inference via a simple API, NEAR AI Cloud runs sensitive AI workloads securely and at scale, from privacy-critical consumer interactions to autonomous systems and critical infrastructure. NEAR AI Private Chat brings the same guarantees to users’ everyday questions and research. Serving over 100 million users across platforms such as Brave Nightly and OpenMind, NEAR AI is proven infrastructure for transforming sensitive data into safe intelligence and advancing a user-owned AI future. Learn more at https://near.ai/.
