NVIDIA has continuously reinvented itself for more than two decades. The invention of the GPU in 1999 fueled the growth of the PC gaming market, redefined modern computer graphics, and revolutionized parallel computing. GPU-powered deep learning helped ignite the era of modern AI, establishing GPUs as the foundation of intelligent applications across productivity, gaming, and creative workflows, and reinforcing NVIDIA’s position as a leading AI computing company.
More recently, there is a growing focus on running AI models locally, closer to where data is generated. This approach reduces latency, enables real-time processing, and addresses privacy concerns by minimizing the need to send data to centralized servers. As technology continues to evolve, client-side AI will play an increasingly important role in shaping the digital landscape. The LocalAI team is seeking a Senior Systems Software Engineer to develop efficient on-device AI software for RTX and DGX-class systems. The role focuses on delivering high-performance local inference with low latency, optimized memory utilization, robust infrastructure, and practical deployment on resource-constrained platforms.
What You’ll Be Doing:
Partner with NVIDIA’s software, research, architecture, and product teams to align technical requirements and strategic priorities, fostering the AI ecosystem on RTX and DGX PCs.
Build and optimize the local AI inference stack for RTX, RTX Pro, and DGX GPUs, with a focus on performance, stability, and scalability across diverse hardware architectures.
Design and develop modern inference runtimes and execution stacks using frameworks such as llama.cpp, vLLM, PyTorch, WinML, DXCGC, and TensorRT-RTX, supporting LLM, vision-language, TTS, ASR, and diffusion-based AI workloads.
Perform end-to-end optimization of AI models, data pipelines, and inference runtimes to maximize performance on current and next-generation GPU architectures. Apply model optimization techniques, including quantization, pruning, sparsity, and distillation, to enable efficient deployment of large models on local and edge devices.
Conduct system-level debugging, performance tuning, and performance-accuracy trade-off analysis; develop infrastructure for performance and accuracy sweeps; analyse results to identify gaps and drive fixes; and establish engineering guidelines to accelerate bring-up and ensure production readiness of new models and inference backends.
What we need to see:
5+ Years of experience with Bachelor’s, Master’s, or PhD in Computer Science, Software Engineering, Mathematics, or a related field, or equivalent experience.
Excellent C++ programming and debugging skills, with a strong foundation in data structures, algorithms, and machine learning.
Proven experience developing and optimizing AI inference pipelines and applications using ML/DL frameworks such as Llama.cpp, vLLM, PyTorch, Windows ML, DXCGC, and TensorRT.
Deep understanding of inference backends and runtime internals, including scheduling, memory management, KV-cache behaviour, graph execution, quantization, and hardware-aware optimization.
Strong analytical and problem-solving skills, with the ability to manage multiple priorities effectively in a fast-paced environment.
Excellent written and verbal communication skills, enabling effective collaboration across engineering teams and management.
Ways to stand out from the crowd:
Understanding of modern machine learning, deep neural network, and generative AI techniques, with relevant contributions to major open-source projects.
Consistent track record of delivering end-to-end products in multinational companies with geographically distributed teams.
Proficiency in low-level system and GPU programming, CUDA, and the development of high-performance systems.
Contributions to open-source inference runtimes, model tooling, or performance infrastructure.
Hands-on experience building applications using frameworks and APIs such as llama.cpp, PyTorch, TensorRT, Vulkan, DirectX, and vLLM.
We're a top employer recognized for innovation, growth, and a commitment to diversity as an equal-opportunity workplace. We offer competitive salaries, a generous benefits package, and the opportunity to work alongside some of the technology industry's most talented and forward-thinking professionals. As our engineering teams continue to grow rapidly, we're looking for creative, self-driven engineers with a passion for technology to join us.
Skills Required
- 5+ years of experience with a degree in Computer Science or related field, or equivalent experience
- Excellent C++ programming and debugging skills, strong foundation in data structures and algorithms
- Proven experience developing and optimizing AI inference pipelines and applications using frameworks such as llama.cpp, vLLM, PyTorch, Windows ML, DXCGC, and TensorRT
- Deep understanding of inference backends and runtime internals (scheduling, memory management, KV-cache behavior, graph execution, quantization, hardware-aware optimization)
- Strong analytical and problem-solving skills; ability to manage multiple priorities in a fast-paced environment
- Excellent written and verbal communication skills for cross-team collaboration
- Proficiency in low-level system and GPU programming and CUDA
- Contributions to open-source inference runtimes, model tooling, or performance infrastructure
- Hands-on experience with Vulkan, DirectX, and building applications using llama.cpp, PyTorch, TensorRT, vLLM
- Track record delivering end-to-end products in multinational, distributed teams
NVIDIA Compensation & Benefits Highlights
The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about NVIDIA and has not been reviewed or approved by NVIDIA.
-
Equity Value & Accessibility — Company stock and RSUs have driven exceptional total compensation, with a generous employee stock purchase plan and ongoing equity refreshers playing a central role. Feedback suggests historic share-price gains created significant financial upside for many.
-
Healthcare Strength — Multiple plan options (HSA, PPO, and in some locations Kaiser) with in-network preventive care covered at 100% and company HSA contributions indicate a robust offering. Mental-health sessions and broad dental, vision, and disability coverage further strengthen the package.
-
Parental & Family Support — Up to 22 weeks of fully paid parental leave, adoption and surrogacy reimbursement, and coverage for IVF and egg freezing are highlighted as standout features. Additional family-care resources and caregiving support broaden the appeal.
NVIDIA Insights
What We Do
NVIDIA’s invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined modern computer graphics, and revolutionized parallel computing. More recently, GPU deep learning ignited modern AI — the next era of computing — with the GPU acting as the brain of computers, robots, and self-driving cars that can perceive and understand the world. Today, NVIDIA is increasingly known as “the AI computing company.”
