What You Will Do:
Optimize Inference: Build and tune production LLM serving with vLLM and SGLang—maximizing throughput and minimizing latency through batching, paged attention, quantization, and KV-cache strategies
Profile & Accelerate Training: Instrument and profile training runs to find bottlenecks, then resolve them with the right attention implementations (e.g. FlashAttention) tuned to the underlying hardware (H200, GB200)
Engineer for the Hardware: Apply a working understanding of GPU architecture and attention internals to choose the right approach per accelerator, rather than relying on defaults
Serve at Scale: Deploy and operate multiple models within shared GPU clusters on GKE, with autoscaling, efficient bin-packing, and graceful handling of mixed workloads
Drive Efficiency: Own GPU utilization as a first-class metric—measure it, improve throughput-per-dollar, and continuously raise the ceiling on what our fleet can deliver
Collaborate & Consult: Work directly with clients to understand performance, latency, and cost requirements, and translate them into pragmatic serving and training architectures
Your Technical Toolkit:
Core Languages: Mastery of Python and shell scripting; comfort reading and reasoning about lower-level (CUDA-adjacent) performance code is a strong plus
Inference Frameworks: Hands-on experience with vLLM, SGLash, or comparable high-performance serving stacks
GPU & Model Internals: Solid grasp of GPU architecture, the fundamentals of LLM inference, and the attention mechanism—including where the bottlenecks live and how FlashAttention and similar techniques address them across hardware generations (H200, GB200)
Profiling: Fluency with profiling tools to diagnose training and inference bottlenecks (compute-bound vs. memory-bound, kernel-level analysis)
Infrastructure: Strong Kubernetes (GKE) experience—deploying and autoscaling multiple models on shared GPU clusters on Google Cloud
Mindset: A strong software engineering foundation—you write clean, maintainable code, measure before optimizing, and understand the full SDLC
Basic Qualifications:
Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field
5+ years of experience in ML/AI engineering, with a meaningful portion focused on performance, infrastructure, or systems
Proven track record of deploying and optimizing models in a production environment
Demonstrated experience profiling and improving GPU utilization for training and/or inference
Experience with Classic Machine Learning (neural nets, training, tuning) is a strong plus
Knowledge of Data Engineering and SQL
Personal Attributes:
Ownership: You take pride in your work and see optimizations through from profile to production
Curiosity: Hardware and serving frameworks change fast; you are a lifelong learner who stays ahead of the curve
Rigor: You measure before you optimize and let data, not intuition, guide where you spend effort
Consultative Spirit: You enjoy interacting with clients and can translate technical complexity into business value
Ethics: You prioritize responsible AI development and data privacy
Skills Required
- Bachelor's or Master's degree in Computer Science, Engineering, or related technical field
- 5+ years of experience in ML/AI engineering with focus on performance, infrastructure, or systems
- Proven track record deploying and optimizing models in production
- Demonstrated experience profiling and improving GPU utilization for training and/or inference
- Mastery of Python and shell scripting
- Hands-on experience with vLLM, SGLang, SGLash, or comparable high-performance serving stacks
- Strong Kubernetes (GKE) experience deploying and autoscaling models on shared GPU clusters
- Solid grasp of GPU architecture, LLM inference fundamentals, and attention mechanisms (e.g., FlashAttention)
- Fluency with profiling tools to diagnose training and inference bottlenecks (compute vs memory bound, kernel analysis)
- Knowledge of Data Engineering and SQL
- Comfort reading and reasoning about lower-level (CUDA-adjacent) performance code
- Experience with classic machine learning (neural nets, training, tuning)
- Experience tuning for accelerators such as H200 or GB200
What We Do
Egen is a data engineering and cloud modernization firm partnering with leading Chicagoland companies to launch, scale, and modernize industry-changing technologies. We are catalysts for change who create digital breakthroughs at warp speed. Our team of cloud and data engineering experts are trusted by top clients in pursuit of the extraordinary. Our mission is to be an enabler of amazing possibilities for companies looking to use the power of cloud and data. We want to stand shoulder to shoulder with clients, as true technology partners, and make sure they succeed at what they have set out to do. We want to be disruptors, game-changers, and innovators who have played an important part in moving the world forward.
