Orion Innovation is a premier, award-winning, global business and technology services firm. Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and engineering, with a unique combination of agility, scale, and maturity. We work with a wide range of clients across many industries including financial services, professional services, telecommunications and media, consumer products, automotive, industrial automation, professional sports and entertainment, life sciences, ecommerce, and education.
Project Overview:
We're building a large-scale document intelligence platform that processes text files up to 5 TB in size, extracts insights using BERT-class NLP models, and surfaces answers to analysts via a low-latency query interface. The platform runs on Azure Kubernetes Service (AKS) with dedicated GPU node pools, uses KEDA for event-driven autoscaling, and integrates with Azure Data Lake Storage Gen2 and Azure OpenAI.
This is a hands-on role that sits at the intersection of platform engineering and applied ML, and requires someone who is equally comfortable debugging a CUDA out-of-memory error and designing a Kubernetes autoscaling policy. As the Senior ML Infrastructure Engineer the resource will own the end-to-end infrastructure layer — from GPU cluster configuration and CUDA runtime management to Kubernetes job orchestration and model serving.
Skill / Technology:
- Level: Kubernetes / AKS
- Expert: Multi-node-pool design, taint/toleration, autoscaler, GPU node pools (NC/ND series)
- Senior: Device plugin, driver compat, resource limits, KEDA
- Senior: Scaled Job, queue triggers, cooldown tuning, CUDA / cuDNN
- Mid–Senior: Runtime config via PyTorch; raw kernel dev not required, PyTorch (GPU inference)
- Senior: Batching, FP16, memory management, profiling, Hugging Face Transformers
- Senior: BERT/DistilBERT/BGE loading, pipeline API, tokenization, Python (production)
- Senior: Async workers, Azure SDK, queue consumers, Azure infrastructure
- Senior: VNet, private endpoints, Key Vault, ADLS, AD, Docker / Helm
- Senior: Multi-stage builds, Helm chart authoring, IaC (Terraform / Bicep)
- Preferred: willingness to learn is acceptable
Orion is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, sexual orientation, gender identity or expression, pregnancy, age, national origin, citizenship status, disability status, genetic information, protected veteran status, or any other characteristic protected by law.
Candidate Privacy Policy
Orion Systems Integrators, LLC and its subsidiaries and its affiliates (collectively, “Orion,” “we” or “us”) are committed to protecting your privacy. This Candidate Privacy Policy (orioninc.com) (“Notice”) explains:
- What information we collect during our application and recruitment process and why we collect it;
- How we handle that information; and
- How to access and update that information.
Your use of Orion services is governed by any applicable terms in this notice and our general Privacy Policy.
Skills Required
- Kubernetes and AKS administration including multi-node-pool design, taints/tolerations, autoscaler, and GPU node pools (NC/ND series)
- KEDA experience including ScaledJob, queue triggers, cooldown tuning
- CUDA and cuDNN runtime management and debugging (GPU OOM, memory management)
- PyTorch for GPU inference and runtime configuration
- Experience with Hugging Face Transformers and BERT/DistilBERT/BGE model loading and tokenization
- Python (production-grade) and async worker / queue consumer development
- Azure integration: Azure SDK, ADLS Gen2, Azure OpenAI, Key Vault, VNet, Private Endpoints, Active Directory
- Docker authoring including multi-stage builds and Helm chart authoring
- Infrastructure as Code: Terraform or Bicep
- Performance optimization: batching, FP16, profiling and memory management for large-model inference
- Experience with device plugins and driver compatibility for GPU workloads (NVIDIA device plugin)
- Willingness to learn new technologies and adapt
