The Role
Responsible for defining product roadmaps for AI inference capabilities, collaborating with engineers to optimize resources and improve latency and throughput. Engage with technical users and validate features with customers.
Summary Generated by Built In
About Alluxio:
Proven at a global scale in production for modern AI and data services, Alluxio is the premier developer of data orchestration software. Alluxio is in production use today at eight out of the top ten internet companies, and seven of the ten highest valued companies in the world. Our mission is to orchestrate data for all data driven applications in any cloud!
As customers continuously push the envelope for ways to extract value from data, Alluxio is driving innovation in data access and management for large-scale distributed systems. Alluxio is charging full steam to help customers navigate the digital transformation. Venture-backed by Andreessen Horowitz, Hillhouse Capital, and Seven Seas Partners. Alluxio was founded at UC Berkeley’s AMPLab by the creators of the Tachyon open-source project.
Alluxio's data orchestration platform is a meta-layer that sits between storage and compute engines, serving data to large-scale AI and analytics in any cloud across clusters, regions, clouds, and countries, providing simplified data access to files and objects. Features like intelligent caching, unified namespace, and data management provide agility and cost efficiency to customers in financial services, high-tech, retail, and telecommunications.
Alluxio is trusted by Meta, Uber, Tencent, Tiktok, Alibaba, Expedia, Rakuten, Microsoft, Walmart, and more! Please review Wikipedia to learn more about us! Join our world-class team of empathetic, enthusiastic, and creative people who can work on some of the toughest big data problems.
About the Role:
As a Technical Product Manager, you will own the product strategy and roadmap for AI inference capabilities, collaborating with ML engineers and practitioners to deliver low‑latency, high‑throughput data access for LLMs, generative AI, computer vision, and NLP. Your focus is on features that optimize resource utilization and reduce total cost of ownership.
Responsibilities:
- Define product roadmaps for AI‑inference workflows, focusing on latency, throughput, and GPU utilization improvements.
- Engage deeply with technical users to understand bottlenecks in model serving, caching, and scaling and translate them into product specifications.
- Partner with engineering to design and deliver inference‑oriented features such as GPU scheduling, sharding, and streaming data access.
- Work with customers to validate features and measure their impact, incorporating feedback into iteration cycles.
- Stay current with AI infrastructure trends (hybrid clouds, edge inference, multi‑model serving) and integrate them into product planning.
Qualifications:
- 4-8 years of experience in product management, AI infrastructure, or ML engineering roles, with at least 2 years focused on AI/ML workloads.
- Deep understanding of AI/ML workflows, including model deployment, inference optimization, and data‑access patterns.
- Proven track record of delivering features with measurable improvements in latency, throughput, or GPU utilization, or equivalent experience.
- Technical proficiency in distributed systems and cloud platforms (Kubernetes, AWS/GCP/Azure) and familiarity with frameworks like PyTorch, TensorFlow, Triton Inference Server or similar.
- Excellent communication and cross‑functional leadership skills, enabling clear translation of complex technical concepts to varied audiences.
Why Join Alluxio?
- Be part of a world-class team dedicated to solving some of the toughest challenges in big data.
- Work in a dynamic and innovative environment with opportunities for professional growth and development.
- Enjoy a collaborative culture that values empathy, enthusiasm, and creativity.
Alluxio is an equal opportunity employer and does not discriminate in employment on the basis of race, color, religion, sex (including pregnancy and gender identity), national origin, political affiliation, sexual orientation, marital status, disability, genetic information, age, membership in an employee organization, retaliation, parental status, military service, or other non-merit factors.
The salary range for this full-time position in the United States is $190,000 - $240,000 depending on experience and level, subject to standard withholding and applicable taxes. All candidates will also receive equity (stock options) and access to a comprehensive benefits package. The base salary range reflects the minimum and maximum target for candidates across all US locations. The specific compensation awarded will be based on factors such as work location, skills, experience, and relevant education or training. Our Recruiting Team or Hiring Manager will provide more details about the specific salary range during the recruitment process.
Top Skills
AWS
Azure
GCP
Kubernetes
PyTorch
TensorFlow
Triton Inference Server
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Proven at global web scale in production for modern data services, Alluxio is the developer of open source data orchestration software for the cloud. Alluxio moves data closer to big data and machine learning compute frameworks in any cloud across clusters, regions, clouds and countries, providing memory-speed data access to files and objects. Intelligent data tiering and data management deliver consistent high performance to customers in financial services, high tech, retail and telecommunications. Alluxio is in production use today at seven out of the top ten internet companies. Venture-backed by Andreessen Horowitz and Seven Seas Partners, Alluxio was founded at UC Berkeley’s AMPLab by the creators of the Tachyon open source project. For more information, contact [email protected].
