ML Performance Engineer

You will:

• Own a significant piece of our ML platform while rapidly building and iterating scalable, robust distributed infrastructure for ML training, inference, and evaluation on large-scale time-series and weather datasets.
• Optimize throughput and cost by supporting model training and deployment across multiple clusters and clouds.
• Improve the efficiency of machine learning models and other workloads by optimizing latency, throughput, and memory consumption. This involves pushing the boundaries of current hardware capabilities through techniques like GPU performance engineering.
• Help define the long-term vision for Gridmatic’s ML platform.
• Play a key role in mentoring junior engineers and interns, contributing to a collaborative, innovative, and growth-oriented team culture.

You must be:

• A strong engineer with 3+ years of full-time industry experience working on ML systems.. You possess a deep understanding of the codebases you work in and write readable, scalable code.
• Experienced in optimizing GPU throughput in deep learning models.
• Experienced in distributed training and inference of large models on GPU clusters, utilizing core libraries and frameworks such as PyTorch, PyTorch Lightning, and Ray.
• A self-starter with a strong sense of independence and ownership, and the capability to engineer large, robust systems from the initial design and conceptualization to productionization.
• Hold a Masters or Doctorate degree in engineering or a related technical field.
• A mission-driven individual who is enthusiastic about working toward a renewable grid and diving into the intersection of ML and energy. No prior energy experience required, but curiosity and a willingness to learn are must-haves!

Nice to haves:

• End to end proficiency in building, maintaining, and debugging cluster infrastructure, utilizing Kubernetes and Terraform.
• Expertise in identifying performance bottlenecks and designing and writing high-performance code for large-scale ML workloads.
• Experience with at least one of: torch.profiler, TorchDynamo, TorchInductor, Triton, or other deep learning compiler stacks.
• Understanding of GPU architectures or experience with GPU kernel programming.
• Knowledge of cluster communication protocols such as nccl or gloo.
• Experience working with any of the following: weather data, energy systems, time-series forecasting, electricity markets, or financial trading.