Position:
- Model Training & Optimization:
- Design, fine-tune, and optimize transformer-based models with a focus on quantization, distillation, pruning, and other compression techniques. Select and justify approaches based on deployment goals, model constraints, and resource availability.
- Advise on architectural tradeoffs and deploy models across varied environments (cloud, on-prem, edge).
- Profile models and optimize performance across different hardware (e.g., consumer-grade GPUs, low-end data center cards). Use and interpret CUDA-level metrics to inform optimizations.
- Evaluation Frameworks:
- Develop and maintain rigorous model evaluation pipelines including both standardized benchmarks (e.g., MMLU, SuperGLUE) and custom task-specific tests. Define and monitor performance trade-offs such as accuracy vs latency or cost vs throughput. Design input evaluation strategies (e.g., few-shot vs zero-shot, prompt engineering, sequence length variations).
- Collaborative Dataset Engineering:
- Work with domain experts, data engineers, and curators to source, label, clean, and structure high-quality datasets.
- Evaluate data quality issues and create tooling for dataset diagnostics.
- Research and Prototyping:
- Stay current with advancements in model compression, efficient inference, and deployment strategies.
- Rapidly prototype and test new ideas, bringing practical innovations into the team’s workflow.
- Documentation & Communication:
- Clearly document experiments, design decisions, and trade-off analyses. Share findings with both technical and non-technical stakeholders, contributing to engineering design and product planning.
- PhD OR Master's Degree plus 3+ years of progressive experience
- Strong understanding of transformer-based architectures
- Experience with model optimization: quantization, pruning, distillation, or low-rank adaptation
- Familiarity with deployment trade-offs: latency, memory, throughput, model size vs accuracy
- Ability to reason about and debug performance issues across compute environments (cloud vs on-prem, various GPU types)
- Familiar with CUDA basics – enough to analyze compute requirements, understand bottlenecks, and suggest improvements
- Hands-on experience with fine-tuning language models on real-world datasets
- Proficiency with PyTorch
- Experience with Linux, SSH, scripting, and working on remote machines
- Strong written and verbal communication skills, including documentation of experiments and design rationale
- Experience designing evaluation protocols beyond standard metrics (e.g., human-in-the-loop evaluation, complexity-based slicing)
- Experience with automated benchmarking and robustness testing.
- Nice to haves: experience with APIs (e.g., Django, Flask, FastAPI)
Similar Jobs
What We Do
RegScale overcomes speed, timeliness, and cost effectiveness limitations in legacy GRC by bridging security, risk, and compliance through our Continuous Controls Monitoring platform.
Our CCM pipeline of automation, dashboards, and AI tools deliver lower program costs, strengthen security, and minimize painful handoffs between teams. Achieve rapid certification for faster market entry, anticipate threats via proactive risk management, and automate evidence collection, access reviews, and controls mapping. Improve the Return on
Investment (ROI) of existing tools by seamlessly exchanging data with our centralized CCM data lake, enabling continuous monitoring of security, risk, and compliance controls. Heavily regulated organizations, including Fortune 500 enterprises – both financial institutions and other sectors – as well as the government and entities that serve them, use RegScale to enhance stakeholder trust, lower costs, adapt to evolving risks, and start and stay compliant. Our customers report a 90% faster path to compliance certifications and a 60% reduction in audit preparation efforts, strengthening security programs and reducing costs. For more information, visit www.regscale.com
