The Role
Lead high-impact applied ML research for construction-focused vision-language models. Responsibilities include architecting and training VLMs, developing post-training methods, building video and temporal-reasoning benchmarks, creating scalable training and evaluation pipelines, and optimizing inference through distillation, quantization, and model routing. The role addresses long-video understanding, expert-level perception, efficient large-scale training, and deployment across thousands of hours of daily footage.
Summary Generated by Built In
About Ironsite
The Role
Open Problems You Could Own in Your First Year
What You'll Do
Technical Challenges You'll Solve
What We're Looking For
Why You'll Love Working at Ironsite
Location & Compensation
Ironsite is building the intelligence layer for the physical world. We design our own wearable hardware, deploy it alongside craft workers, and transform a shift's footage into a next-morning report. Our internal team and purpose-built models label the data overnight and deliver actionable insights to superintendents by 5 AM.
We are accelerating the speed, efficiency, and predictability of construction, especially for complex, mission-critical infrastructure projects, including data centers, LNG facilities, sports stadiums, hospitals, and other large-scale developments, by training AI models on egocentric construction footage and labor productivity data. We are built with a pro-worker philosophy at our core: we believe technology should empower the workforce, not replace it. We're working to give craft workers and project leaders better visibility into what's happening on-site, while creating a system where the reality of construction and the chaos of each day is finally available to the people running the project.
Ironsite is deployed across several of the largest active construction projects in the country. To date, we've captured more than 100,000 hours of construction footage across seven states, now process thousands of hours of site activity every day, and maintain a worker opt-out rate below two percent. This is enabled by a workforce-first architecture that anonymizes devices, captures no audio, and never releases raw video.
Ironsite is backed by leading investors (8VC, South Park Commons, Saga Ventures) and prominent operators across technology and construction, including Eric Schmidt, Jeff Dean, Jeff Rothschild, Mark Leslie, Scott Wu, Eric Glyman, Karim Atiyeh, Russell Kaplan, and others, alongside over a dozen construction industry operators who have joined us as partners in building this.
Longer term, we believe Ironsite is the foundation for what construction becomes in the next decade. We think the systems we're building are the operating system for how the physical world gets built, and will unlock a fundamentally different way of respect for our workforce. One where craft workers are more valued, more visible, and better paid for the skill they bring, and where the industry finally has the intelligence layer that makes autonomous construction possible. Both futures start with the same foundation.
We're hiring an exceptional Staff Applied ML Researcher to help build the vision-language models that turn Ironsite's data into the intelligence layer we're building.
You'll work directly with our Chief Science Officer on the research problems that decide how good Ironsite's models become. Training, benchmarking, and deploying state-of-the-art VLMs that can interpret the complexity of a real construction site, built on data that no other lab in the world has access to. This is a role for someone who wants to do frontier research on frontier data, and see their work ship to jobsites where it actually changes how things get built.
Ironsite operates one of the most distinctive research environments in AI today. Our dataset is proprietary, growing by thousands of hours per day, expert-labeled, and structured around a taxonomy built for a specific real-world domain. Our compute footprint spans edge, on-prem, and cloud. And our models don't just ship to a benchmark, they ship to production, running on active jobsites within days of training. Very few research seats in the world offer that combination.
- Long-video understanding. Reasoning over multi-hour egocentric footage where the events that matter are sparse. This requires temporal grounding, long-context modeling, and memory beyond what current VLMs offer.
- Post-training for expert-level perception. Using SFT and RL (e.g., GRPO) to push VLM labeling of fine-grained construction activity to parity with expert human taggers, across every trade.
- Inference at fleet scale. We will soon collect over 10,000 hours of new video per day, expanding quickly from there. Making frontier-quality inference cheap enough to run on all of it, through distillation, quantization, and model routing, is partially unsolved.
- Evals that predict reality. Building benchmarks that track real field performance, not just standard academic metrics.
Train and iterate on Ironsite's core models.
- Design, train, and iterate on vision-language models fine-tuned for spatial intelligence in construction environments. Model quality is the single most important output of this role.
- Own the model training and evaluation pipelines end to end, so we can rapidly experiment, measure performance, and deploy models into production.
- Push the frontier of what's possible on our data, from establishing baselines with state-of-the-art models to developing novel post-training recipes, long-context architectures, and visual reasoning techniques.
Own the Construction Intelligence Benchmark.
- Build and expand our benchmark suite across video question answering, temporal reasoning, activity recognition, and site-level analytical reasoning.
- Design evaluation metrics that measure real-world construction task performance, not just standard academic benchmarks.
- Make our benchmark suite the reference point that future construction AI research measures against.
Ship models to production at fleet scale.
- Apply distillation, quantization, and model routing so state-of-the-art understanding runs affordably across thousands of hours of daily footage.
- Partner with our hardware and infrastructure teams on system design so research decisions and deployment realities inform each other.
- Own the end-to-end story from research paper to production model running on active jobsites.
Shape the research roadmap.
- Work directly with the Chief Science Officer to define the research priorities that matter most for Ironsite's next twelve months and next three years.
- Contribute to the intellectual culture of the team through paper reading, technical mentorship, and setting the bar for how we do research at Ironsite.
- Training large-scale models efficiently under real compute budgets while maximizing performance on the problems that matter for our customers.
- Developing novel pre-training and post-training objectives that capture construction-specific knowledge, temporal reasoning, and fine-grained perception.
- Implementing efficient attention mechanisms and architectural innovations for long-context understanding of construction workflows that span hours or days.
- Designing evaluation metrics that measure real-world construction task performance beyond standard benchmarks, and holding the whole team accountable to what those metrics actually predict.
- Balancing model capability with deployment constraints for edge, on-prem, and cloud inference across the fleet.
What We're Looking For
Required
- 5+ years of hands-on experience designing and training large-scale deep learning models, particularly transformer-based architectures, with at least 2 spent operating at a senior research level at a company or lab you're proud of.
- Deep expertise with modern deep learning frameworks (PyTorch, JAX, or similar) and strong proficiency in Python with solid software engineering fundamentals.
- Experience working with and creating large-scale vision or language datasets.
- Comfort operating at the edge of what's known. You've worked on research problems where the right answer wasn't in a paper yet, and you've figured it out anyway.
- A background in Computer Science, Machine Learning, AI, Robotics, or a related field.
Strongly preferred
- Deep expertise in fine-tuning and post-training large language or vision-language models (SFT, GRPO and other RL methods, parameter-efficient tuning such as LoRA).
- Hands-on experience with the challenges of video data, including temporal reasoning, long-context modeling, and efficient processing.
- Experience optimizing inference at scale, including quantization, distillation, sparsity, and efficient serving.
- Track record of publications in top-tier AI, ML, or CV conferences.
- Foundational impact. Solve fundamental AI problems to transform one of the world's largest and least-digitized industries. Your models ship to real jobsites, not just papers.
- Ownership and autonomy. We're a fast-paced startup where you'll have significant ownership over core research directions. We value intellectual curiosity, first-principles thinking, and iterating quickly to turn ambitious ideas into reality.
- Dream dataset. Exclusive access to a massive, proprietary, and continuously growing corpus of egocentric jobsite video from hundreds of devices deployed on active construction sites. A moat that enables frontier research.
- World-class team. Collaborate with a small, elite team of researchers and engineers who have shipped cutting-edge AI products at companies like DeepMind, Etched, Meta, Apple, and NVIDIA.
- San Francisco Bay Area (on-site)
- Base salary: $200k-$400k per year, commensurate with experience
- Significant early-stage equity
- Full benefits including health, dental, vision, and 401(k) with 6% match
- Access to dedicated GPU compute resources for research and experimentation
- Daily catered breakfast and lunch
- Office in San Francisco, next to Oracle Park and the Caltrain
The base pay range for this role is $200,000 – $400,000 per year.
Skills Required
- 6+ years of hands-on experience designing and training large-scale deep learning models, particularly transformer-based architectures
- Background in Computer Science, Machine Learning, AI, Robotics, or a related field
- Experience with major deep learning frameworks such as PyTorch or JAX
- Strong proficiency in Python and solid software engineering fundamentals
- Experience working with and creating large-scale vision and/or language datasets
- Publications in top-tier AI, machine learning, or computer vision conferences
- Expertise in fine-tuning and post-training large language or vision-language models, including SFT, GRPO, reinforcement learning, and LoRA
- Experience with video data, temporal reasoning, long-context modeling, and efficient processing
- Experience optimizing inference through quantization, distillation, sparsity, or efficient serving
- Familiarity with MLOps tools for scalable model training and deployment
- Strong interest in vision-language models and applying AI to real-world physical problems
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Ironsite AI is a construction technology company that leverages wearable cameras and AI vision models to drive on-site productivity, safety, and training. By equipping workers with smart hard hats, the platform captures real-time data to analyze field activities, optimize labor allocation, and identify safety risks. Their mission is to modernize construction management by providing data-driven insights that help contractors reduce labor costs and deliver projects more efficiently.
