The Role
Design, train, and deploy vision-language models for construction intelligence using large-scale first-person video. Build training/evaluation pipelines, optimize models for edge and server inference, define benchmarks (video QA, temporal reasoning, activity recognition), and collaborate with hardware and data teams to productionize models.
Summary Generated by Built In
About Ironsite
The Role
What You'll Build
Technical Challenges You'll Solve
What We're Looking For
Preferred Qualifications
Location & Compensation
Compensation
Construction is one of the most complex and labor-intensive industries, spending $7 trillion annually on labor, but productivity losses cost $1.6 trillion per year due to outdated management tools.
Ironsite leverages wearable cameras combined with human labeling and AI vision language models to drive on-site productivity, safety & training for crafts workers. We put cameras on construction workers' hard hats and vests, then use advanced computer vision to analyze what's actually happening on job sites.
We help teams reduce labor costs, improve safety, and deliver projects faster. To date, we’ve captured 50,000+ hours of construction footage across 7 states and have recently partnered with the nation’s #2 hard-hat manufacturer, Studson, to develop custom hardware purpose-built for the field.
We’ve raised over $13M from 8VC, South Park Commons, and 30+ leading operators and technologists, including Eric Glyman (Ramp), Jeff Dean (Google), and Scott Wu (Cognition). Now, we’re building the team to scale nationwide.
As an Applied ML Researcher on our team, you will report directly to the Chief Science Officer and be at the heart of our mission to build foundational spatial intelligence. You will tackle high-risk, high-impact research, leveraging our unparalleled proprietary dataset to train, benchmark, and deploy state-of-the-art VLMs that can interpret the complexity of a real-world construction site.
- Architect & Train Novel VLMs: Design, train, and iterate on general-purpose Vision-Language Models fine-tuned for "Construction Intelligence" using our massive, proprietary dataset of first-person video.
- Drive the Research Roadmap: Take a leading role in executing our research goals, including establishing baselines with state-of-the-art models and developing novel fine-tuning methodologies, long context architectures, and visual reasoning techniques.
- Build Scalable Pipelines: Develop and own the model training and evaluation pipelines, ensuring we can rapidly experiment, measure performance, and deploy models into production.
- Optimize for the Edge: Design and implement a hierarchical set of models for efficient, on-device inference on our wearable hardware as well as server-side inference. This includes developing lightweight, coarse classifiers for real-time analysis (e.g., safety event detection, info density classification), as well as heavy-weight server-side VLMs to deeply understand complex tasks.
- Define the Future of Spatial Intelligence: Spearhead the development and expansion of our "Construction Intelligence" benchmark, a comprehensive suite of tasks including video-question answering, temporal reasoning, activity recognition, and higher level data analysis reasoning across the construction site that will guide our research.
- Collaborate on System Design: Work closely with the hardware and data teams to explore model architectures such as a two-model system (lightweight segmenter, heavyweight insight extractor) and tool-use agents, to improve spatial understanding.
- Training large-scale models efficiently with limited compute budgets while maximizing performance
- Developing novel pre-training objectives that capture construction-specific knowledge and temporal reasoning
- Implementing efficient attention mechanisms and architectural innovations for long-context understanding of construction projects
- Designing evaluation metrics that measure real-world construction task performance beyond standard benchmarks
- Balancing model capability with deployment constraints for edge and mobile applications
What We're Looking For
Technical Excellence
- Fast learner eager to grow and expand their knowledge and capabilities, welcoming new challenges with a passion for their work
- Background in Computer Science, Machine Learning, AI, Robotics, Data Science or a related field with multiple relevant classes completed.
- Prior experience, internship, or personal project with hands-on experience designing and training deep learning models, particularly transformer-based architectures.
- Familiarity with one deep learning framework (e.g., PyTorch, TensorFlow, JAX).
- Proficiency in Python
Preferred Qualifications
- At least one publication in an AI/ML/CV conference.
- Experience doing research as part of a larger research lab or team
- Demonstrated experience with major deep learning frameworks (e.g., PyTorch, TensorFlow, JAX).
- Strong proficiency in Python and a solid foundation in software engineering principles.
- Experience working with and creating large-scale vision and/or language datasets.
- A strong interest in vision language models and the application of AI to solve real-world physical problems, including working with and understanding the day-to-day lives of construction workers.
Location & Compensation
- San Francisco Bay Area (on-site)
- Competitive salary and significant equity package
- Full benefits including health, dental, vision, and 401k +6% match
- Access to dedicated GPU compute resources for research and experimentation
The base pay range for this role is $180,000 – $350,000 per year.
Skills Required
- Background in Computer Science, Machine Learning, AI, Robotics, Data Science or related field with multiple relevant classes completed.
- Hands-on experience designing and training deep learning models, particularly transformer-based architectures (projects, internships, or prior work).
- Familiarity with at least one deep learning framework (PyTorch, TensorFlow, or JAX).
- Proficiency in Python.
- Willingness to work on-site in the San Francisco Bay Area.
- Fast learner with strong interest in applied research and real-world problem solving.
- At least one publication in an AI/ML/CV conference.
- Experience doing research as part of a larger research lab or team.
- Demonstrated experience with major deep learning frameworks (PyTorch, TensorFlow, JAX).
- Experience working with and creating large-scale vision and/or language datasets.
- Strong foundation in software engineering principles.
- Strong interest in vision-language models and applying AI to physical-world problems (construction domain).
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Ironsite AI is a construction technology company that leverages wearable cameras and AI vision models to drive on-site productivity, safety, and training. By equipping workers with smart hard hats, the platform captures real-time data to analyze field activities, optimize labor allocation, and identify safety risks. Their mission is to modernize construction management by providing data-driven insights that help contractors reduce labor costs and deliver projects more efficiently.
.png)
