Be an Early Applicant
The Role
Leads development of agentic AI capabilities that execute multi-step workflows across desktop engineering tools. Owns agent architecture, evaluation frameworks, benchmarking, cost efficiency, user-story validation, and production delivery. Serves as player-coach for a small AI engineering team, writing code, reviewing designs, and setting technical direction. Requires expertise in LLM application architecture, real-world agents, Python, desktop automation, engineering software domains, and enterprise deployment constraints.
Summary Generated by Built In
Senior Technical Leader — Agentic AI (Engineering Tools)
San Francisco, CA, on-site · Full-time · No sponsorship
The build: Our client is hiring a senior
technical leader to own the core agent intelligence that turns mechanical
engineers' intent into reliable, cost-efficient multi-step workflows across
desktop engineering tools. This role sits at the intersection of applied agentic
AI, user research, and product delivery — and will determine the product's
real-world value to enterprise customers.
What you'll do
- Lead
development of the core agent intelligence layer that executes multi-step
workflows across complex desktop engineering software.
- Report
to the CTO and serve as technical lead for a small team of AI engineers, a
user researcher, and domain expert contractors.
- Own
the full product loop: define agent capabilities from user stories, build
implementations, and benchmark against real workflows.
- Drive
agent task success rate by defining evaluation frameworks, establishing
baselines, and iterating to improve completion metrics.
- Set
and enforce per-task token budgets and track cost per completed workflow
to ensure commercial viability.
- Build
rigorous, reproducible evaluation infrastructure grounded in validated
user stories.
- Lead
user story mapping and validation through interviews and close
collaboration with domain experts.
- Translate
validated user stories into testable evals and close the loop between
research and benchmarking.
- Own
agent architecture decisions including tool-calling, state management,
error recovery, model routing, and context management.
- Act as
a player-coach: write production code, review designs, unblock the team,
raise engineering standards.
- Collaborate
cross-functionally with integrations, product, and customers during POCs.
Requirements
What we're looking for
- 7+
years software engineering, including at least 2 years building agentic
LLM-based agents that act in the real world.
- Deep
experience designing LLM application architectures: model selection,
context/window management, retrieval, orchestration patterns.
- Proven
ability to build evaluation and benchmarking frameworks measuring task
completion, cost efficiency, and failure modes.
- Technical
leadership experience setting direction for small teams (3–6 engineers)
and performing meaningful code review.
- Strong
Python skills and familiarity with LLM tooling (function calling, tool
APIs, observability/tracing, evaluation frameworks).
- Experience
with desktop automation or programmatic control of applications (COM or
similar).
- Domain
experience in mechanical engineering, CAD/CAE, PLM, or adjacent
industries.
- Understanding
of enterprise deployment constraints on locked-down corporate
workstations.
- Track
record contributing to public benchmarks, publications, or open-source
agentic AI projects.
Logistics: San Francisco, on-site.
No sponsorship. Relocation not
available.
Skills Required
- 7+ years of software engineering experience
- At least 2 years building agentic LLM-based agents that act in the real world
- Deep experience designing LLM application architectures, including model selection, context management, retrieval, and orchestration
- Experience building evaluation and benchmarking frameworks for task completion, cost efficiency, and failure modes
- Technical leadership experience directing small teams of 3–6 engineers
- Meaningful code review experience
- Strong Python skills
- Familiarity with LLM tooling, including function calling, tool APIs, observability, tracing, and evaluation frameworks
- Experience with desktop automation or programmatic application control, such as COM
- Domain experience in mechanical engineering, CAD, CAE, PLM, or adjacent industries
- Understanding of enterprise deployment constraints on locked-down corporate workstations
- Track record contributing to public benchmarks, publications, or open-source agentic AI projects
- Ability to work on-site in San Francisco, California
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Fruition Executive Search is a boutique executive search firm based in San Francisco. They specialize in sourcing high-caliber leadership talent across various industries, including Accounting, Finance, AI, Software, and Healthcare. Utilizing a dual approach of retained and contingent search, they connect organizations with C-suite and management professionals equipped to drive strategic direction, profitability, and digital transformation through human-centered leadership.
