LLM Platform Engineer

Posted Yesterday
2 Locations
Hybrid
160K-190K Annually
Mid level
Enterprise Web • Hardware • Information Technology • Productivity • Software
AllSpice helps hardware teams communicate effectively, reduce prototype errors, and accelerate time to market.
The Role
Build and own AI platform tooling, evaluation and observability systems, internal applications, data pipelines, integrations, and RAG repositories. The role focuses on improving LLM quality, reliability, cost, latency, and customer outcomes while supporting engineers and product managers. Responsibilities include architecture, implementation, documentation, stakeholder communication, testing, mentorship, and potential contributions to customer-facing product features.
Summary Generated by Built In

Help define the future of hardware development! At AllSpice, we're building the automation engine that powers the next generation of circuit design, enabling breakthroughs in smart vehicles, robotics, rockets, IoT, medical devices, and more.

We're creating the agile development environment for hardware engineers with a Git-friendly translation layer and automated CI/CD framework for native circuit designs. Think GitHub/GitLab + Copilot for electronics.

Our mission is to bring the same modern workflows that revolutionized software: version control, automation, and collaboration to the hardware world.

Learn more about our journey in TechCrunch and our recent Series A announcement.

If you're passionate about building tools that empower teams and delight customers, and you want to shape the foundation of modern hardware DevOps, we'd love to meet you!

Read more at https://allspice.io

What will you do?

This is a senior, high-impact software engineering role at the intersection of AI platform development, data engineering, and full-stack product development. You will own the tooling, evaluation systems, and data platforms that make AllSpice's Generative AI team more effective. Your primary users will be internal engineers and product managers, with opportunities to build customer-facing experiences where they support our AI products.

We’re looking for someone who can identify the right problems, make thoughtful technical decisions, and take solutions from discovery through implementation, adoption, and ongoing improvement. As you build domain knowledge and establish the AI platform foundation, we’d also like you to grow into contributing directly to AllSpice’s core product alongside the product engineering team. This is an aspirational growth opportunity, not a prerequisite or the role’s initial primary focus.

You will:

  • Partner with engineers and product managers to discover workflow bottlenecks, prioritize the highest-impact problems, and own solutions end-to-end—from requirements and architecture to rollout and iteration.

  • Own the evaluation and observability layer for our AI products, including automated and human review workflows, regression detection, and comparisons across models, prompts, and pipelines. Make quality, cost, latency, usage, and customer outcomes visible and actionable.

  • Build usable internal applications and self-service tools that help engineers and product managers inspect AI results, investigate failures, and provide feedback. Where useful, extend these capabilities into customer-facing experiences.

  • Build and maintain scripts, reliable data pipelines, and integrations that connect internal systems, third-party services, LLMs, and Model Context Protocol (MCP) tools, with attention to data quality, schema evolution, and customer data privacy.

  • Write clear documentation and communicate findings to both technical and non-technical audiences, including leadership and customers.

  • Champion code quality, testing, and reliability across the tooling stack through reviews, mentorship, and thoughtful design.

Example Projects
  • Build an internal evaluation workspace for DRCY, our AI design review product, that captures run configurations, inputs, outputs, and telemetry; compares model and prompt changes; and tracks quality, false positives, cost, and latency over time.

  • Develop review workflows that let engineers and product managers inspect AI-generated findings, apply scoring rubrics, override automated scores, and investigate regressions—with opportunities to surface relevant insights and feedback workflows to customers.

  • Create and maintain data pipelines that aggregate evaluation results, customer feedback, and telemetry into a unified analytics layer.

  • Build scripting tools and automations to streamline the Gen AI team's development and testing workflows (e.g., batch evaluation runs, report generation).

  • Design and implement RAG-based information repositories and tooling to capture component data and domain knowledge for use by our suite of AI products.

  • Integrate with CI/CD systems (GitHub Actions) to surface AI insights directly in engineering workflows.

  • Partner with infrastructure to measure cache effectiveness, usage, and cost, and use those insights to improve the reliability and efficiency of AI workflows.

  • As an aspirational next step, contribute directly to core AllSpice product features—for example, integrating AI review findings and feedback into existing engineering workflows—in partnership with the product engineering team.

Our stack
  • Languages: Python, TypeScript/JavaScript, SQL

  • Analytics & Data: SQL databases (Postgres), Jupyter Notebooks, business intelligence/analytics tools

  • AI/LLM: Langchain, Pydantic AI, Langfuse

  • Web: Vue/TypeScript front-end, common web architectures and APIs

  • Infrastructure: GitHub Actions for CI/CD, Docker Swarm & Terraform, AWS

  • Our Hub application is a soft fork of Gitea

    • Go [server-side]

    • Vue/TypeScript Front-end

Expectations

Our ideal candidate should have the following:

  • 4+ years of software engineering experience with a focus on tooling, data platforms, or full-stack web development, with demonstrated senior-level ownership of systems or products that other people depend on.

  • Strong SQL skills and experience designing data pipelines and storage schemas, handling data quality and schema evolution, and using data/analytics tools (e.g., Jupyter, Metabase, Looker, or similar BI platforms).

  • Proficiency in Python scripting and automation for data pipelines, integrations, and internal tools.

  • Experience building web applications or customer-facing tools using modern frameworks (e.g., Vue, React, TypeScript).

  • Practical experience shipping or maintaining LLM-enabled systems and building infrastructure around them, such as evaluation pipelines, observability, or integrations. You understand non-determinism, quality and reliability challenges, and tradeoffs across cost, latency, and model performance.

  • Excellent communication skills — you can translate technical findings into clear insights for engineers, product managers, and customers alike.

  • A customer-first mindset: you think about the end-user experience and proactively seek feedback to improve the tools you build.

  • Bachelor's degree or higher in a technology-related field, or equivalent practical experience.

  • Project management skills with the ability to lead and collaborate with varied stakeholders to move projects forward.

  • Comfort with ambiguity and autonomy: you independently identify problems, propose solutions, make pragmatic build-vs.-buy and architectural decisions, and bring stakeholders along in a fast-moving environment.

  • (Preferred) Availability to work out of our flex offices in San Francisco or Boston on a hybrid basis (2–3 times/week), but we are flexible for the right candidate.

Relevant Skills

You don't necessarily have to be proficient in all of these, but the more you know, the better:

  • Python, TypeScript, SQL

  • Business analytics and data visualization tools

  • Langchain, Pydantic AI, and LLM evaluation frameworks

  • GitHub Actions development and CI/CD

  • Common web architectures, APIs, and front-end frameworks (Vue, React)

  • Monitoring and observability tools (e.g., Langfuse, DataDog, Grafana)

  • Project management and stakeholder communication

Benefits
  • Supportive and smart colleagues, flexible work, opportunity to make a big impact, competitive salary & equity, health, dental, vision, generous PTO, home office stipend.

Skills Required

  • 4+ years of software engineering experience focused on tooling, data platforms, or full-stack web development
  • Senior-level ownership of systems or products used by others
  • Strong SQL skills
  • Experience designing data pipelines and storage schemas
  • Experience handling data quality and schema evolution
  • Experience using data or business analytics tools
  • Proficiency in Python scripting and automation
  • Experience building web applications or customer-facing tools with modern frameworks such as Vue, React, or TypeScript
  • Practical experience shipping or maintaining LLM-enabled systems
  • Experience building LLM infrastructure such as evaluation pipelines, observability, or integrations
  • Understanding of LLM nondeterminism, quality, reliability, cost, latency, and model-performance tradeoffs
  • Excellent communication skills with technical and non-technical audiences
  • Customer-first mindset and willingness to seek user feedback
  • Bachelor's degree or higher in a technology-related field, or equivalent practical experience
  • Project management skills and ability to collaborate with varied stakeholders
  • Comfort with ambiguity, autonomy, and architectural decision-making
  • Availability to work from flex offices in San Francisco or Boston on a hybrid basis two to three times per week
  • Experience with business analytics and data visualization tools
  • Experience with LangChain, Pydantic AI, or LLM evaluation frameworks
  • Experience developing GitHub Actions and CI/CD systems
  • Experience with common web architectures, APIs, Vue, or React
  • Experience with monitoring and observability tools such as Langfuse, DataDog, or Grafana
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Lahore
10 Employees
Year Founded: 2020

What We Do

What if we could design hardware with the ease and speed of software? AllSpice is building the first ever developer-led platform for collaborating on and seamlessly validating hardware designs. The world around us is enabled by novel circuit designs. Consider the seemingly trivial act of hailing a taxi from the comfort of your home. This simple process (for you) relies on a 3 million light-emitting diode display with a user interface that is logically routed to a capacitive sensor capable of tracking your finger at sub-millimeter resolution, an antenna capable of transmitting to a radio tower kilometers away at a rate of 20 Gb/s, repeated 35,000 km to a satellite, which is able to use it’s own integrated sensors and drivers to maintain geosynchronous orbit over decades without human interference. The infrastructure we've taken for granted wasn't designed overnight and isn't staying put. It's constantly evolving to enable the next generation of life-changing products. But this evolution is currently hamstrung by incumbent and proprietary development infrastructure built for waterfall project management (mostly in the 90s), requiring manual pdf exports, emails, and in-person meetings at each design revision. The next generate of human innovation, like putting the first human colonies on Mars, requires a step-change in how hardware designs are managed. At AllSpice, we've experienced this problem as hardware engineers and we've seen the solutions as software leaders. Today, AllSpice Hub is allowing electrical engineering teams to dramatically accelerate their development by enabling a truly agile workflow. They can seamlessly push a new design update in git, open a design review, tag stakeholders, notify them by email and slack, all while AllSpice automatically collects review artifacts, like visual diffs and review checklists. See it live at https://hub.allspice.io/AllSpice/Archimajor What now? This is just the tip of the iceberg. Our users are adding collaborators from firmware, software, mechanical engineering, management, and logistics begging for even more ways connect their design data to revolutionize their workflow and connect their teams.

Why Work With Us

We encourage exploration and growth in all areas of our work. We never put a pin in something we don't understand. As a team, we set priorities more often than processes. We enjoy the freedom from overbearing bureaucracy that comes with being part of a small team. We're a remote-first team, and believe amazing progress can happen from anywhere.

Gallery

Gallery

Similar Jobs

Tapestry - Coach and Kate Spade Logo Tapestry - Coach and Kate Spade

Temporary Sales Support Associate

eCommerce • Fashion • Retail • Sales • Wearables • Design
Hybrid
Adams Shore, Braintree, MA, USA
16000 Employees
15-22 Hourly

Zscaler Logo Zscaler

Architect

Cloud • Information Technology • Security • Software • Cybersecurity
Easy Apply
Remote or Hybrid
USA
8697 Employees
165K-235K Annually

Liberty Mutual Insurance Logo Liberty Mutual Insurance

Senior Technical Claims Specialist, Construction Complex

Artificial Intelligence • Fintech • Insurance • Marketing Tech • Software • Analytics
Remote or Hybrid
4 Locations
40000 Employees
83K-157K Annually

PwC Logo PwC

Deals Delivering Deal Value Human Capital Diligence Senior Manager

Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Hybrid
14 Locations
370000 Employees
124K-280K Annually

Similar Companies Hiring

Revel Thumbnail
Aerospace • Hardware • Robotics • Software
Marina Del Rey, California
70 Employees
Unusual Machines, Inc. Thumbnail
Hardware • Robotics
Orlando, FL
190 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
30 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account