Embedded AI Engineer, On-Device Models

Reposted 3 Months Ago
Hiring Remotely in USA
Remote
219K-274K Annually
Senior level
Artificial Intelligence • Machine Learning • Natural Language Processing • Software • Conversational AI
Deepgram builds research-driven Voice AI to power the human-to-machine interactions of the future.
The Role
Take Deepgram's speech models and optimize, compile, and run them on resource-constrained embedded and edge hardware. Work across model compression, runtime performance (C/C++/Rust), RTOS/bare-metal and embedded Linux environments, vendor NPUs/DSPs and inference runtimes, benchmarking, deployment pipelines, OTA updates, and partner with silicon vendors to enable real-time, low-power on-device voice experiences.
Summary Generated by Built In
Company Overview

Deepgram is bringing the world’s most natural interface, voice, to the technology we use every day. Our speech models and real-time infrastructure power AI systems that keep up with the way people actually speak. More than 5,000 organizations run conversational voice AI on Deepgram in production, including Twilio, Cloudflare, Sierra, Decagon, Vapi, Cresta, Granola, and Jack in the Box.

The Deepgram Voice Agent Platform brings together speech-to-text, text-to-speech and speech-to-speech, with voice-native foundation models available through cloud APIs or self-hosted and on-premises deployments. Built for accuracy, low latency and cost efficiency, Deepgram makes high-quality voice experiences practical to build and run at scale. More than 650,000 developers have used Deepgram to process more than 200,000 years of audio. Every day, more of the world’s conversations with AI run on our models.

Operating Principles

Deepgram is a research company that productizes as fast as possible. Four Operating Principles shape how we think, decide, and work together:

  1. Compress before you communicate. You think it through first, then lead with the bottom line.

  2. Resist entropy. You question what already exists before adding more process or tools.

  3. Solve the category. You fix recurring problems at the source, not one case at a time.

  4. Work at the frontier. You go after problems no one has solved yet.

We look for people who already work this way, and these principles shape how we grow and evaluate our team.

AI-First by Default

We build AI that millions of people talk to, and we run the company the same way. AI is how we centralize work here, and is core to how we operate.

That means more than using the tools. We expect you to:

  • Build, not just use. Design your own workflows and agents, and use AI to remove whole categories of work instead of doing the same work faster.

  • Own the output. AI can speed up the work. It doesn’t make anyone less accountable for it. You understand, verify, and stand behind everything you share, including what an agent produces for you.

  • Spend the time you get back at the frontier. The capacity AI creates goes toward problems no one has solved yet.

  • Keep pace. Models and tools change every month, and so will your work. You adopt what’s new quickly and drop what no longer earns its place.

We judge AI by what it helps you deliver and whether you stand behind the result, not by how often you use it. In interviews, expect to walk us through a workflow you built and where you drew the line between AI and your own judgment.

About the Role

Deepgram's speech models are among the fastest and most accurate in the world, and we have deep machinery for running them on NVIDIA GPUs. Our customers need them on everything else: non-NVIDIA accelerators, embedded SoCs, mobile application processors, DSPs and NPUs, and purpose-built devices with tight memory, compute, thermal, and power budgets. When a target platform's standard kernels and runtime can't run a Deepgram model well enough, someone has to go below them. That is this role.

As an Embedded AI Engineer on the Partner Platform Engineering team, you work at the lowest layer of our edge stack. You write and optimize custom kernels and operators for specific hardware, collapse models onto device-specific execution units, and do the target-side quantization and assembly-level tuning that standard toolchains can't. You hand what you build up to Applied ML Engineers, who fit Deepgram models to your kernels. Your work is what makes a new hardware platform viable for Deepgram at all.

This role is a great fit for a senior embedded engineer who has spent their career close to the metal and wants to point that at speech AI, or a staff-level engineer who wants to define how Deepgram's models get onto new silicon. We'll set the level to your experience.

What You'll Do
  • Write and optimize custom kernels and operators (C, C++, Rust, and platform assembly or intrinsics) for non-NVIDIA accelerators, embedded SoCs, DSPs, and NPUs where the vendor's standard operator set is insufficient for Deepgram models.

  • Own target-side optimization: collapse models onto device execution units through quantization, operator fusion, memory layout, and architecture-specific compilation to meet latency, memory, power, and thermal budgets.

  • Integrate with vendor NPU/DSP toolchains and edge inference runtimes, and extend them with custom operators when the graph doesn't map cleanly.

  • Deliver kernels and runtime components as reusable building blocks that Applied ML Engineers can target when adapting models, with clear interfaces and documented constraints.

  • Build performance-critical runtime code for embedded environments, including embedded Linux, bare-metal, and RTOS targets.

  • Establish per-platform benchmarking and validation for latency, accuracy, power, memory footprint, and utilization, and catch regressions before they ship.

  • Partner with silicon and platform vendors on SDK integration and low-level performance tuning for new chipsets and reference platforms.

  • Feed hardware constraints back to Applied ML and Research so model designs are easier to land on constrained targets.

You'll Love This Role If You
  • Find deep satisfaction in making a large model run on hardware that was never meant to run it, and still hitting accuracy and latency targets.

  • Reach for the profiler and the ISA manual before you reach for a bigger chip.

  • Would rather write the kernel than wait for the vendor to ship it.

  • Care about the details that don't show up in a cloud benchmark: cold start, power draw, thermals, memory fragmentation, cache behavior.

  • Prefer hard, constrained, ship-it problems over open-ended research.

  • Care about the details that don't show up in a cloud benchmark: cold-start time, power draw, thermals, and memory fragmentation.

It's Important To Us That You Have
  • Experience delivering production systems on resource-constrained hardware — embedded systems, mobile, edge AI, or small low-power devices.

  • Strong proficiency in C, C++, and/or Rust, with experience writing performance-critical code for constrained environments.

  • Hands-on experience with model optimization for on-device deployment, including quantization, pruning, knowledge distillation, or architecture-specific compilation.

  • Familiarity with edge inference runtimes (e.g., ONNX Runtime, TensorRT, TFLite, ExecuTorch) and/or vendor-specific NPU/DSP toolchains.

  • A strong understanding of hardware-software interaction — CPU/GPU/NPU/DSP architectures, memory hierarchies, fixed-point/integer arithmetic, and power management — and how they affect inference performance.

  • Experience working close to the metal: bare-metal or RTOS environments (e.g., FreeRTOS, Zephyr), embedded Linux, or microcontroller and edge SoC development.

  • Strong communication skills and a builder mindset — you can scope an ambiguous optimization problem, drive it to a measurable result, and explain the tradeoffs clearly.

It Would Be Great if You Had
  • Experience with real-time audio processing on embedded platforms — DSP pipelines, audio codec optimization, wake-word or always-on listening, or streaming inference on microcontrollers and edge SoCs.

  • Depth in ML optimization techniques — custom quantization schemes, mixed-precision inference, or neural architecture search for edge targets.

  • Background in hardware evaluation and benchmarking — systematically comparing accelerators, SoCs, or GPUs for specific workload profiles.

  • Experience shipping AI features in consumer products at scale, and the instinct for what "production quality" means on a battery-powered device.

  • Familiarity with model compilation and optimization toolchains and their tradeoffs across hardware targets.

  • Experience with secure, robust on-device deployment practices — code signing, encrypted model storage, and safe update mechanisms.

Notice: We're aware of individuals impersonating Deepgram recruiters. All legitimate Deepgram recruiting communication comes from an @deepgram.com email address. If you've received a message claiming to be Deepgram, please forward it to [email protected].

Skills Required

  • Experience delivering production systems on resource-constrained hardware (embedded systems, mobile, edge AI, low-power devices).
  • Strong proficiency in C, C++, and/or Rust with performance-critical embedded code experience.
  • Hands-on experience with model optimization for on-device deployment: quantization, pruning, knowledge distillation, operator fusion, or architecture-specific compilation.
  • Familiarity with edge inference runtimes (ONNX Runtime, TensorRT, TFLite, ExecuTorch) and/or vendor-specific NPU/DSP toolchains.
  • Strong understanding of hardware-software interaction: CPU/GPU/NPU/DSP architectures, memory hierarchies, fixed-point/integer arithmetic, and power management.
  • Experience with bare-metal or RTOS environments (e.g., FreeRTOS, Zephyr), embedded Linux, or microcontroller/edge SoC development.
  • Strong communication skills and ability to scope ambiguous optimization problems and deliver measurable results.
  • Experience with real-time audio processing on embedded platforms (DSP pipelines, audio codecs, wake-word, streaming inference).
  • Depth in ML optimization techniques (custom quantization schemes, mixed-precision inference, neural architecture search for edge targets).
  • Background in hardware evaluation and benchmarking of accelerators, SoCs, or GPUs.
  • Experience shipping AI features in consumer products at scale and understanding production-quality constraints on battery-powered devices.
  • Familiarity with model compilation and optimization toolchains and tradeoffs across hardware targets.
  • Experience with secure on-device deployment practices: code signing, encrypted model storage, safe OTA update mechanisms.

What the Team is Saying

Deepgram Employee
Deepgram Employee
Deepgram Employee
Deepgram Employee
Ralphie
Natalie
Ingrid
Tivo

Deepgram Compensation & Benefits Highlights

  • Healthcare Strength — Health coverage is described as comprehensive, spanning medical, dental, vision, mental health support, and wellness stipends. Employer contributions toward medical premiums are characterized as substantial, reinforcing the perceived value of the health plans.
  • Parental & Family Support — Paid parental leave is portrayed as a standout element, with 16 weeks of fully paid leave cited as a key benefit. This depth of support signals a strong commitment to families during critical life events.
  • Leave & Time Off Breadth — Unlimited PTO, flexible schedules, and paid company holidays are consistently highlighted as core parts of the package. The breadth of time-off options indicates strong institutional support for rest and flexibility.

Deepgram Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: San Francisco, CA
150 Employees
Year Founded: 2015

What We Do

Deepgram is the leading voice AI platform for developers building speech-to-text (STT), text-to-speech (TTS) and full speech-to-speech (STS) offerings. 200,000+ developers build with Deepgram’s voice-native foundational models – accessed through APIs or as self-managed software – due to our unmatched accuracy, latency and pricing. Customers include software companies building voice products, co-sell partners working with large enterprises, and enterprises solving internal voice AI use cases. The company ended 2024 cash-flow positive with 400+ enterprise customers, 3.3x annual usage growth across the past 4 years, over 50,000 years of audio processed and over 1 trillion words transcribed. There is no organization in the world that understands voice better than Deepgram.

Why Work With Us

Our culture, like our product, is constantly learning and evolving, but the heart of our team is enduring. We are a self-motivated, positive, passionate, and competitive group of people. At Deepgram, we put an emphasis on being ourselves, being curious, growing together, and being human. We are a unique bunch who celebrate our differences.

Gallery

Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery

Deepgram Offices

Remote Workspace

Employees work remotely.

Typical time on-site: None
HQSan Francisco, CA
Ann Arbor, MI
Learn more

Similar Jobs

Deepgram Logo Deepgram

Operations Specialist

Artificial Intelligence • Machine Learning • Natural Language Processing • Software • Conversational AI
In-Office or Remote
3 Locations
150 Employees
110K-150K Annually

Deepgram Logo Deepgram

Development Platform Partner Manager

Artificial Intelligence • Machine Learning • Natural Language Processing • Software • Conversational AI
Remote
USA
150 Employees
175K-230K Annually

Deepgram Logo Deepgram

Senior Product Manager

Artificial Intelligence • Machine Learning • Natural Language Processing • Software • Conversational AI
Remote
USA
150 Employees
165K-223K Annually

Deepgram Logo Deepgram

New Media & Social Lead

Artificial Intelligence • Machine Learning • Natural Language Processing • Software • Conversational AI
Remote
USA
150 Employees
175K-215K Annually

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account