AI Engineer

Posted 3 Days Ago
Hiring Remotely in San Francisco, CA, USA
In-Office or Remote
Mid level
Artificial Intelligence • Machine Learning • Consulting • Big Data Analytics
The Role
Deploys and operates production LLM inference services across Kubernetes and GPU clusters. Builds AI gateways, routing layers, RAG pipelines, agent workflows, APIs, automation, observability, benchmarks, and capacity recommendations. Integrates services with customer systems, documents deployments, and supports customer engagements, pre-sales, and platform development. The role is remote, customer-embedded, and includes occasional U.S. travel.
Summary Generated by Built In

Insight Softmax is a leading organization in the fields of machine learning, data science, and high-performance computing (HPC). We are dedicated to providing innovative solutions in computing technology to solve complex problems in various sectors including research, science, manufacturing, finance, and data analytics.

Description:

  • We are hiring an AI Engineer for our engineering team. You will implement the AI platforms and services our architects design and our customers run: inference stacks, AI gateways, RAG and agent systems, and the automation around them.
  • You will work directly inside customer environments, from neoclouds with thousands of GPUs to enterprises deploying their first production LLM service, and on our own platform product work for vendor partners.
  • The role is forward-deployed in spirit. You will spend much of your time embedded with a customer team, shipping working systems and leaving behind documentation and automation they can run.
  • Our team requires individuals with focus on delivery and results, hunger to learn, ability to adapt, and can thrive in a self-directed environment where you’ll figure out what’s needed to deliver rather than wait for detailed instructions.

Key Responsibilities:

  • Deploy and operate LLM inference services on Kubernetes and bare metal GPU clusters, including model serving, autoscaling, GPU scheduling and performance tuning.
  • Build and operate AI gateway and routing layers: tenant keys, model pools, quotas, rate limits, caching, failover and usage metrics.
  • Implement RAG pipelines, agentic workflows, evaluation harnesses and the APIs that expose them to customer applications.
  • Write production-quality services and automation in Python, with infrastructure as code for the surrounding platform.
  • Instrument AI services for observability: latency, throughput, token and cost accounting, quality signals.
  • Run benchmarks and load tests, and turn results into tuning changes and capacity recommendations.
  • Integrate AI services with customer identity, security and data systems.
  • Document deployments and hand over runbooks to customer and internal operations teams.
  • Reserve time each week for pre-sales support, internal platform work and knowledge sharing.

Qualifications:

  • At least 4 years of software or platform engineering experience, with at least 2 years building and running LLM or ML services in production.
  • Hands-on with at least one inference server (for example vLLM, TensorRT-LLM, Triton, llm-d) and with running GPU workloads on Kubernetes.
  • Strong Python. Comfortable building services, CLIs and automation, with tests.
  • Working knowledge of Kubernetes, containers, Helm and GitOps workflows.
  • Strong linux chops.
  • Deep knowledge of at least one of AWS, GCP, or Azure. Preferably AWS.
  • Experience with LLM application patterns: prompt and context management, retrieval, tool use, structured output, evaluation.
  • Able to work independently inside a customer environment and communicate progress clearly.

Optional Experience and Skills:

  • AI gateway tooling (for example LiteLLM, Envoy or Traefik based gateways, Kubernetes
    Gateway API inference extensions).
  • Vector databases and embedding pipelines.
  • Fine-tuning, quantization and model optimization.
  • Multi-tenant GPU platforms, GPU partitioning and Dynamic Resource Allocation.
  • Observability stacks (OpenTelemetry, Prometheus, Grafana, Datadog) applied to AI services.
  • Spectro Cloud Palette and PaletteAI, Run:ai, or similar GPU platform products.
  • Slurm or other batch schedulers.

Bonus Experience and Skills:

  • Go or Rust.
  • Terraform and Ansible.
  • Air-gapped or regulated environment deployments.
  • Using linux as your primary desktop workstation environment.

Attributes that we value:

  • Transparency
  • Grit
  • Effectiveness
  • Willingness to learn

Location and Hours:

  • Full time position
  • Remote work environment
  • 80% of your work schedule will be during the business time zones for Latin America and Canada.
  • Some customer meetings will occur each week at non-standard / nighttime hours, as we
    currently support a global team across 3-4 continents.
  • Some customer engagements include on-site work at US customer sites, typically for
    workshops, platform bring-up and go-live. Expect occasional US travel.

References:

  • 3 references will be required.

Notes:

  • AI systems may be used with the recruitment and screening processes for this job requisition.

Skills Required

  • At least 4 years of software or platform engineering experience
  • At least 2 years building and running LLM or ML services in production
  • Hands-on experience with at least one inference server, such as vLLM, TensorRT-LLM, Triton, or llm-d
  • Experience running GPU workloads on Kubernetes
  • Strong Python skills for production services, CLIs, automation, and testing
  • Working knowledge of Kubernetes, containers, Helm, and GitOps workflows
  • Strong Linux skills
  • Deep knowledge of at least one major cloud platform: AWS, GCP, or Azure
  • Experience with LLM application patterns including prompt and context management, retrieval, tool use, structured output, and evaluation
  • Ability to work independently in customer environments and communicate progress clearly
  • Experience with AI gateway tooling, vector databases, embedding pipelines, fine-tuning, quantization, or model optimization
  • Experience with multi-tenant GPU platforms, GPU partitioning, or Dynamic Resource Allocation
  • Experience applying observability stacks to AI services
  • Experience with Spectro Cloud Palette, PaletteAI, Run:ai, or similar GPU platform products
  • Experience with Slurm or other batch schedulers
  • Experience with Go or Rust
  • Experience with Terraform and Ansible
  • Experience deploying in air-gapped or regulated environments
  • Three professional references
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: San Francisco, CA
20 Employees
Year Founded: 2016

What We Do

Insight Softmax Consulting helps organizations solve business problems through data science, software engineering, and high-performance computing. Its services include cloud-based data analytics, business intelligence, data engineering, machine learning, and generative AI, delivered through advisory, implementation, and managed-service engagements. The company also builds and optimizes HPC infrastructure and scientific-computing solutions, helping clients turn data and advanced computing into practical improvements in decision-making, efficiency, and business outcomes.

Similar Jobs

Samsara Logo Samsara

Senior Software Engineer

Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Easy Apply
Remote or Hybrid
United States
4000 Employees
155K-260K Annually

Samsara Logo Samsara

Artificial Intelligence Engineer

Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Easy Apply
Remote or Hybrid
United States
4000 Employees
162K-290K Annually

Toast Logo Toast

Machine Learning Engineer

Cloud • Fintech • Food • Information Technology • Software • Hospitality
Remote
US
5000 Employees
193K-309K Annually

Toast Logo Toast

Machine Learning Engineer

Cloud • Fintech • Food • Information Technology • Software • Hospitality
Remote
US
5000 Employees
230K-368K Annually

Similar Companies Hiring

Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
33 Employees
Vega Thumbnail
Artificial Intelligence • Automotive • Insurance • Transportation
US
65 Employees
RapidScale Thumbnail
Cloud • Information Technology • Consulting
Raleigh, NC
600 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account