Applied AI Engineer

Posted 7 Days Ago
Hiring Remotely in CAN
Remote
Entry level
Agency
The Role
Designs, builds, tests, and maintains AI conversations and agentic workflows for learner-facing products. Responsibilities include prompt and context engineering, structured outputs, RAG, tool integrations, state management, evaluation, observability, security, guardrails, and compliance controls. The engineer collaborates with Product, Engineering, Content Studios, and subject-matter experts to prototype, validate, deploy, and monitor reliable AI behavior.
Summary Generated by Built In

Teaching AI to say yes is easy. Teaching it when to say ‘no,’ ‘I don’t know,’ or ‘let me get a human’ is the actual job. Our client is hiring an Applied AI Engineer to build the second kind.



The Applied AI Engineer designs, builds and tests the AI conversations and agents inside the client’s learner-facing products: advisory experiences, practice simulations and diagnostic coaching.

They work with Product, Content Studios, Engineering and subject-matter experts. They turn product and instructional requirements into reliable AI behavior, prove it through testing, and keep it performing after release.

This company is at the forefront of learning technologies, providing e-learning programs for professionals in regulated industries such as healthcare and finance. They’re constantly evolving and growing, run efficient Agile-first teams, and have a broadly distributed staff. They value good-natured, inquisitive, and methodical IT professionals who come prepared to collaborate and share their knowledge.


Your responsibilities will include:

  • Turn scenario specs, decision models and assessment rubrics into prompts, application logic and testable behavior.
  • Spot missing requirements and technical constraints, and raise them with Product, Engineering and SMEs.
  • Define how AI models connect to application services and approved data sources: APIs, tool interactions, data structures, validation, error handling, permitted actions.
  • Design prompts, structured outputs and validation logic, and catch invalid responses before they reach learners.
  • Build conversational runtimes with controlled personas, session state and explicit state transitions.
  • Keep simulation behavior, assessment logic and feedback separate, so feedback cannot change assessment results.
  • Implement grounding and retrieval from approved knowledge sources, with source references where required.
  • Keep code, prompts, schemas and evaluation assets under version control, and document them.
  • Run bounded experiments with Product, with clear questions and success measures.
  • Take validated prototypes into production with Engineering, and check behavior stays the same.
  • Help build the technical controls that keep AI features compliant with accreditation requirements (issue flagging, review steps, disclosure, logging).

What are the required skills?

  • Expert prompt and context engineering: system prompts, few-shot examples, prompt chaining, routing, breaking complex tasks into steps
  • Structured outputs with JSON Schema, function calling and validation layers, with retry and fallback logic
  • RAG design: chunking, metadata filtering, hybrid search, reranking, citation grounding
  • Managing token budgets, prompt caching, context compaction and long-session memory
  • Agentic systems: tool use, planner/executor patterns, multi-agent orchestration, state machines, human-in-the-loop checkpoints
  • Defining tool schemas, permission scopes and action limits for agents
  • AI evaluation: golden datasets, rubric-based scoring, LLM-as-judge calibrated against human raters, regression testing
  • LLM observability and tracing
  • LLM security: prompt injection, data leakage, jailbreaks, unsafe tool actions, and guardrails
  • Comparing prompting, retrieval and model options on quality, latency and cost
  • Python or TypeScript, including API integrations and automated testing
  • Git (e.g. Bitbucket), reproducible tests and CI
  • Works with AI coding agents by default
  • Explains technical trade-offs to non-technical colleagues
  • Sound judgment on privacy, human oversight and the limits of AI-generated assessment or advice
  • Hands-on experience building LLM applications with multi-step conversational workflows
  • Experience collaborating with subject-matter experts

Preferred
  • Background in learning products, assessment, simulations or regulated professional content



If this looks interesting, please use the form below to apply! Remember, this is a fully remote, long-term opportunity :)

Skills Required

  • Expertise in prompt and context engineering, including system prompts, few-shot examples, prompt chaining, routing, and task decomposition
  • Experience with structured outputs, JSON Schema, function calling, validation, retry, and fallback logic
  • Experience designing RAG systems, including chunking, metadata filtering, hybrid search, reranking, and citation grounding
  • Knowledge of token budgets, prompt caching, context compaction, and long-session memory
  • Experience building agentic systems with tool use, planner-executor patterns, multi-agent orchestration, state machines, and human-in-the-loop checkpoints
  • Ability to define tool schemas, permission scopes, and action limits for agents
  • Experience with AI evaluation, golden datasets, rubric-based scoring, calibrated LLM-as-judge evaluation, and regression testing
  • Experience with LLM observability and tracing
  • Knowledge of LLM security, including prompt injection, data leakage, jailbreaks, unsafe tool actions, and guardrails
  • Ability to compare prompting, retrieval, and model options based on quality, latency, and cost
  • Proficiency in Python or TypeScript, including API integrations and automated testing
  • Experience with Git, reproducible tests, and CI
  • Experience building LLM applications with multi-step conversational workflows
  • Experience collaborating with subject-matter experts
  • Ability to explain technical trade-offs to non-technical colleagues
  • Sound judgment regarding privacy, human oversight, and limitations of AI-generated assessment or advice
  • Experience working with AI coding agents
  • Background in learning products, assessment, simulations, or regulated professional content
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Quebec
49 Employees
Year Founded: 2009

What We Do

DistantJob is a remote recruitment agency that connects U.S. and Canadian companies with pre-vetted global tech professionals. They specialize in sourcing, screening, and facilitating remote hiring, handling aspects like international payroll and compliance to help companies hire top engineering talent.

Similar Jobs

Retell AI Logo Retell AI

Artificial Intelligence Engineer

Artificial Intelligence • Information Technology • Software
In-Office or Remote
7 Locations
73 Employees
180K-300K Annually

Clariti Cloud Inc. Logo Clariti Cloud Inc.

Artificial Intelligence Engineer

Cloud • Information Technology • Software
Remote
Canada
135 Employees
150K-180K Annually

Float Logo Float

Staff Engineer

Fintech • Payments • Software • Financial Services
In-Office or Remote
Toronto, ON, CAN
173 Employees

Blue J Logo Blue J

Software Engineer

Artificial Intelligence • Information Technology
Remote
Ontario, ON, CAN
45 Employees

Similar Companies Hiring

Federal Reserve Bank of Chicago Thumbnail
Agency • Fintech • Payments • Sharing Economy • Social Impact
Chicago, IL
1515 Employees
Caxy Thumbnail
Agency • Artificial Intelligence • Enterprise Web • Mobile • Software
Chicago, IL
45 Employees
Fora Thumbnail
Agency • On-Demand • Professional Services • Sales • Software • Travel • Hospitality
New York, NY
250 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account