Principal AI Engineer

Posted 19 Days Ago
Be an Early Applicant
Toronto, ON, CAN
In-Office
306K-384K Annually
Senior level
Artificial Intelligence • eCommerce • Software
The Role
Lead architecture and build an internal, scalable evaluation platform for LLMs and agentic systems. Own orchestration, automation, and observability to turn manual prompt iteration into reproducible evaluation loops. Act as AI technical lead for squads, define quality metrics, mentor engineers, and collaborate with infrastructure teams to run large-scale evaluations and agent simulations.
Summary Generated by Built In

We believe conversations will become the #1 way to shop.

At Gorgias, we’re building the platform that makes this real: a unified AI agent that sells, supports, and re-engages customers across the entire journey. Conversational Commerce is the future of ecommerce, and we’re leading that shift.

Our mission is to turn every interaction between a brand and its customers into a relationship: personal, seamless, and intelligent. By combining deep product expertise with the latest in AI, we’re making shopping feel more natural, human, and connected than ever before.

To win, we focus relentlessly on:

  • Quality: conversations that feel authentic and on-brand.

  • Experience: effortless shopping from chat to checkout.

  • Re-engagement: personal, 1-1 dialogue instead of noisy marketing.

The opportunity is massive. As AI reshapes how people buy, Gorgias is building the foundation for the next decade of ecommerce, where every brand has its own intelligent agent and every customer feels understood.

Join us to make Conversational Commerce real.

Team & Context

Gorgias is an AI-first company building products powered by LLMs and agent-based systems.

As we scale our AI capabilities, we need to improve how we evaluate, iterate, and operate these systems in production. Today, parts of this process remain manual or fragmented, especially around prompt iteration, validation, and evaluation workflows.

This role will focus on building and scaling the systems that support AI evaluation and iteration, helping the team move faster and more reliably.

About the Role

You’ll have a chance to:

  • 🤖 Work on production AI systems used by thousands of businesses

  • 🧠 Define how we evaluate and improve AI performance at scale

  • 🏗️ Build internal platforms and tooling used by AI and engineering teams

  • 📈 Reduce manual processes and improve iteration speed on AI features

  • 👥 Collaborate across AI, ML, and product teams

  • 🧑‍🏫 Raise the engineering bar and mentor others

What You’ll Do

1. Architect the Evaluation "Factory"

  • End-to-End Platform Ownership: Architect and lead the development of our internal evaluation platform, moving the needle from manual testing to a fully automated lifecycle (from LLM-as-a-judge creation to production monitoring).

  • Accelerate Time-to-Market: Directly impact our primary KPI by designing tools and workflows that drastically reduce the time it takes to deliver a calibrated, production-ready agent.

  • Infrastructure Collaboration: Partner with the Orchestration team to build the robust, scalable infrastructure required to run complex evals and agentic simulations at scale.

2. Scaling AI Expertise

  • Squad Empowerment: Serve as the "AI Technical Lead" for product squads, guiding them through the complexities of agent design, failure analysis, and prompting best practices.

  • Decentralize Quality: Instead of being a bottleneck, you will build the "paved road" that allows product squads to become autonomous in measuring and maintaining their own agent quality.

  • Standard Setting: Define what "good" looks like for AI at [Company Name]. You’ll translate non-deterministic AI behavior into predictable engineering metrics that the whole organization can trust.

3. Engineering Leadership

  • Mentor & Level Up: Bridge the gap between traditional software engineering and AI. You’ll mentor engineers on how to apply rigorous system design to the world of LLMs and agents.

  • Continuous Observability: Take ownership of the feedback loop, ensuring that production insights from our agents directly inform the next iteration of our evaluation datasets.

Who You Are
  • 8+ Years of Engineering Excellence: You are a Staff-level engineer first. You’ve built systems that handle high scale, and you know how to architect for long-term maintainability and performance.

  • Agentic Curiosity: You’ve moved beyond the "chatbot" phase and are actively experimenting with AI Agents. You understand that the challenge isn't the prompt, but the orchestration, state management, and reliability of the agent's actions.

  • Systems Thinker (Non-Deterministic Mindset): You recognize that AI is probabilistic. You are excited by the challenge of building deterministic "wrappers" and Evaluation loops around models to make them safe for production.

  • The "Applied" Edge: You likely come from a background in distributed systems, internal platforms, or developer tooling, and you're now applying that rigor to the AI stack.

What We’re Looking For
  • Beyond the Wrapper: You have serious experience moving beyond simple API calls to architecting multi-stage AI orchestrations (agents, chained workflows, or custom runtime logic).

  • Orchestration Experience: Even if you aren't an AI researcher, you have experience building complex, multi-step workflows (e.g., temporal systems, state machines, or event-driven architectures) and want to apply this to Agentic loops.

  • Reliability Obsession: You understand why "vibes-based" testing doesn't work. You’ve started exploring or building Eval frameworks to measure how models perform against real-world data.

  • Infrastructure Mindset: You are comfortable with the "glue" that makes AI work: vector databases, semantic caching, and API integration with third-party tools.

Tech Stack & Experience
  • Strong backend experience (Python preferred)

  • Experience with distributed systems and event-driven architectures

  • Familiarity with tools like Kafka, Pub/Sub, or equivalent

  • Experience working with LLMs (prompting, RAG, agents, evaluation workflows)

  • Experience building APIs and scalable services

  • Understanding of monitoring, observability, and system performance

Hiring Process
  • Recruiter phone screen

  • HM Interview

  • System Design Interview

  • AI Case Study (take-home, ~1–2 hours)

  • Technical Deep Dive of case study

  • Final Leadership Interview


Perks & Benefits

🏖️ 5-week vacation (We follow each country's appropriate PTO Laws)

🤕 Paid sick leave

🧸 Paid parental leave (16 weeks)

💻 MacBook Pro

🏥 Private health insurance and retirement pension (RRSP with Gorgias matching up to 4%)

✈️ For a smooth onboarding, we invite you to our Toronto office for one week (flights and accommodation handled by Gorgias)

💆🏻‍♀️ Get up to $900 CAD to set up your workstation at home

📚 Get up to $2,600 CAD of learning material per year (books, courses, training, and individual coaching)

🥰 Every quarter, we organize a company-wide summit to discuss where we're going and strengthen social bonds. Once per year we organize offsite team retreats and company retreats!

AI at Gorgias
At Gorgias, AI is a natural extension of how we work and build. Our teams use it every day to research, write, analyze, code, and craft better customer experiences. Everyone has access to premium AI tools (ChatGPT, Claude, Granola & others) and an annual L&D budget to explore new ones.

The real magic happens when we share what we learn. Our #powerup Slack channel is a digital petri dish of new tools and workflows, and each team has AI champions who showcase fresh ideas during weekly company-wide standups, now practically AI demo sessions.

We see AI not as a replacement for creativity or empathy, but as a multiplier, helping us move faster, think deeper, and serve customers better.
AI use in Recruiting at Gorgias
By submitting your application, you agree that Gorgias may collect and process your personal data for recruiting, workforce planning, and related purposes. For more information about how we process your data and your rights, please refer to our Applicant Privacy Policy.
Diversity & Inclusion at Gorgias
We’re committed to creating an inclusive environment where everyone can thrive. We welcome applicants from all backgrounds, experiences, and perspectives because diverse teams drive innovation and better decision-making.

If you need accommodations during the application or interview process, please contact us at [email protected].

Skills Required

  • 8+ years of engineering experience; Staff/Principal-level expectations
  • Experience designing and implementing AI agents and multi-stage orchestrations
  • Experience building evaluation frameworks and automated LLM evaluation workflows
  • Strong backend engineering experience
  • Python
  • Experience with distributed systems and event-driven architectures (Temporal, state machines)
  • Familiarity with Kafka, Pub/Sub, or equivalent messaging systems
  • Experience with LLMs: prompting, RAG, agents, evaluation workflows
  • Experience building APIs and scalable services
  • Experience with vector databases and semantic caching
  • Understanding of monitoring, observability, and system performance
  • Mentoring and engineering leadership experience
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Uxbridge
400 Employees
Year Founded: 2015

What We Do

Gorgias empowers ecommerce brands to grow through AI-powered customer experience. We are the #1 CX platform in the industry, trusted by over 15,000 merchants worldwide – from small independent shops to some of the largest ecommerce brands in the world. We offer the most integrations of any tool on Shopify (100+) and the ability to get setup fast, without the need for complex onboarding. Gorgias offers its users a unified platform to manage every aspect of their customer support on every channel. We can automate 60% of a brand’s support so that agents can focus on high-value conversations and driving sales. Plus, we offer purpose-built marketing tools to help merchants convert more shoppers into customers, driving GMV.

Why Work With Us

We're a data-driven team which encourages collaboration and ensures every employee has the opportunity to learn and grow. At Gorgias, we believe in our mission and to make it happen, we are creating a cross-collaborative team of talented people who live by our values, believe in the product and are willing to learn. Join our team and grow with us!

Gallery

Gallery

Similar Jobs

Block Logo Block

Principal Engineer

Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
In-Office or Remote
8 Locations
12000 Employees
319K-479K Annually

Autodesk Logo Autodesk

Principal Engineer

Big Data • Cloud • Digital Media • Machine Learning • Mobile • Software • Industrial
In-Office
2 Locations
13285 Employees
131K-191K Annually

NXP Semiconductors Logo NXP Semiconductors

Software Engineer

Automotive • Internet of Things • Mobile • Semiconductor • Industrial
In-Office
Kanata, ON, CAN
21993 Employees

Caseware Logo Caseware

Principal Software Engineer

Cloud • Fintech • Software • Analytics
Hybrid
Toronto, ON, CAN
570 Employees

Similar Companies Hiring

Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
178 Employees
Kepler  Thumbnail
Fintech • Software
New York, New York
6 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account