Senior Software Engineer - Search & AI

Posted 3 Days Ago
Be an Early Applicant
Toronto, ON, CAN
Hybrid
135K-155K Annually
Senior level
Marketing Tech
The Role
Build and operate production search systems using Elasticsearch, Python services, AI inference, embeddings, and LLMs. Responsibilities include index design, relevance tuning, ingestion pipelines, Elastic Cloud operations, Kubernetes deployments, incident response, relevance evaluation, security, CI/CD, monitoring, and API development. The role also involves technical design, documentation, mentoring, and end-to-end ownership of search and AI capabilities.
Summary Generated by Built In
Overview: 

We are hiring a Senior Software Engineer to build and run the search systems that connect Guidepoint clients with the right experts. You will join the Search team in Data Engineering and own work end to end: Elasticsearch index design and cluster operations, the inference services behind semantic search, the Python services in front of it all, and the AI features that turn a plain-language question into a ranked list of people.

Our search sits on the critical path of the business. When a client asks for experts, the quality and speed of what comes back decides whether a project starts. This role is for someone who cares about relevance as much as uptime, and who can explain why a result ranked where it did.

This role will be hybrid based in our Toronto office. 

What You'll Do:

Search and indexing

  • Design index mappings, analyzers, and tokenizers for expert, company, project, and lead data, including semantic_text and dense_vector fields
  • Write and tune queries: bool, function_score, nested, aggregations, kNN, and hybrid retrieval
  • Plan and run reindexes, Painless data fixes, and alias cutovers with zero downtime (we recently retired Solr completely)
  • Build and maintain ingestion: Logstash JDBC pipelines and incremental change feeds from PostgreSQL and SQL Server, including tuning the source SQL when indexing slows down

Cluster operations and maintenance

  • Operate Elastic Cloud deployments on Azure across prod, QA, and dev: sizing, shard allocation, ILM, snapshots, and version upgrades
  • Monitor cluster health in Datadog and Kibana; tune heap, circuit breakers, and fetch sizes; diagnose shard and recovery problems with the _cat and _cluster APIs
  • Manage access: roles, service API keys and their rotation, least-privilege accounts for every consumer
  • Lead incident response for latency spikes, circuit breaker trips, and ingestion failures, then write the RCA and ship the follow-up fixes
  • Load test search under realistic traffic before major mapping, query, or capacity changes

Inference and AI services

  • Run the inference layer behind semantic search: Elasticsearch inference endpoints, embedding models (ELSER, E5, or hosted models), ML node capacity, and cost
  • Build services that turn natural-language requests into structured search queries and filters using LLMs
  • Build and harden MCP servers that let AI agents call search as a tool: auth, consistent error contracts, lean payloads
  • Benchmark LLMs for accuracy, latency, and cost against fixed query batteries, and pick models on evidence
  • Set up offline relevance evaluation (judgment lists, nDCG, recall@k) and run A/B tests online

Python services and APIs

  • Build and maintain the Python APIs and workers, running on AKS (Kubernetes), that sit between clients, AI agents, and the search cluster
  • Own services in production on AKS behind Azure APIM: tests, CI/CD, Docker images, Kubernetes deployments and scaling, dashboards, and alerts
  • Fix findings from security scans and keep secrets out of code and images
  • Review code and raise the bar on design, testing, and readability across the team

Across the team

  • Turn ambiguous product asks into technical designs and break them into deliverable pieces
  • Write design docs and runbooks that others can pick up without you in the room
  • Mentor engineers on search and backend practices
What You Have:
  • 6+ years of professional software engineering, much of it building and running backend systems in production
  • Bachelor’s Degree in Computer Science, Engineering, Information Technology or related field
  • Search engines, in depth (3+ years hands-on with Elasticsearch, Solr, or OpenSearch): mappings and analyzers, Query DSL, relevance tuning, aggregations, cluster sizing and operations, reindexing strategies. You've owned a cluster, not just queried one.
  • Python: strong, idiomatic Python 3 (typing, async, packaging, pytest) and at least one web framework such as Flask or FastAPI
  • Software fundamentals: data structures and algorithms, API design, concurrency, caching, and SQL. You test your code, profile before you optimize, and can reason about failure modes in distributed systems.
  • AI in practice: you've shipped at least one feature that uses embeddings or LLMs, and you know the tradeoffs between lexical, semantic, and hybrid retrieval, including what it takes to run embedding inference in production
  • Operational ownership: comfortable with Docker, Kubernetes (ideally AKS), CI/CD, and observability tooling; you've been on call, run a search cluster through upgrades and incidents, and written the RCA afterward
  • Communication: clear written design docs and the ability to explain search behavior to non-engineers
Nice to Have:
  • Elastic certification (Certified Engineer) or deep experience with Elastic 8.x vector search, ELSER, or ES|QL
  • Migrated a production workload from Solr or OpenSearch to Elasticsearch
  • Learning-to-rank, cross-encoder rerankers, or relevance feedback loops
  • Built RAG pipelines or LLM agents, including evaluation and guardrails
  • Logstash, Kafka, or RabbitMQ for indexing pipelines; load testing with JMeter or k6
  • Azure (AKS, ACR, Pipelines) and Datadog
  • Some PHP or TypeScript, since a few of our consumers are written in them
What We Offer: 

The annual base salary range for this position is 135,000-155,000. Additionally, this position is eligible for an annual discretionary bonus based on performance.

You will also be eligible for the following benefits:

  • Paid Time Off
  • Comprehensive benefits plan
  • Company RRSP Match
  • Development opportunities through the LinkedIn Learning platform
About Guidepoint: 

Guidepoint powers end-to-end research workflows for the world’s best research teams.

Backed by a global network of more than 2 million experts and over 1,600 employees, Guidepoint delivers real-time access to expertise, primary research, and actionable knowledge that help organizations make informed decisions. Through consultations, surveys, events, proprietary content, and AI-enabled tools, we support every stage of the decision-making process—from developing hypotheses and gathering insights to validating assumptions and building conviction for critical decisions.

At Guidepoint, our success relies on the diversity of our employees, experts, and clients, which enables us to foster meaningful connections and a broad range of perspectives. We are committed to creating an inclusive and welcoming environment where individuals of all backgrounds, identities, and experiences can contribute and succeed.

#LI-AP1

#LI-HYBRID

This job posting is for an existing vacancy.  The salary offered will be based on the successful candidate’s skills, experience, qualifications, and location.

Compensation
$135,000—$155,000 CAD

Skills Required

  • 6+ years of professional software engineering experience, including production backend systems
  • Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field
  • 3+ years of hands-on experience with Elasticsearch, Solr, or OpenSearch, including cluster ownership and operations
  • Strong, idiomatic Python 3 skills, including typing, async programming, packaging, and pytest
  • Experience with Flask, FastAPI, or another Python web framework
  • Knowledge of data structures, algorithms, API design, concurrency, caching, SQL, testing, profiling, and distributed-system failure modes
  • Experience shipping at least one production feature using embeddings or large language models
  • Understanding of lexical, semantic, and hybrid retrieval and production embedding inference
  • Operational experience with Docker, Kubernetes, CI/CD, observability, production on-call, search upgrades, incidents, and root-cause analysis
  • Clear written communication, technical design documentation, and ability to explain search behavior to non-engineers
  • Elastic Certified Engineer certification or deep experience with Elastic 8.x vector search, ELSER, or ES|QL
  • Experience migrating production workloads from Solr or OpenSearch to Elasticsearch
  • Experience with learning-to-rank, cross-encoder rerankers, or relevance feedback loops
  • Experience building RAG pipelines or LLM agents with evaluation and guardrails
  • Experience with Logstash, Kafka, RabbitMQ, JMeter, or k6
  • Experience with Azure, AKS, ACR, Azure Pipelines, and Datadog
  • Experience with PHP or TypeScript

Guidepoint Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Guidepoint and has not been reviewed or approved by Guidepoint.

  • Leave & Time Off Breadth — Paid time off and sick days are considered competitive for early‑career roles, with separate allocations for PTO and sick time. This breadth supports baseline work-life needs beyond just holidays.
  • Retirement Support — A 401(k) plan with company matching is highlighted and aligned with mid‑market expectations. Employer verification of the match underscores a tangible long‑term benefit.
  • Wellbeing & Lifestyle Benefits — A casual hybrid work environment, wellness activities, team events, commuter support, and office perks are emphasized. These culture‑oriented offerings add non‑cash value to the overall package.

Guidepoint Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: New York, NY
2,882 Employees
Year Founded: 2003

What We Do

Guidepoint connects clients with vetted subject matter experts—Advisors—from our global professional network. Our clients leverage the insights and perspectives shared by our Advisors to stay informed and make better business decisions. Our multinational client list includes nine of the top 10 global consulting firms, hundreds of hedge funds (including five of the largest firms), and many of the largest private equity firms and Fortune-ranked companies. Guidepoint’s fourteen offices on three continents provide 24/7, quick and agile service.

Similar Jobs

TransUnion Logo TransUnion

Bilingual Associate Lead - Incident Coordinator

Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Remote or Hybrid
Ontario, ON, CAN
13000 Employees
55K-70K Annually

Capital One Logo Capital One

Distinguished Engineer

Fintech • Machine Learning • Payments • Software • Financial Services
Hybrid
Toronto, ON, CAN
55000 Employees

Samsara Logo Samsara

Account Development Representative

Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Easy Apply
Remote or Hybrid
2 Locations
4000 Employees
82K-105K Annually

Cash App Logo Cash App

Senior Software Engineer

Blockchain • Fintech • Mobile • Payments • Software • Financial Services
Remote or Hybrid
8 Locations
3500 Employees
185K-327K Annually

Similar Companies Hiring

MFour Data Research Thumbnail
Marketing Tech • Software
Irvine, CA
98 Employees
Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
567 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
33 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account