AI developer (Founding team)

You will be the intelligence architect of Exterview. As an AI Engineer, you’ll design, implement, and optimize the multi-agent orchestration system that powers AI-driven candidate screening, interview agents, scoring agents, and feedback agents.

This is applied AI at scale: integrating Azure OpenAI models, Cognitive Services, Pub/Sub messaging, and Neo4j graphs into a production-grade talent intelligence platform. You’ll work closely with Backend, Prompt Engineering, and Frontend teams to ensure AI capabilities feel seamless, fast, and trustworthy for enterprises managing thousands of interviews concurrently.

Execution is tracked via Linear (tasks), Notion (PRDs and experiments), and GitHub (code reviews), ensuring the AI system’s evolution is transparent and collaborative.

• Implement candidate-facing agents and sub-agents (ScreeningAgent, InterviewAgent, FeedbackAgent) using a modular, production-ready framework.

• Ensure reliability, fault-tolerance, and asynchronous coordination across agents.

• Integrate Azure OpenAI models, Cognitive Services (Text, Vision, Speech), and other Azure AI APIs for multimodal reasoning (text, voice, video).

• Optimize hybrid model selection for performance vs. accuracy tradeoffs.

• Design and maintain Agent-to-Agent (A2A) communication via Azure Service Bus / Event Grid.

• Guarantee low-latency, asynchronous messaging for real-time interview and scoring flows.

• Build working memory in Redis, long-term candidate data in Cosmos DB, and Neo4j-based Candidate Graph to support adaptive interview flows.

• Collaborate with Prompt Engineers to generate scorecards, skill assessments, interview transcripts, and AI-driven feedback.

• Ensure structured outputs usable by both FE dashboards and analytics modules.

• Build evaluation pipelines to test prompt stability, reduce hallucinations, and monitor model drift using LangSmith / Langfuse or internal Azure monitoring tools.

• Integrate retrieval-augmented generation (RAG) pipelines with candidate profiles, GitHub repos, and job-related documentation for contextualized insights.

• Implement performance-tracking and adaptive feedback systems that adjust AI interview and assessment flow based on candidate results.

• Benchmark and optimize model orchestration for concurrent interviews and batch processing of AI-driven assessments.

• Implement guardrails against unsafe generations, prompt injection, and biased outputs.

• Ensure compliance with enterprise data privacy standards.

• Translate PM requirements into AI system designs, coordinate with Backend for APIs, and FE for real-time outputs.

• Maintain shared understanding of agent contracts and orchestration patterns.

• AI InterviewAgent and ScreeningAgent integrated with Azure OpenAI + Service Bus messaging.

• Redis working memory and Cosmos DB candidate schema live.

• LangSmith evaluation pipeline created with baseline metrics (<15% hallucination rate).

• Orchestrate 10+ specialized agents in production.

• Reduce end-to-end agent orchestration latency <3s.

• Adaptive assessment system live with >40% improvement in candidate evaluation consistency.

• AI drift monitoring and guardrails automated in CI/CD pipelines.

• 8–10+ years in applied AI/ML engineering.

• Expertise in LLM orchestration frameworks (LangChain, CrewAI, or Azure-native equivalents).

• Deep experience with multimodal AI integration (text, speech, vision).

• Strong knowledge of messaging/event systems (Azure Service Bus, Event Grid, Kafka).

• Proven delivery of AI-first products in production environments.

• Research experience in NLP, RLHF, or agentic AI systems.

• Contributions to open-source AI frameworks.

• Prior work on enterprise-focused AI products.

• Startup or founding engineer experience.

• Core AI: Azure OpenAI, Cognitive Services (Speech, Vision, Language)

• Communication: Azure Service Bus, Event Grid

• Memory: Redis (working), Cosmos DB (archival), Neo4j Aura (Candidate Graph)

• Evaluation & Prompting: LangSmith, Langfuse, hybrid model router

• Infra: Azure Functions / App Service, API Management, GitHub Actions

• Tools: Linear (execution), Notion (PRDs/experiments), GitHub (repos)

Validate your ability to design and implement a production-ready, AI-driven multi-agent hiring system using a MERN-based UI with Azure cloud orchestration.

Build an AI Interview System that evaluates candidates for a specific role (e.g. “Frontend Engineer”) using multiple autonomous agents.

The system should:

1. Take an input like:
   “Evaluate candidate for Frontend Engineer.”

2. Orchestrate multiple sub-agents:

   • ScreeningAgent → generates an initial fit score

   • InterviewAgent → conducts a simulated technical Q&A

   • FeedbackAgent → generates a skill-based scorecard and analysis

3. Store results in:

   • Cosmos DB (Mongo API)

   • Candidate Graph (Neo4j)

4. Communicate between agents using Azure Service Bus.

5. Include an Adaptive Feedback Loop:

   • If the overall score is below 60%, regenerate questions or trigger an alternate evaluation.

6. Include Evaluation & Guardrails:

   • LangSmith (or equivalent) evaluation pipeline with 20+ golden prompts

   • Guardrails to filter unsafe or hallucinated outputs

7. Meet performance goals:

   • End-to-end orchestration: < 3 seconds

   • Each sub-agent response: < 1.5 seconds

• Agents communicate asynchronously via Azure Service Bus (topics & subscriptions).

• Coordinator service aggregates sub-agent results and applies the adaptive loop.

• Use structured JSON messages for communication between agents.

• Cosmos DB (Mongo API) → store candidate, evaluation, and feedback data.

• Neo4j → maintain a relationship graph between candidates, roles, and skills.

Build a React + Express (Node.js) UI with these views:

1. Dashboard – list of evaluations and their statuses.

2. New Evaluation Form – start a new evaluation by entering role and candidate info.

3. Live Orchestration View – show progress of sub-agents (Screening, Interview, Feedback).

4. Result Report – display scorecards, feedback, and regenerate button for low scores.

• If total score < 60%, trigger a second evaluation or re-generate interview questions.

• Integrate a LangSmith (or equivalent) evaluation pipeline with at least 20 golden prompts.

• Guardrails should:

  • Prevent unsafe or biased outputs.

  • Filter hallucinated or unverifiable facts.

  • Enforce output schema compliance.

Frontend: React (TypeScript) + Tailwind CSS
Backend:Node.js (Express) + TypeScript
Database: Azure Cosmos DB (Mongo API)
Graph DB: Neo4j
Messaging: Azure Service Bus
LLMs Azure: OpenAI or OpenAI API (mock/local acceptable)
Evaluation :LangSmith (or mock evaluator)
CI/CD: GitHub Actions
Hosting: Azure (optional for PoC)

MetricTargetEnd-to-End Orchestration< 3 secondsSub-Agent Response< 1.5 seconds eachEvaluation Accuracy≥ 85% vs golden promptsHallucination Rate< 5%CI/CD Build< 2 minutes

You are expected to submit the following items:

1. Complete codebase:

   • Frontend (React)

   • Backend (Express/Node)

   • Agent services (Screening, Interview, Feedback)

2. Database schemas:

   • Cosmos DB document examples

   • Neo4j Cypher schema

3. Evaluation report:

   • Accuracy, latency, hallucination rate

4. Guardrail implementation:

   • With example of filtered unsafe output

5. CI/CD Pipeline:

   • GitHub Actions for build, lint, test, deploy

6. Demo video (Loom):

   • ≤ 5 minutes walkthrough of architecture and demo run

7. Documentation:

   • Setup & run instructions

   • Architecture diagram

   • System flow summary