Principal AI Architect

Posted 3 Days Ago
Be an Early Applicant
3 Locations
In-Office or Remote
Senior level
Artificial Intelligence • Legal Tech • Professional Services • Consulting
The Role
Design and build a reusable internal platform for secure, governed agentic AI applications. Define agent architectures, tool and data connectors, retrieval, evaluation, observability, security, model routing, memory, and deployment standards. Lead initial applications from discovery through production, establish engineering practices, mentor developers, manage cost and usage, and report business value. The role requires hands-on production coding and enterprise experience with security, compliance, integrations, and operational ownership.
Summary Generated by Built In
We are looking for a technical architect to design and build our internal agentic harness: the shared framework that lets AI agents plan, call tools, read company data, act in our systems and be evaluated and governed safely. On top of that framework, the architect will lead delivery of the first internal agentic applications (for example, contract analysis, integration support for SAP S/4HANA and Ariba, pre-sales and solution design, and support triage).

The harness is the product. Individual agents should be quick to build because the hard parts are solved once: model access, tool and data connectors, memory, permissions, evaluation, observability and human approval.

Key responsibilities

•         Define the reference architecture for agents: orchestration loop, tool calling, context management, memory, sub-agents and human-in-the-loop checkpoints.

•         Build the harness as a reusable internal platform with SDKs, templates and documentation that other engineering teams can adopt.

•         Design the tool and connector layer, including Model Context Protocol (MCP) servers for internal systems (CLM, CRM, ticketing, document stores, SAP sandboxes).

•         Own the evaluation framework: test sets, automated graders, regression gates in CI, and online quality monitoring.

•         Set security and governance controls: identity and permission propagation, sandboxed code execution, prompt-injection defenses, data classification, audit logging.

•         Make model choices and routing decisions on quality, cost and latency; keep the harness model-agnostic.

•         Lead delivery of the first two or three internal applications end to end, from discovery with business owners to production.

•         Set engineering standards, review designs and mentor engineers moving into agent development.

•         Track cost, usage and value per agent, and report outcomes to leadership.

Core technical skills

Area

What the architect must be able to do

Priority

Agent architecture

Design agent loops (plan, act, observe), multi-agent and sub-agent patterns, workflows vs. autonomous agents, and know when an agent is the wrong tool

Must have

LLM engineering

Prompt and system-prompt design, structured outputs, tool/function calling, context-window management, caching, streaming, model routing across providers

Must have

Tools and integration

Build tool schemas and MCP servers; wrap REST, OData and SOAP APIs safely; handle auth (OAuth 2.0, service accounts) and rate limits

Must have

Retrieval and data

RAG design: chunking, embeddings, hybrid search, reranking, vector stores; document parsing (PDF, Word, scanned contracts); permission-aware retrieval

Must have

Evaluation

Build eval sets from real tasks, LLM-as-judge and rule-based graders, regression suites in CI, A/B and online metrics

Must have

Safety and security

Prompt-injection and data-exfiltration defenses, least-privilege tools, sandboxed code execution, approval gates for side-effecting actions, PII handling

Must have

Observability

Tracing of every model and tool call, cost and latency dashboards, replay of failed runs, feedback capture

Must have

Backend engineering

Production Python and/or TypeScript; async and event-driven design; queues, workflow engines, idempotency and retries

Must have

Cloud and platform

Containers, Kubernetes or serverless, infrastructure as code, secrets management, CI/CD on AWS, Azure or GCP

Must have

Memory and state

Short- and long-term memory design, durable state for long-running agents, checkpointing and resumption

Strong plus

Agent frameworks

Hands-on with at least one framework (e.g. LangGraph, Claude Agent SDK, OpenAI Agents SDK, Semantic Kernel) and the judgment to build thin abstractions instead of locking in

Strong plus

Application layer

Chat, copilot and background-agent UX; human review queues; embedding agents in Slack, Teams, browser or existing apps

Strong plus

Cost management

Token budgeting, model tiering, caching, batch processing, unit economics per agent

Strong plus

Fine-tuning and model ops

When to fine-tune vs. prompt vs. retrieve; open-weight model hosting

Nice to have

Experience requirements

  • 10+ years in software engineering, with at least 4 years in an architect or principal engineer role owning platform-level designs.
  • 2+ years building LLM-based systems that reached production use, including at least one agentic system with tool calling and real side effects (not only chat or RAG demos).
  • Has built an internal platform or framework adopted by other teams, and can show how adoption was earned (SDKs, docs, templates, support).
  • Has designed an evaluation approach that caught regressions before release, and can explain how its graders were validated.
  • Has run production systems with on-call responsibility, incident reviews and cost accountability.
  • Has worked in an enterprise environment with SSO, data classification, security reviews and audit requirements.
  • Bachelor’s or master’s in computer science or equivalent practical experience.

Nice-to-have and domain experience

  • Enterprise integration: SAP S/4HANA or SAP Ariba APIs, SAP Integration Suite, Jitterbit or another iPaaS.
  • Contract lifecycle management, procurement or legal tech: clause extraction, obligation tracking, document comparison.
  • Document AI: OCR, layout-aware parsing, table extraction from contracts and PDFs.
  • Workflow engines (Temporal, Step Functions or similar) for long-running agents.
  • Experience with regulated customers (financial services, life sciences) and their AI governance requirements, e.g. ISO 42001, SOC 2, GDPR.
  • Open-source contributions or published writing on agent design or evaluation.

Leadership and working style

  • Builder first. Writes production code, not only diagrams; prototypes quickly, then hardens what works.
  • Evidence over hype. Decides with evals and measurements, and is comfortable saying a use case does not need an agent.
  • Platform mindset. Designs for the tenth agent, not just the first; treats internal teams as customers.
  • Clear communicator. Explains trade-offs to engineers, security, legal and executives in their own terms.
  • Security-minded. Assumes agents will be attacked through their inputs and designs accordingly.
  • Mentor. Raises the team’s skill in a field where most engineers are still learning.


Skills Required

  • 10+ years of software engineering experience
  • At least 4 years in an architect or principal engineer role owning platform-level designs
  • At least 2 years building production LLM-based systems
  • Production experience with an agentic system using tool calling and real side effects
  • Experience building an internal platform or framework adopted by other teams
  • Experience designing evaluation approaches that catch regressions before release
  • Production operations experience including on-call responsibility, incident reviews, and cost accountability
  • Enterprise experience with SSO, data classification, security reviews, and audit requirements
  • Bachelor's or master's degree in computer science, or equivalent practical experience
  • Production Python and/or TypeScript development
  • Experience with agent architecture, LLM engineering, tools and integrations, retrieval, evaluation, safety, observability, backend engineering, and cloud platforms
  • Experience designing memory and durable state for long-running agents
  • Hands-on experience with an agent framework such as LangGraph, Claude Agent SDK, OpenAI Agents SDK, or Semantic Kernel
  • Experience with chat, copilot, background-agent interfaces and human review workflows
  • Experience with token budgeting, model tiering, caching, batch processing, and agent unit economics
  • Experience deciding between fine-tuning, prompting, retrieval, and open-weight model hosting
  • Enterprise integration experience with SAP S/4HANA, SAP Ariba, SAP Integration Suite, Jitterbit, or another iPaaS
  • Contract lifecycle management, procurement, legal technology, or document AI experience
  • Experience with OCR, layout-aware parsing, table extraction, or contract document processing
  • Experience with Temporal, Step Functions, or similar workflow engines
  • Experience with regulated customers and AI governance requirements such as ISO 42001, SOC 2, or GDPR
  • Open-source contributions or published writing on agent design or evaluation
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Fontaine
Year Founded: 2023

What We Do

Execo is a next-generation legal services provider that closes the gap between legal AI investment and real results by delivering AI-powered services for contracts, compliance, and legal operations, blending technology with human expertise.

Similar Jobs

Remote or Hybrid
2 Locations
132624 Employees
Remote
Nānakrāmguda, Rangareddi, Telangāna, IND
80303 Employees

Atlassian Logo Atlassian

Senior Forward Deployed Engineer, AI

Cloud • Information Technology • Productivity • Security • Software • App development • Automation
In-Office or Remote
560071, Domlur, Karnataka, IND
11000 Employees

Cloudflare Logo Cloudflare

Engineering Manager

Cloud • Information Technology • Security • Software • Cybersecurity
Remote or Hybrid
India
4400 Employees

Similar Companies Hiring

Vega Thumbnail
Artificial Intelligence • Automotive • Insurance • Transportation
US
65 Employees
VCA Animal Hospitals Thumbnail
Pet • Professional Services
Los Angeles, CA
35000 Employees
RapidScale Thumbnail
Cloud • Information Technology • Consulting
Raleigh, NC
600 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account