AI Engineer – AI Ops (Monitoring AI Systems in Production)
Role purpose
An AI Engineer in AI Ops & Governance is responsible for operating, monitoring, and maintaining AI models and agent-based systems once they are live in production. The role ensures AI systems remain reliable, performant, compliant, and aligned with business expectations over time, closing the “last‑mile” gap between model development and sustainable production use
Key Responsibilities
Production Monitoring & Health
Monitor AI models and agents in production for performance, latency, errors, and availability.
Track statistical health indicators such as model drift, data distribution changes, and output stability.
Observe business KPIs linked to AI behaviour (e.g. accuracy impact, false‑positive cost, efficiency).
Incident Management & Recovery
Detect and triage production incidents related to AI behaviour or degradation.
Execute rollbacks, throttling, or model disabling where thresholds are breached.
Support root‑cause analysis and post‑incident reviews to prevent recurrence.
Model & Agent Lifecycle Operations
Support deployment, versioning, and release of AI models and agents using CI/CD‑style pipelines.
Maintain registries and metadata covering model ownership, lineage, risk classification, and approvals.
Support models and agents move safely through environments (dev → test → production).
Governance, Risk & Compliance
Ensure AI systems adhere to Responsible AI principles, internal controls, and audit requirements.
Maintain audit trails, logs, and approval artefacts required by risk, compliance, and regulators.
Support fairness, bias, explainability, and transparency monitoring in production.
Tooling & Platform Integration
Integrate AI systems with monitoring, logging, and alerting platforms (e.g. dashboards, metrics stores).
Work with cloud infrastructure (containers, event streaming, APIs) supporting scalable AI operations.
Collaborate with product, engineering, and data teams to standardise AI Ops patterns and blueprints.
Skills & Experience (Baseline)
Strong Python skills and experience supporting ML or LLM‑based systems.
Understanding of Model Ops / MLOps, especially the operational phase after deployment.
Experience with:
Monitoring and logging systems
CI/CD pipelines
Containerised deployments (e.g. Docker‑based runtimes)
Familiarity with cloud platforms (Azure preferred) and production troubleshooting.
Ability to work cross‑functionally with product, data science, engineering, and risk teams
Skills Required
- Strong Python skills
- Experience supporting machine learning or LLM-based systems
- Understanding of Model Ops or MLOps, especially post-deployment operations
- Experience with monitoring and logging systems
- Experience with CI/CD pipelines
- Experience with containerized deployments, such as Docker-based runtimes
- Familiarity with cloud platforms
- Production troubleshooting experience
- Ability to work cross-functionally with product, data science, engineering, and risk teams
- Azure cloud platform experience
What We Do
VALCE Talent Solutions is a recruitment agency and consulting firm specializing in IT talent acquisition and nearshoring, primarily in Mexico. They design customized solutions in IT talent and process optimization to help businesses scale intelligently and profitably. Their expertise focuses on specialized industries including Information Technology, Operations Management, and Supply Chain, providing end-to-end talent offerings to attract and retain top professional profiles.
