The Role
Owns the deployment, monitoring, scaling, and reliability of machine learning models and production pipelines. Responsibilities include building ML CI/CD workflows, containerizing and orchestrating services with Docker and Kubernetes, managing cloud GPU infrastructure, tracking experiments and models, monitoring drift and system health, and optimizing inference performance using Triton, TensorRT, and ONNX Runtime. The role also supports LLM inference, infrastructure as code, observability, and bare-metal Linux troubleshooting.
Summary Generated by Built In
About Tarento:
Tarento is a fast-growing technology consulting company headquartered in Stockholm, with a strong presence in India and clients across the globe. We specialize in digital transformation, product engineering, and enterprise solutions, working across diverse industries including retail, manufacturing, and healthcare. Our teams combine Nordic values with Indian expertise to deliver innovative, scalable, and high-impact solutions.
We're proud to be recognized as a Great Place to Work, a testament to our inclusive culture, strong leadership, and commitment to employee well-being and growth. At Tarento, you’ll be part of a collaborative environment where ideas are valued, learning is continuous, and careers are built on passion and purpose.
Designation: Senior Software Engineer – Analytics
Location: Indore
Educational Qualifications: B.E/B.Tech
Exp: 3-5 Years
Mode of work: Hybrid
Role Overview
We are seeking an AI/ML Ops Engineer to own the deployment, monitoring, and scaling of ML models and pipelines in production. You will build the infrastructure and automation that lets models move reliably from training to production and stay healthy once there.
What you'll do
- Deploy and scale ASR, NMT and TTS models on GPU servers using Docker, Kubernetes and Triton Inference Server
- Build and maintain CI/CD pipelines that take models from training to production
- Monitor services for latency, throughput, errors and GPU utilization, and help debug production issues
- Help optimize inference performance for real-time use cases such as live transcription and voice agents
- Help manage GPU capacity across cloud and on-prem environments
Must have
- 2+ years of hands-on Docker and Kubernetes experience
- CI/CD tooling (GitHub Actions, GitLab CI or Jenkins)
- Strong Python for automation and tooling
- Experience on any cloud platform (AWS, Azure or GCP)
- Has deployed at least one ML model to production with any serving framework (Triton, TorchServe, FastAPI or similar)
- Comfortable with Linux and debugging services
Good to have
- Triton Inference Server: model repositories and dynamic batching
- GPU infrastructure (NVIDIA drivers/CUDA, GPUs on Kubernetes)
- Prometheus and Grafana
- ONNX or TensorRT model conversion
- Serving speech or LLM models where latency matters
- gRPC or streaming services
- NVIDIA NVCF or NIM
- Helm or Terraform
Skills Required
- B.E. or B.Tech degree
- 3-5 years of professional experience
- Hands-on experience with Docker and Kubernetes
- Experience with CI/CD tooling such as GitHub Actions, Jenkins, or GitLab CI
- Experience with model-serving frameworks such as Triton Inference Server or TorchServe
- Cloud platform experience with AWS, Azure, or GCP, especially GPU infrastructure
- Python for automation and tooling
- Experience with experiment tracking and model registries such as MLflow, DVC, or Weights & Biases
- Experience with monitoring and observability tooling such as Prometheus or Grafana
- Production experience with Triton Inference Server, including model repositories, ensembles, dynamic batching, and instance groups
- GPU operations on Kubernetes, including NVIDIA GPU Operator, MIG or time-slicing, node pools, and driver or CUDA management
- Hands-on experience with TensorRT or ONNX Runtime conversion and performance profiling
- Experience with gRPC, streaming service patterns, and load-testing tools such as Locust or k6
- Strong Linux, networking, and debugging fundamentals for bare-metal environments
- Experience deploying LLM or NLP inference services
- Familiarity with vLLM, Ollama, or similar local LLM hosting tools
- Infrastructure-as-code experience with Terraform or Helm
- Experience with NVIDIA NVCF, NeMo, or NIM deployments
- Experience serving TTS or ASR models
- Familiarity with Loki, OpenTelemetry, ELK, and on-call tooling
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Tarento is a Nordic-Indian IT services company that helps organizations navigate digitalization through enterprise applications, data and information management, mobile and custom solutions. Its offerings span digital, data, AI, cloud, consulting, software development, integration, testing, and application management. The company also builds scalable web and mobile apps, portals, platforms, and products for clients across industries, combining technology expertise with data-management capabilities.
