Machine Learning Analyst - Data Science

The BidSwitch Data Science team works on the ML systems at the core of a large-scale programmatic trading platform — the algorithms that decide, billions of times a day, which bid requests are worth processing and which demand partners they should be routed to. Alongside model research, the team owns a broad analytical scope: making sure those models and the system around them run efficiently and profitably. This role is dedicated to that analytical scope, working side by side with our ML researchers. 

You won't be building ML models. You will be the person who understands how they perform in production, finds inefficiencies in the system around them, and drives fixes through to implementation. If your idea of a great week is discovering a hidden $100k/year inefficiency in traffic or infrastructure costs — and then actually shipping the fix, not just filing it in a backlog — this role is for you. 

Rough split of the work: ~45% insight hunting & efficiency analysis (traffic, trading pairs, unit costs) · ~30% monitoring, dashboards, tooling · ~25% stakeholder support and ad-hoc analysis. 

Key Responsibilities 

Efficiency & unit economics 

Own hardware unit-cost management: cost recalculation, financial forecasts, and making sure every processed request is profitable. 

Proactively hunt inefficiencies in incoming traffic and system setup: bidstream management (QPS spikes, unprofitable traffic segments), supply path optimization, duplication/multisize handling, latency (t_max) tuning. 

Drive each finding from detection to executed change, and measure the realized $ impact. 

Launch and manage A/B tests to validate your changes: you own the experiment from design and rollout control to the final verdict — so every improvement ships with measured, defensible impact. 

ML performance insights 

Team up with Data Scientists on model performance investigations: they go deep into model internals and algorithm details, you work the data side — testing hypotheses, localizing inefficiencies, and turning findings into concrete research cases for the team. 

Go below the top-line aggregates: analyze performance at the level of mid- and small-size partners and segments, where inefficiencies are individually small but cumulatively worth real money. 

Analyze cross-component inefficiencies and trading-pair performance; turn recurring manual checks into automated reports and tooling. 

Perform root-cause analysis on incidents affecting performance or margin. 

Collaboration 

Work day-to-day with Data Scientists, R&D, Product, TAM, and Client Services; turn their questions into evidence-backed answers and proactive insights. 

Communicate findings clearly to both technical and non-technical audiences, in writing and in person. 

What You Won't Do 

Develop new ML models or algorithms from scratch — that stays with our ML researchers, whom you'll work alongside daily. 

Build production infrastructure or model-serving systems. 

Sit in a support-ticket queue. Ad-hoc requests from stakeholders are a real part of the role, but the goal is to automate recurring ones so no investigation is done by hand twice. 

• Bias to implementation: you get satisfaction from initiatives that reach production and generate or save money — not from accumulating a backlog of ideas. 

• Proactive ownership: you'll often be the first to notice something is wrong, and you'll need to convince others to act on it. 

• Critical thinking: you validate before you trust — whether it's a data source, a stakeholder's assumption, or an LLM's answer. In the age of AI-assisted workflows, this matters more than ever. 

• Strong communicator and relationship builder: able to explain a technical finding clearly to a non-technical reader, and to build working relationships across DS, R&D, Product, TAM, and Client Services. 

• SQL — confident with window functions, aggregation over TB-scale event logs, and writing cost-aware queries. Comfortable being handed raw data and a vague question. 

• Python — pandas, numpy, visualization; able to turn one-off analyses into repeatable reports and small pipelines. 

• Solid grasp of statistics and A/B testing fundamentals. 

• Working understanding of ML metrics and model behavior — enough to monitor, interpret, and debug model performance. You don't need to design algorithms. 

• Experience with BI and monitoring tooling (Grafana, Tableau, Superset, or comparable in-house/OLAP systems). We rely heavily on internal tools, so tool-agnostic thinking matters more than any specific product. 

• 3+ years of hands-on analytical experience with production-scale data. 

• Fluent English and Russian, written and spoken. 

Strong Plus (not required) 

• We will teach the domain — but if you already have it, you'll be productive months earlier: Programmatic / RTB fundamentals: bid request lifecycle, auction mechanics, win rate, QPS, timeouts, no-bid reasons. 

• Ecosystem roles: DSP, SSP, ad exchange; supply chain (schain) analysis. 

• Unit economics or infrastructure cost management experience. 

• Interest in LLM-based tooling and workflow automation — the team actively uses agentic AI tools in daily work. 

Room to Grow 

If you're curious about the ML side, there's an open door: you can get hands-on with our experiment pipeline — launching lightweight ML experiments and validating product hypotheses together with Data Scientists and PMs. This is optional, not a core expectation of the role. 

🏢 Ways of working – Our hybrid model blends home with in-office experiences, making space for both.  
📈 Grow with us – Learning, mentorship & career development programs.  
💪 Your wellbeing matters – Health benefits, wellness perks & mental health support.  
🤝 A team that cares – Diverse, inclusive, and globally connected.  
💸 Fair pay & perks – Attractive salary, with performance-based rewards and family-friendly policies, plus the potential for equity depending on role and level.  

Additional benefits may vary depending on the country where you work and the nature of your employment with Criteo.