About Us
At Zyte, we eat data for breakfast and you can eat your breakfast anywhere and work for Zyte. Founded in 2010, we are a globally distributed team of over 250 Zytans working from over 28 countries who are on a mission to enable our customers to extract the data they need to continue to innovate and grow their businesses. We believe that all businesses deserve a smooth pathway to data.
For more than a decade, Zyte has led the way in building powerful, easy-to-use tools to collect, format, and deliver web data, quickly, dependably, and at scale. And today, the data we extract helps thousands of organizations make smarter business decisions, secure competitive advantage, and drive sustainable growth. Today, over 3,000 companies and 1 million developers rely on our tools and services to get the data they need from the web.
Zyte is seeking an experienced Team Lead to manage our Core & MLOps Squad, responsible for "Building the bedrock infrastructure that powers Zyte at scale." This hands-on technical leadership role requires expertise across MLOps, systems programming, and orchestration to lead a cross-functional team in designing and maintaining the scalable foundation that enables all Zyte teams to build and run their services with confidence.
- Design and evolve the core platform (Kubernetes, Mesos, GPU scheduling/autoscaling, distributed compute).
- Own the model platform: registry, experiment tracking, training orchestration, evaluation, serving, and monitoring.
- Build the Golden Path: reference repos, a scaffold CLI, opinionated CI/CD pipelines, runtime contracts (health/metrics/tracing/SLOs), high-performance clients, circuit breakers and other production‑ready defaults.
- Operate a secure, multi‑tenant model registry and training platform with standardized experiment/evaluation harnesses.
- Provide turnkey serving patterns (online + batch), drift/quality monitoring, and rollback playbooks.
- Integrate public/open‑source AI capabilities as managed platform services with cost and data‑governance guardrails.
- Run the squad: roadmap/prioritization, delivery, mentoring, and high engineering standards.
- Partner with product engineering (Zyte API, Scrapy Cloud), Prod Ops, and Security on adoption and rollout plans.
- Mentor the team and foster a platform-thinking mindset.
- Container orchestration (Kubernetes/Knative), GPU provisioning & autoscaling, environment & secret management.
- Operators, sidecars, and internal SDKs/libraries (Go/Rust/Python/Java) that enforce the golden path contract.
- Model platform: registry, experiment tracking, training orchestration, evaluation framework, serving infra, model monitoring.
- Observability: logging/metrics/tracing pipelines;
- Billing pipeline: metering/events/cost tracking abstractions.
- Golden Path: Java, Python, ML templates + CI/CD blueprints + docs + scaffold CLI.
- Reliability enablement (SRE practices), cost governance, supply‑chain security (SBOM, image signing).
- 5+ years experience building distributed systems; 3+ years in MLOps/ML platform engineering (or equivalent impact).
- Knowledge of Linux/OS internals (process model, cgroups/namespaces), networking (TCP/IP, HTTP/2), concurrency, and performance profiling.
- Deep understanding of Kubernetes (bonus: Mesos)
- Proficiency developing high-performance services in Java, Rust, Go or C++ (bonus: familiarity with vert.x and Netty frameworks); strong Python skills.
- Experience with GPU infrastructure (scheduling, containerization, optimization).
- Track record of designing and operating model platforms (registry, training, serving, monitoring) in production.
- Demonstrated success leading technical teams and implementing organization-wide platform solutions.
- Streaming & workflows: Kafka plus Argo/Temporal/Airflow or equivalents.
- eBPF‑based observability, perf tooling, or io_uring experience
- Cost optimization for ML/AI; multi‑tenant quotas and fairness.
- Hands‑on experience authoring Golden Paths (service chassis/templates, CI/CD blueprints, CLI scaffolds).
- SRE practices (SLIs/SLOs, incident management)
Benefits:
- We love fostering and nourishing new ideas and bringing them to market
- Become part of a self-motivated, progressive, multi-cultural team.
- Have the freedom and flexibility to work from where you do your best work, as we are a completely remote company.
- Get the chance to work with cutting-edge open-source technologies and tools.
Top Skills
What We Do
At Zyte, we’re all about empowering data-driven organizations to ethically and accurately collect web data to power their business. With over 14 years experience and our early authorship and ongoing maintenance of Scrapy, we’ve shaped the web scraping industry from Day 1.
We help our clients…
- With easy-to-use ways to collect, format and deliver web data, quickly, dependably and at scale,
- Spend more time gleaning insights from highly accurate, business-critical data, and
- Spend less money on the total cost of ownership in web data extraction.
Zyte API abstracts away a historically disparate web data extraction tech stack into a single tool. Zyte API automates most anti-bot and proxy management, so developers can spend more time on strategy.
Zyte API is a full-stack solution that crawls, unblocks and extracts data in minutes with the power of AI. Developers skip the hassle of creating manual parsing code and extract public data at unlimited scale.
Zyte Data is an expert web data extraction team in your pocket. Our white glove service extracts any web data your business needs, regardless of project size and complexity. This includes a dedicated team and round-the-clock support.
Zyte’s legal team is our backbone and is made up of the leading minds in web data extraction compliance. They stay on top of the ever-changing and opaque laws that loom over the industry. They evaluate compliance risks and inform customers about best practices.
Zyte is certified by and a co-founder of the Ethical Web Data Collection Initiative (EWDCI) which recognizes web data providers operating with the highest level of ethical and legal standards.
Come work for us!
We encourage a flexible and diverse work environment, so we embraced the benefits of remote work from our very early beginnings. Our team includes over 200 employees in over 30 countries. All sharing the same drive, to do more with web data.
