Responsibilities
Own and drive the end-to-end CADET architecture, integration contracts, and technical execution for the customer-signal and evaluation system, spanning capture, enrichment, prioritization, triage, asset generation, execution, diagnosis, reporting, and retention. Resolve architectural ambiguity, accelerate delivery, and create reusable capabilities across partner teams while retaining accountability for CADET-specific workflows, interfaces, and system behavior and reusing shared platform capabilities.
Create scalable services and pipelines that reproduce customer pain points across scenarios, business segments, and interventions through automated grounding-data generation, intent classification, deduplication, clustering, taxonomy assignment, coverage analysis, loss-pattern detection, and work-item routing.
Automate recurring evaluation execution and quality gates for top intents and customers, including scheduling, versioning, reproducibility, failure handling, and result publication.
Establish data contracts and extensible schemas for customer signals, workflows, evaluation cases, rubrics, graders, runs, results, attribution, and regression assets.
Develop quality-attribution workflows that combine automated diagnosis with human review; route losses to the appropriate product, data, model, tool, orchestration, or evaluation owner; track remediation; add regression protection; and verify improvement in customer outcomes.
Create observability and benchmarks for pipeline health, agent behavior, data quality, grader health, coverage, cost, latency, operational reliability, and regression detection across CADET workflows and tooling.
Design safe human-in-the-loop gates for ambiguous data, sensitive customer evidence, taxonomy changes, evaluation publication, and consequential quality decisions.
Mentor engineers across the team and raise engineering quality through clear interfaces, testing, security, operational discipline, and technical documentation.
Qualifications
Required Qualifications:
- Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- OR equivalent experience.
Preferred Qualifications:
- Extensive experience designing and delivering large-scale distributed systems, data platforms, developer platforms, or AI infrastructure in production, with practical exposure to large language model technologies, agent systems, or AI/ML platforms.
- Demonstrated principal-level technical leadership across teams and organizational boundaries.
- Solid understanding of security, privacy, data governance, and responsible engineering for sensitive data.
- Excellent communication and collaboration skills with the ability to influence senior engineering, science, and product leaders.
- Experience building AI/ML platforms, LLM or agent systems, evaluation infrastructure, experimentation systems, or model pipelines.
- Experience with retrieval, tool-using agents, prompt and rubric management, human-in-the-loop systems, or automated diagnosis.
- Experience building data and workflow platforms that combine batch, streaming, unstructured, and human-generated inputs.
- Experience with RLEs, synthetic environments, post-training pipelines, or large-scale evaluation execution.
- Experience integrating customer feedback, product telemetry, and engineering work-management systems.
- Experience operating high-availability services and driving platform adoption across multiple product teams, with knowledge of competing AI models, platforms, and developer ecosystems and the ability to apply relevant external advances to Microsoft’s evaluation and quality systems.
#cadets
Software Engineering IC5 - The typical base pay range for this role across the U.S. is USD $142,800 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
Skills Required
- Bachelor's degree in Computer Science or a related technical field and 6+ years of technical engineering experience with coding in C, C++, C#, Java, JavaScript, Python, or comparable languages; equivalent experience accepted.
- Experience designing and delivering large-scale distributed systems, data platforms, developer platforms, or AI infrastructure in production.
- Principal-level technical leadership across teams and organizational boundaries.
- Understanding of security, privacy, data governance, and responsible engineering for sensitive data.
- Experience building AI/ML platforms, LLM or agent systems, evaluation infrastructure, experimentation systems, or model pipelines.
- Experience with retrieval, tool-using agents, prompt and rubric management, human-in-the-loop systems, or automated diagnosis.
- Experience building data and workflow platforms combining batch, streaming, unstructured, and human-generated inputs.
- Experience with RLEs, synthetic environments, post-training pipelines, or large-scale evaluation execution.
- Experience integrating customer feedback, product telemetry, and engineering work-management systems.
- Experience operating high-availability services and driving platform adoption across multiple product teams.
- Knowledge of competing AI models, platforms, and developer ecosystems, with ability to apply relevant external advances.
- Excellent communication and collaboration skills with senior engineering, science, and product leaders.
Microsoft Compensation & Benefits Highlights
The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Microsoft and has not been reviewed or approved by Microsoft.
-
Fair & Transparent Compensation — Pay is presented as broadly competitive overall, with clear role/level/location variation and an emphasis on using posted ranges and band information for apples-to-apples comparisons.
-
Retirement Support — Retirement benefits are described as a standout, highlighted by a strong 401(k) match structure and immediate vesting, plus additional plan features for tax-advantaged saving.
-
Parental & Family Support — Family-oriented benefits are portrayed as a meaningful strength, with substantial paid parental leave and added supports like back-up care and adoption/surrogacy assistance.
Microsoft Insights
Similar Jobs
What We Do
At Microsoft, our mission is to empower every person and every organization on the planet to achieve more. Our mission is grounded in both the world in which we live and the future we strive to create. Today, we live in a mobile-first, cloud-first world, and the transformation we are driving across our businesses is designed to enable Microsoft and our customers to thrive in this world.
