Azure Core powers the platform that runs Microsoft Azure, from datacenter infrastructure through the services that enable customers and Microsoft products to run at global scale. The Software-Defined Datacenter team combines distributed systems, machine learning, and cross-layer optimization to improve the efficiency of Azure's fleet, including Graphics Processing Unit (GPU) capacity used for artificial intelligence inference. You will work on initiatives that reduce inference cost, improve utilization, and unlock more serving capacity from existing GPU infrastructure in close partnership with Microsoft Research, Azure Artificial Intelligence engineering teams, and Azure Core platform teams.
As a Principal Software Engineer, you will be a hands-on technical leader for GPU efficiency. You will design, build, and operate cloud-scale systems that measure, model, and optimize inference workloads across Azure, translating research ideas into reliable production services. This opportunity will allow you to deepen your expertise in artificial intelligence infrastructure, distributed systems, and fleet-wide optimization while collaborating across research, hardware, and Azure Artificial Intelligence teams.
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Responsibilities
- Design, implement, and operate cloud-scale services that improve GPU inference efficiency, utilization, and cost across Azure.
- Partner with Azure Systems Research and Azure AI infrastructure teams to translate algorithms, models, and systems research into production-ready infrastructure.
- Build telemetry, experimentation, and control-plane capabilities that identify inference-efficiency opportunities and safely validate improvements at fleet scale.
- Write high-quality production code in languages such as C#, Rust, Python, or C++ with engineering practices for reliability, security, and maintainability.
- Drive architecture and technical strategy for workload placement, capacity management, performance isolation, and optimization of inference-serving systems.
- Collaborate across Azure Core, hardware, platform, and artificial intelligence teams to land cross-organizational initiatives in production.
- Mentor engineers, lead technical reviews, and help build an inclusive engineering culture focused on customer impact and operational excellence.
Qualifications
Required Qualifications:
- Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- OR equivalent experience.
Other Requirements:
- Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings:
- Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.
- Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- OR Bachelor's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- OR equivalent experience.
- 4+ years of cloud infrastructure or distributed systems development experience.
- 2+ years of experience developing, deploying, or optimizing machine learning, artificial intelligence, or inference-serving systems at scale.
- 4+ years of experience analyzing and troubleshooting large-scale distributed systems in critical production online service environments, including experience with GPU infrastructure, machine learning inference systems, workload scheduling, capacity optimization, or performance engineering.
#azurecorejobs
Software Engineering IC5 - The typical base pay range for this role across the U.S. is USD $142,800 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
Skills Required
- Bachelor's degree in Computer Science or related technical field and 6+ years technical engineering experience (or equivalent).
- Proficiency coding in languages such as C, C++, C#, Java, JavaScript, or Python.
- Ability to meet Microsoft, customer and/or government security screening requirements (Microsoft Cloud Background Check).
- Master's degree in Computer Science and 8+ years experience, or Bachelor's with 12+ years experience, or equivalent.
- 4+ years cloud infrastructure or distributed systems development experience.
- 2+ years developing, deploying, or optimizing machine learning or inference-serving systems at scale.
- 4+ years analyzing and troubleshooting large-scale distributed systems in critical production online service environments, including GPU infrastructure, workload scheduling, capacity optimization, or performance engineering.
Microsoft Compensation & Benefits Highlights
The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Microsoft and has not been reviewed or approved by Microsoft.
-
Fair & Transparent Compensation — Pay is presented as broadly competitive overall, with clear role/level/location variation and an emphasis on using posted ranges and band information for apples-to-apples comparisons.
-
Retirement Support — Retirement benefits are described as a standout, highlighted by a strong 401(k) match structure and immediate vesting, plus additional plan features for tax-advantaged saving.
-
Parental & Family Support — Family-oriented benefits are portrayed as a meaningful strength, with substantial paid parental leave and added supports like back-up care and adoption/surrogacy assistance.
Microsoft Insights
What We Do
At Microsoft, our mission is to empower every person and every organization on the planet to achieve more. Our mission is grounded in both the world in which we live and the future we strive to create. Today, we live in a mobile-first, cloud-first world, and the transformation we are driving across our businesses is designed to enable Microsoft and our customers to thrive in this world.
