AI & High Performance Computing Infrastructure Engineering Manager, Senior Director

AI & High Performance Computing (HPC) Infrastructure Engineering Manager, Senior Director

We’re seeking a future team member for the role of AI & High Performance Computing Infrastructure Engineering Manager, Senior Director to join our Enterprise Infrastructure Delivery organization. This role is in New York, NY.

The AI and HPC Infrastructure Engineering Manager will lead the engineering and strategic evolution of the bank’s AI, machine learning, and high-performance computing infrastructure platforms. Responsible for managing a team of AI Infrastructure Engineers resources who design, operate, automate, and scale GPU-based infrastructure supporting model training, inference, agentic AI, data pipelines, and production AI workloads.

 In this role, you'll make an impact in the following ways

• Lead and develop a team of AI Infrastructure Engineers responsible for building, operating, and scaling the firm's AI, machine learning, and high-performance computing (HPC) platforms.
• Define and execute the strategic roadmap for AI infrastructure, partnering across Engineering, Architecture, Security, Risk, Compliance, Production Services, and AI/ML teams to deliver secure, scalable, and resilient platforms.
• Drive the design, deployment, and optimization of GPU-based infrastructure supporting model training, inference, retrieval-augmented generation (RAG), agentic AI, data pipelines, and other production AI workloads across on-premises, hybrid, and cloud environments.
• Serve as the senior technical leader for AI infrastructure, guiding decisions related to Kubernetes, distributed systems, GPU orchestration, infrastructure automation, observability, performance engineering, capacity planning, and operational resilience.
• Develop and mature enterprise platform capabilities including AI model serving, scalable data and compute infrastructure, vector and graph database ecosystems, AI orchestration frameworks, and microservices architectures.
• Partner closely with AI/ML engineering teams to accelerate model deployment, improve platform adoption, optimize performance, and enable the delivery of innovative AI solutions.
• Advance automation across infrastructure provisioning, configuration management, monitoring, incident response, workload onboarding, and platform lifecycle management.
• Establish and govern operating models, service standards, production readiness requirements, support processes, and reliability objectives to ensure exceptional platform availability and user experience.
• Manage strategic technology vendor relationships, influencing product roadmaps, driving technical evaluations and proof-of-concepts, and ensuring alignment with business objectives.
• Communicate platform strategy, investment priorities, operational performance, risks, and roadmap progress to executive leadership and technology governance forums.

To be successful in this role, we're seeking the following

• Proven leadership experience managing infrastructure engineering, platform engineering, DevOps, SRE, or production operations teams in large-scale enterprise environments.
• Deep expertise designing and operating AI, machine learning, GPU, HPC, or distributed computing platforms in production environments.
• Strong experience with Kubernetes, containerized platforms, GPU orchestration technologies, and modern infrastructure automation practices.
• Expertise with NVIDIA GPU ecosystems, including GPU lifecycle management, workload optimization, and large-scale compute environments.
• Solid understanding of AI infrastructure patterns, including model training, inference, model serving, RAG architectures, data pipelines, and emerging agentic AI frameworks.
• Experience deploying and supporting hybrid cloud infrastructure, with working knowledge of Azure, GCP, or similar cloud platforms.
• Strong technical foundation in Linux, networking, storage, distributed systems, observability, performance engineering, and production operations.
• Experience leveraging Infrastructure as Code, CI/CD, and automation tools such as Terraform, Ansible, Helm, ArgoCD, GitLab, Jenkins, or similar technologies.
• Ability to influence senior stakeholders and effectively communicate complex technical concepts to both engineering and executive audiences.
• Demonstrated success developing technology strategies, operating models, governance frameworks, and long-term infrastructure roadmaps.
• Strong vendor management experience, including strategic partnerships, architecture reviews, technical assessments, and escalation management.

Preferred Qualifications

• Experience within financial services or other highly regulated industries.
• Expertise supporting AI platforms subject to stringent security, compliance, audit, and operational resilience requirements.
• Experience with large-scale GPU environments, DGX platforms, SuperPOD architectures, InfiniBand, high-performance storage, or enterprise AI infrastructure.
• Familiarity with modern AI serving and orchestration technologies such as Triton, vLLM, NVIDIA NIM, KServe, Seldon, LangGraph, vector databases, graph databases, and multi-agent frameworks.
• Experience with capacity planning, usage analytics, cost optimization, GPU quota management, and chargeback/showback models.
• 10+ years of experience in a related field with a 6-8 years’ experience managing staff 
• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related discipline; advanced STEM degrees or relevant cloud, Kubernetes, AI, or infrastructure certifications are a plus.

At BNY, our culture allows us to run our company better and enables employees’ growth and success. As a leading global financial services company at the heart of the global financial system, we influence nearly 20% of the world’s investible assets. Every day, our teams harness cutting-edge AI and breakthrough technologies to collaborate with clients, driving transformative solutions that redefine industries and uplift communities worldwide.

At BNY, our culture speaks for itself, check out the latest BNY news at BNY Newsroom & BNY LinkedIn

 Here’s a few of our recent awards:

• America’s Most Innovative Companies, Fortune, 2025
• World’s Most Admired Companies, Fortune 2025
• “Most Just Companies”, Just Capital and CNBC, 2025

Our Benefits and Rewards:

BNY offers highly competitive compensation, benefits, and wellbeing programs rooted in a strong culture of excellence and our pay-for-performance philosophy. We provide access to flexible global resources and tools for your life’s journey. Focus on your health, foster your personal resilience, and reach your financial goals as a valued member of our team, along with generous paid leaves, including paid volunteer time, that can support you and your family through moments that matter.

BNY is an Equal Employment Opportunity/Affirmative Action Employer - Underrepresented racial and ethnic groups/Females/Individuals with Disabilities/Protected Veterans.

BNY assesses market data to ensure a competitive compensation package for our employees. The expected base salary for this position when employment commences can be found in the Job Info section at the bottom of the posting. 

Base salary offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, and experience. Base salary is only part of the total rewards package, which may include eligibility for an annual discretionary incentive award. Subject to the terms and conditions of the applicable plans then in effect, eligible employees may enroll in a 401(k) plan as well as participate in Company-sponsored medical, dental, vision, and basic life insurance plans for the employee and the employee’s eligible dependents. Eligible employees also may receive other benefits (including various paid time off benefits, such as vacation and sick time), dependent on the position offered. Details of participation in these benefit plans will be provided if an employee receives an offer of employment.

If hired, the employee will be in an “at will” position and the Company reserves the right to modify base salary (as well as any other discretionary payments or compensation programs) at any time, including for reasons related to individual performance, Company or individual department/team performance, and market factors.