Lead Data Engineer

Our Purpose
Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential.
Title and Summary
Lead Data Engineer
Lead Data Engineer
Mastercard is a global technology company. Our mission is to connect and power an inclusive, digital economy that benefits everyone, everywhere by making payment and data transactions safe, simple, smart, and accessible. Using secure data and networks, partnerships and passion, our innovations and solutions help individuals, financial institutions, governments, and businesses realize their greatest potential.
Overview
ML Engineering team leads AI/ML deployments across Mastercard platforms. The team is responsible for planning the implementation of solutions, choosing the right technologies, and evaluating the evolution of the architecture as the needs change. Data is key for AI/ML.
For this team, MasterCard is seeking a Lead Engineer who is passionate about implementation of Data Engineering and its use in AI/ML across platform (on premise, on cloud, hybrid). The person would be working closely with AI Engineering / Data Science team and platform teams.
Responsibilities -
Design & Develop Scalable Retrieval Data engineering Solutions:• Design and Implement Batch, real-time (RT) and near-real-time (NRT) data engineering pipelines between Mastercard systems to support AI deployments. Engineer scalable and efficient ETL/ELT processes to transform raw data into actionable insights and AI-driven velocities.• Enhance data engineering solutions ensuring reliability, maintainability, and scalability with improved deployment, monitoring, and fault tolerance, features and processes.• Develop monitoring solutions to assess the health and performance of systems.• Participate in technical discussion, ensuring alignment with business goals and data engineering objectives.• Reduce Technical debt and have continuous improvement processes.• KPI first approaches.
2. Ensure Data Security, Integrity, Privacy, and Compliance:• Implement/use data validations frameworks• Implement robust data security and compliance mechanisms, including anonymization and encryption, to handle sensitive data in retrieval systems.• Collaborate with compliance teams to align retrieval infrastructure with Mastercard's data governance policies.• Ensure the quality, accuracy, and integrity of datasets across the organization.• Implement data versioning, lineage tracking, and auditability systems .
3. Innovate in Data Solution Technologies:• Ensure pipeline performance, monitoring and optimization balancing business and technology trade-offs.• Integrate advanced AI infrastructure for scalability, emerging data strategies and distributed systems for improved AI application performance.• Under Architect's directions, implement solutions utilizing platform-as-a-service (PaaS), containerization technologies, and cloud-native architectures.• Design and implement high-transaction volume financial systems with global scalability and extreme uptime requirements.• Design and develop global-scale back-end microservices using Java, Kafka, RabbitMQ, and related technologies.• Build scalable data pipelines with Big Data technologies (e.g., Hadoop, Apache Spark, Spark SQL, Kafka, NiFi) for batch and incremental data loads.• Deploy AI/ML pipelines in production environments using MLOps best practices.• Continuously improve data systems to meet the evolving needs of AI applications and business objectives.
4. Collaborate with AI and Product Engineering Teams:• Partner with business stakeholders, product managers, and leadership teams to translate business requirements into technical solutions.• Collaborate with software engineers and architects to integrate data platforms with Mastercard's enterprise systems.
Corporate Security Responsibility
All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must:
• Abide by Mastercard's security policies and practices.• Ensure the confidentiality and integrity of the information being accessed.• Report any suspected information security violation or breach, and• Complete all periodic mandatory security trainings in accordance with Mastercard's guidelines.
Required Skills & Qualifications:
The number of years of experience is flexible if all other requirements are met.
Must Have• Demonstrated experience in dealing with data engineering complexities of 5 Vs of Big data. • Proven expertise in leading and driving real-time data pipelines and large-scale distributed systems.• Hands-on experience with tools such as Hadoop, Apache Spark, Kafka, NiFi, and Big Data ecosystems.• Strong knowledge of data security, compliance, and data governance standards.• Proficiency in programming languages such as Java, Python, or Scala.• Experience with microservices architecture and messaging frameworks (e.g., Kafka, RabbitMQ).• Familiarity with cloud platforms (AWS) and container orchestration tools (Kubernetes, Docker).• Background in designing and managing high-performance, scalable and secure financial systems with global-scale architecture.• Able to see the big picture, strong design skills as well as micro-level implementation details. • Aligned to our Mastercard ways - We create value, we grow together and we move fast for our customers and each other. • Good communication/presentation skills
Nice to have• Nice to have experience in MLOps workflows and deploying machine learning models in production environments.• Worked in AI projects, have understanding of various ML algorithms, their impacts on infrastructure.
Corporate Security Responsibility
All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must:

• Abide by Mastercard's security policies and practices;
• Ensure the confidentiality and integrity of the information being accessed;
• Report any suspected information security violation or breach, and
• Complete all periodic mandatory security trainings in accordance with Mastercard's guidelines.