Leads the design, development, deployment, and support of scalable data pipelines and platforms across Hadoop, on-premises, and AWS cloud environments. Builds batch and near-real-time processing frameworks, modernizes ETL and data warehouse workloads, implements governance and observability capabilities, and troubleshoots production issues. Mentors engineers, collaborates with cross-functional stakeholders, evaluates emerging technologies, and ensures secure, high-quality delivery for enterprise-scale data ecosystems.
Our Purpose
Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential.
Title and Summary
Lead Data Engineer
Overview
Mastercard powers the global economy by enabling secure, seamless, and intelligent payments across the world. Behind every transaction is a sophisticated technology ecosystem that processes billions of payment events with speed, resilience, and precision.
As a Lead Data Engineer within the Data Collection & Engineering (DC&E) organisation, you will play a critical role in designing, building, and scaling Mastercard's next-generation data platforms and analytical ecosystems. You will develop high-performance, cloud-enabled data pipelines that power Mastercard's enterprise data warehouse and lakehouse environments, enabling advanced analytics, business intelligence, regulatory reporting, and data-driven decision making across the organisation.
This role offers a unique opportunity to solve large-scale data engineering challenges, work with cutting-edge big data technologies, and contribute to a modern cloud transformation programme supporting global payment processing platforms.
Role:
Data Engineering & Development• Design, develop, test, and deploy high-quality, secure, scalable, and resilient data pipelines using Apache Spark, Java/Scala across Hadoop and cloud-native object storage platforms.• Build and maintain batch and near real-time data processing frameworks capable of supporting petabyte-scale workloads.• Develop reusable engineering components and frameworks that accelerate data product delivery while maintaining enterprise standards.
Architecture & Platform Engineering• Design and implement a "build once, run anywhere" architecture supporting seamless deployment across on-premises and public cloud environments without code changes.• Implement data lineage, metadata management, data cataloguing, data quality controls, and observability capabilities across the data ecosystem.• Collaborate with architects and platform teams to establish scalable design patterns and engineering best practices.
Cloud Modernisation• Contribute to migration initiatives moving legacy ETL and data warehouse workloads from on-premises environments to cloud-native architectures.• Leverage cloud services such as Amazon S3, EMR, Glue, and related data services to improve scalability, reliability, and operational efficiency.• Drive adoption of modern lakehouse and distributed compute architectures.
Delivery & Technical Leadership• Lead end-to-end development activities including requirement analysis, solution design, coding, testing, deployment, and production support.• Mentor and guide junior engineers through code reviews, technical coaching, and engineering best practices.• Partner with product owners, analysts, architects, and business stakeholders to deliver high-quality solutions within committed timelines.
Operational Excellence• Troubleshoot complex production incidents and perform root cause analysis to identify and implement long-term remediation strategies.• Ensure compliance with Mastercard's engineering, security, quality assurance, and operational governance standards.• Continuously identify opportunities to improve performance, automation, monitoring, and process efficiency.
Innovation• Evaluate emerging data technologies and conduct proof-of-concept (POC) initiatives to determine their applicability within Mastercard's data ecosystem.• Contribute to engineering innovation and continuous improvement initiatives across the organisation.
All About You:
Required Experience• 10-12 years of experience delivering enterprise-scale Data Warehouse, Data Lake, or Data Lakehouse solutions.• Proven experience implementing multiple end-to-end data engineering projects within large-scale distributed computing environments.• Hands-on experience migrating ETL and analytics workloads from on-premises platforms to cloud-native environments.
Technical Expertise• Strong development experience using:Apache Spark, Scala or Java, Hadoop ecosystem technologies, Object Storage platforms• Experience building orchestration and workflow solutions using: Apache Airflow/ Apache NiFi and Similar enterprise scheduling frameworks• Strong SQL expertise and experience with Relational and NoSQL database technologies: Oracle/ SQL Server, Cassandra, Dynamo DB etc.• Working knowledge of cloud platforms, preferably AWS: Amazon S3, EMR, AWS Glue, Cloud-native data services
Professional Skills• Strong analytical and problem-solving capabilities.• Experience operating within Agile delivery environments.• Excellent written and verbal communication skills.• Proven ability to collaborate within geographically distributed and matrix-based teams.• Self-starter with strong ownership, accountability, and execution focus.• Ability to learn emerging technologies quickly and apply them effectively to business challenges.
Corporate Security Responsibility
All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must:
Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential.
Title and Summary
Lead Data Engineer
Overview
Mastercard powers the global economy by enabling secure, seamless, and intelligent payments across the world. Behind every transaction is a sophisticated technology ecosystem that processes billions of payment events with speed, resilience, and precision.
As a Lead Data Engineer within the Data Collection & Engineering (DC&E) organisation, you will play a critical role in designing, building, and scaling Mastercard's next-generation data platforms and analytical ecosystems. You will develop high-performance, cloud-enabled data pipelines that power Mastercard's enterprise data warehouse and lakehouse environments, enabling advanced analytics, business intelligence, regulatory reporting, and data-driven decision making across the organisation.
This role offers a unique opportunity to solve large-scale data engineering challenges, work with cutting-edge big data technologies, and contribute to a modern cloud transformation programme supporting global payment processing platforms.
Role:
Data Engineering & Development• Design, develop, test, and deploy high-quality, secure, scalable, and resilient data pipelines using Apache Spark, Java/Scala across Hadoop and cloud-native object storage platforms.• Build and maintain batch and near real-time data processing frameworks capable of supporting petabyte-scale workloads.• Develop reusable engineering components and frameworks that accelerate data product delivery while maintaining enterprise standards.
Architecture & Platform Engineering• Design and implement a "build once, run anywhere" architecture supporting seamless deployment across on-premises and public cloud environments without code changes.• Implement data lineage, metadata management, data cataloguing, data quality controls, and observability capabilities across the data ecosystem.• Collaborate with architects and platform teams to establish scalable design patterns and engineering best practices.
Cloud Modernisation• Contribute to migration initiatives moving legacy ETL and data warehouse workloads from on-premises environments to cloud-native architectures.• Leverage cloud services such as Amazon S3, EMR, Glue, and related data services to improve scalability, reliability, and operational efficiency.• Drive adoption of modern lakehouse and distributed compute architectures.
Delivery & Technical Leadership• Lead end-to-end development activities including requirement analysis, solution design, coding, testing, deployment, and production support.• Mentor and guide junior engineers through code reviews, technical coaching, and engineering best practices.• Partner with product owners, analysts, architects, and business stakeholders to deliver high-quality solutions within committed timelines.
Operational Excellence• Troubleshoot complex production incidents and perform root cause analysis to identify and implement long-term remediation strategies.• Ensure compliance with Mastercard's engineering, security, quality assurance, and operational governance standards.• Continuously identify opportunities to improve performance, automation, monitoring, and process efficiency.
Innovation• Evaluate emerging data technologies and conduct proof-of-concept (POC) initiatives to determine their applicability within Mastercard's data ecosystem.• Contribute to engineering innovation and continuous improvement initiatives across the organisation.
All About You:
Required Experience• 10-12 years of experience delivering enterprise-scale Data Warehouse, Data Lake, or Data Lakehouse solutions.• Proven experience implementing multiple end-to-end data engineering projects within large-scale distributed computing environments.• Hands-on experience migrating ETL and analytics workloads from on-premises platforms to cloud-native environments.
Technical Expertise• Strong development experience using:Apache Spark, Scala or Java, Hadoop ecosystem technologies, Object Storage platforms• Experience building orchestration and workflow solutions using: Apache Airflow/ Apache NiFi and Similar enterprise scheduling frameworks• Strong SQL expertise and experience with Relational and NoSQL database technologies: Oracle/ SQL Server, Cassandra, Dynamo DB etc.• Working knowledge of cloud platforms, preferably AWS: Amazon S3, EMR, AWS Glue, Cloud-native data services
Professional Skills• Strong analytical and problem-solving capabilities.• Experience operating within Agile delivery environments.• Excellent written and verbal communication skills.• Proven ability to collaborate within geographically distributed and matrix-based teams.• Self-starter with strong ownership, accountability, and execution focus.• Ability to learn emerging technologies quickly and apply them effectively to business challenges.
Corporate Security Responsibility
All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must:
- Abide by Mastercard's security policies and practices;
- Ensure the confidentiality and integrity of the information being accessed;
- Report any suspected information security violation or breach, and
- Complete all periodic mandatory security trainings in accordance with Mastercard's guidelines.
Mastercard Pune, Mahārāshtra, IND Office
Mastercard Pune Tech Hub Office
Poona Club Road, Pune, Maharashtra, India, 411001
Similar Jobs at Mastercard
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Owns the strategy, roadmap, and execution of Mastercard’s Data Integrity and Transaction Reports products within the MCP platform. Responsibilities include defining product vision, prioritizing investments and backlogs, gathering requirements, partnering with Engineering and cross-functional teams, leading initiatives from concept through launch, improving data quality and customer experience, and measuring adoption and business outcomes.
Top Skills:
My Company Performance (Mcp) Platform
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Leads end-to-end project and marketing operations for European initiatives, including campaign workflows, governance, stakeholder coordination, risk management, executive reporting, dashboards, compliance, and process improvement. Partners with marketing, communications, HR, legal, finance, and agency teams to deliver campaigns on schedule and establish scalable operational frameworks.
Top Skills:
ExcelMicrosoft PowerpointMicrosoft SharepointMicrosoft TeamsPower BIViva Engage
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Manage global statutory reporting, month-end and year-end close processes, financial statement preparation, reconciliations, regulatory reporting, and statutory audits. Coordinate with external auditors and regional finance teams, implement audit recommendations, strengthen internal controls, and drive process improvements. Lead and develop a high-performing Statutory Center of Excellence team while supporting controllership and finance requirements across multiple entities.
Top Skills:
Ai Tools
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
