To build our Payment services, we got technical cooperation from Paytm (A large payment service company in India). And based on their customer-first technologies , we created and expanded the smartphone payment service in Japan. Therefore, we have decided to establish a development base in India, because it is a major IT country with many talented engineers, as evidenced by the fact that cutting-edge mobile payments can continue to be generated.
OUR VISION IS UNLIMITEDPayPay's rapid growth necessitates the expansion of its product teams and underscores the critical need for a resilient Data Engineering Platform. This platform is vital to support our increasing business demands. The Data Pipeline team is tasked with creating, deploying, and managing this platform, utilizing leading technologies like Databricks, Delta Lake, Spark, PySpark, Scala, and the AWS suite.
We are actively seeking skilled Data Engineers to join our team and contribute to scaling our platform across the organization.
Main Responsibilities
- Create and manage robust data ingestion pipelines leveraging Databricks, Airflow, Kafka, and Terraform.
- Ensure high performance, reliability, and efficiency by optimizing large-scale data pipelines.
- Develop data processing workflows using Databricks, Delta Lake, and Spark technologies.
- Maintain and improve the Data Lakehouse, utilizing Unity Catalog for efficient data management and discovery.
- Construct automation, frameworks, and enhanced tools to streamline data engineering workflows.
- Collaborate across teams to facilitate smooth data flow and integration.
- Enforce best practices in observability, data governance, security, and regulatory compliance
Qualifications
- Minimum 7 years as a Data Engineer or similar role.
- Hands-on experience with Databricks, Delta Lake, Spark, and Scala.
- Proven ability to design, build, and operate Data Lakes or Data Warehouses.
- Proficiency with Data Orchestration tools (Airflow, Dagster, Prefect).
- Familiarity with Change Data Capture tools (Canal, Debezium, Maxwell).
- Strong command of at least one primary language (Scala, Python, etc.) and SQL.
- Experience with data catalog and metadata management (Unity Catalog, Lakeformation).
- Experience in Infrastructure as Code (IaC) using Terraform.
- Excellent problem-solving and debugging abilities for complex data challenges.
- Strong communication and collaboration skills.
- Capability to make informed decisions, learn quickly, and consider complex technical contexts.
Similar Jobs
What We Do
PayPay, a FinTech company, that achieved more than 60M users within around 5 years since its launch in 2018 has hugely diversified employees who are from about 50 different countries. The number of employees by now is more than 1,000 staff members, and yet the company is still incomplete and at a growing stage.
To build "PayPay", we allied with Paytm, the biggest payment service company in India. Based on their customer-first technologies , we created and expanded the smartphone payment service in Japan.
Our biggest competitor is "cash". We are seeking for people who can accept this challenge positively, brush up the product at a tremendous speed that other companies could never achieve, and who are passionate about promoting and spreading such a financial life platform in a short time along with professionalism.
About our working style, we had introduced a new personnel system called WFA (Working From Anywhere*). At PayPay, we work fully remotely in Japan. Therefore, the working place or your living place doesn't matter. Even if you are living in a local area, the salary and benefit package will be at an international level while you enjoy working with world class colleagues. You will find such a way of working at PayPay! *This WFA policy is applicable only within Japan
