Job Title: Big Data Engineer - SCala
About Us
“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery.
WHY JOIN CAPCO?
You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.
MAKE AN IMPACT
Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.
#BEYOURSELFATWORK
Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.
CAREER ADVANCEMENT
With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.
DIVERSITY & INCLUSION
We believe that diversity of people and perspective gives us a competitive advantage.
MAKE AN IMPACT
Job Description: Big Data Engineer – Scala
We are looking for an experienced Big Data Engineer with strong expertise in Scala and Apache Spark to design, develop, and optimize scalable data-processing solutions. The ideal candidate should have hands-on experience with distributed data platforms, large-scale ETL/data pipelines, and big data technologies.
Key Responsibilities- Design, develop, and maintain scalable Big Data applications and data pipelines.
- Develop high-performance data-processing solutions using Scala and Apache Spark.
- Build and optimize ETL/ELT pipelines for large volumes of structured and unstructured data.
- Work with technologies such as Hadoop, HDFS, Hive, Kafka, and Spark SQL.
- Perform Spark job performance tuning and optimization.
- Develop reusable and maintainable data-processing frameworks.
- Troubleshoot production issues related to data pipelines and distributed applications.
- Implement data-quality checks, monitoring, and error-handling mechanisms.
- Collaborate with data architects, analysts, business teams, and other engineering teams.
- Follow software engineering best practices, including Git, code reviews, testing, and CI/CD.
- Strong hands-on programming experience in Scala.
- Strong experience with Apache Spark / Spark SQL.
- Good understanding of Big Data and distributed computing concepts.
- Experience with Hadoop ecosystem technologies, including Hadoop/HDFS and Hive.
- Experience building large-scale ETL/data engineering pipelines.
- Strong SQL skills.
- Experience working with Kafka or similar messaging/streaming technologies.
- Understanding of data formats such as Parquet, Avro, and JSON.
- Experience with performance optimization and troubleshooting of Spark applications.
- Good understanding of software development lifecycle and version-control systems.
- Experience with AWS, Azure, or GCP.
- Knowledge of Databricks.
- Experience with Airflow or similar orchestration tools.
- Exposure to Delta Lake or other modern lakehouse technologies.
- Knowledge of Docker, Kubernetes, and CI/CD pipelines.
- Experience working in Agile/Scrum environments.
Scala | Apache Spark | Spark SQL | SQL | Hadoop | HDFS | Hive | Kafka | ETL | Data Engineering | Big Data
Skills Required
- Strong hands-on programming experience in Scala
- Strong experience with Apache Spark and Spark SQL
- Understanding of big data and distributed computing concepts
- Experience with Hadoop ecosystem technologies, including Hadoop, HDFS, and Hive
- Experience building large-scale ETL and data engineering pipelines
- Strong SQL skills
- Experience with Kafka or similar messaging and streaming technologies
- Understanding of Parquet, Avro, and JSON data formats
- Experience optimizing and troubleshooting Spark applications
- Understanding of software development lifecycle and version-control systems
- Experience with AWS, Azure, or GCP
- Knowledge of Databricks
- Experience with Airflow or similar orchestration tools
- Exposure to Delta Lake or other modern lakehouse technologies
- Knowledge of Docker, Kubernetes, and CI/CD pipelines
- Experience working in Agile/Scrum environments
What the Team is Saying
Capco Compensation & Benefits Highlights
-
Healthcare Strength — Health insurance is described as comprehensive with multiple plan options and generally low copays, and mental-health support and EAP resources reinforce this strength.
-
Leave & Time Off Breadth — Paid time off, holidays, and sick leave are often highlighted as generous, with flexible time-off arrangements supporting work-life balance.
-
Parental & Family Support — Parental leave is characterized as generous, and family-planning benefits such as fertility, adoption assistance, childcare support, and coverage for family members are emphasized.
Capco Insights
What We Do
Capco, a Wipro company, is a global management and technology consultancy specializing in driving transformation in the energy and financial services industries. Capco operates at the intersection of business and technology by combining innovative thinking with unrivalled industry knowledge to fast-track digital initiatives for banking and payments, capital markets, wealth and asset management, insurance, and the energy sector. Capco’s cutting edge ingenuity is brought to life through its award-winning Be Yourself At Work culture and diverse talent. TRUSTED ADVISORS - Our collaborative and entrepreneurial approach position us as trusted long-term partners to our clients. DEEP EXPERTISE - Clients look to us for clear guidance, proven expertise, and the support they need to set their companies up for success. DISRUPTIVE SOLUTIONS- We do not provide off -the-shelf answers to clients’ challenges. Instead, we work in close collaboration to understand the issues, and craft disruptive, bespoke solutions. ACCOUNTABILITY AND OWNERSHIP - We recognize every client has distinct needs and expectations, and we take ownership and accountability for the solutions we design, build, and deliver.
Why Work With Us
We support and encourage an inclusive culture, entrepreneurial outlook and independent thinking. Capco is not about organizational charts and rigid hierarchies – we want all employees to feel that Capco is their firm to own and run. We believe diversity powers innovation. Be Yourself At Work (#BYAW) is an integral and underlying part of our culture
Gallery
Capco Offices
Hybrid Workspace
Employees engage in a combination of remote and on-site work.
