Data Engineer – Python, PySpark, SQL & ETL

Posted 4 Days Ago
Be an Early Applicant
Bellandur, Bangalore, Karnataka, IND
In-Office
Senior level
Fintech • Financial Services
The Role
Designs, develops, optimizes, and supports scalable ETL pipelines, data marts, and distributed data-processing solutions using Python, PySpark, Spark, SQL, Hadoop, and Hive. Responsibilities include data modeling, validation, testing, UAT, deployment, production monitoring, troubleshooting, query optimization, CI/CD support, documentation, and data-quality improvement across enterprise environments.
Summary Generated by Built In

Job Summary

Synechron is seeking a Data Engineer with 5+ years of experience in building data marts, ETL pipelines, and large-scale data-processing solutions.

The role will focus on designing, developing, optimizing, and supporting scalable data pipelines using Python, PySpark, Spark, SQL, and data-warehousing technologies. The position will cover the end-to-end software development lifecycle, including development, testing, UAT support, production deployment, issue resolution, and post-production support.

The role will contribute to business objectives by delivering reliable and efficient data solutions, improving data quality and accessibility, supporting informed decision-making, and enabling consistent data operations across enterprise environments.


Software Requirements

Required Software Skills

  • Python: Strong hands-on experience developing data-processing applications, ETL workflows, automation scripts, and production services.
  • PySpark and Spark: Strong experience building distributed data-processing and ETL solutions.
  • Pandas: Experience with data preparation, analysis, transformation, validation, and profiling.
  • SQL: Strong experience with data extraction, transformation, validation, analysis, and query optimization.
  • Oracle SQL: Strong knowledge of Oracle SQL, complex queries, joins, aggregations, subqueries, stored procedures, and performance optimization.
  • ETL development: Experience designing, developing, testing, deploying, and supporting ETL and data-pipeline solutions.
  • Data warehousing: Strong understanding of data-warehouse concepts, dimensional modelling, data marts, transformations, fact tables, dimensions, and data quality.
  • Hadoop, Hive, and MapReduce: Experience working with distributed data-processing and big-data technologies.
  • SQL and NoSQL databases: Experience working with structured, semi-structured, and unstructured data.
  • Jupyter Notebook: Experience using notebooks for data analysis, development, validation, troubleshooting, and technical exploration.
  • CI/CD pipelines: Familiarity with automated build, testing, deployment, and release processes.
  • Testing and data validation: Experience validating data transformations, pipeline results, data quality, and application behavior.
  • Software engineering practices: Understanding of coding standards, version control, documentation, code reviews, testing, and maintainable development practices.

Preferred Software Skills

  • Experience with banking or financial-services data and reporting requirements.
  • Exposure to cloud-based data platforms and data-processing services.
  • Familiarity with workflow scheduling, orchestration, monitoring, and alerting tools.
  • Experience with automated data-quality checks, reconciliation, lineage, and metadata management.
  • Exposure to infrastructure automation, containerization, and modern DevOps practices.
  • Familiarity with tools supporting data pipeline observability and performance analysis.

Version and Experience Guidance

  • Specific software versions are not mandated in the requirements.
  • Candidates should have experience with current, production-supported versions of Python, PySpark, Spark, Pandas, Oracle SQL, Hadoop, Hive, MapReduce, and related tools.
  • Candidates should be able to assess compatibility, manage dependencies, support upgrades, and follow version-control practices.
  • Experience should include production data engineering, pipeline support, and end-to-end SDLC participation rather than only academic or prototype development.

Overall Responsibilities

  • Design, develop, maintain, and optimize scalable ETL and data-pipeline solutions using Python and PySpark.
  • Build and enhance data marts and data-transformation processes.
  • Develop distributed data-processing solutions using Spark, Hadoop, Hive, and MapReduce.
  • Work with structured, semi-structured, and unstructured data from multiple sources.
  • Develop and optimize Oracle SQL queries for data extraction, transformation, analysis, and validation.
  • Analyze and debug data issues using Oracle SQL, PySpark, Pandas, and structured data-analysis techniques.
  • Perform end-to-end SDLC activities, including development, testing, UAT support, bug fixing, production deployment, and post-production support.
  • Collaborate with cross-functional teams to define data requirements, validate solutions, resolve defects, and improve data quality.
  • Implement data validation, reconciliation, completeness, accuracy, and consistency checks.
  • Support CI/CD implementation for data pipelines, applications, and deployment workflows.
  • Monitor production environments and troubleshoot pipeline failures, data-quality issues, performance problems, and processing delays.
  • Create and maintain technical documentation, data mappings, transformation logic, support procedures, and operational runbooks.
  • Apply software engineering best practices, coding standards, testing practices, and maintainable design principles.
  • Contribute to performance improvements through query optimization, efficient transformations, appropriate partitioning, resource management, and pipeline tuning.
  • Identify opportunities to improve automation, reliability, data quality, maintainability, and sustainable use of processing resources.
  • Contribute to measurable outcomes, including reliable data delivery, reduced processing time, improved data accuracy, fewer production incidents, and successful release execution.

Technical Skills (By Category)


Programming Languages

Essential

  • Strong Python development experience.
  • Strong PySpark and Spark programming experience.
  • Experience using Pandas for data analysis, transformation, profiling, and validation.
  • Ability to write modular, maintainable, testable, and production-ready code.
  • Ability to develop reusable data-processing components, ETL logic, validation utilities, and automation scripts.

Preferred

  • Experience with additional programming or scripting languages used in data engineering and distributed processing.
  • Familiarity with asynchronous or event-driven data-processing patterns.
  • Experience developing reusable libraries and shared data-engineering components.

Databases/Data Management

Essential

  • Strong Oracle SQL experience.
  • Experience with SQL and NoSQL databases.
  • Understanding of data-warehouse concepts and data-mart design.
  • Experience working with structured, semi-structured, and unstructured data.
  • Understanding of data modelling, dimensional modelling, schemas, fact tables, dimensions, keys, joins, aggregations, and transformations.
  • Experience with data quality, validation, reconciliation, completeness, accuracy, and consistency checks.
  • Ability to optimize queries and investigate data-related application or pipeline issues.
  • Understanding of data lineage, source-to-target mapping, metadata, data retention, and controlled data access.

Preferred

  • Experience with banking or financial-services data structures and reporting requirements.
  • Familiarity with data lakes, analytical data stores, and distributed data architectures.
  • Experience with partitioning, indexing, compression, schema evolution, and high-volume data processing.
  • Exposure to automated data-quality frameworks, data reconciliation, and metadata management.

Cloud Technologies

Essential

  • Understanding of scalable data-processing and data-pipeline deployment environments.
  • Ability to support development, testing, UAT, staging, and production data environments.
  • Understanding of resource management, performance, availability, monitoring, and operational support for data platforms.

Preferred

  • Experience with cloud-based data platforms and distributed processing services.
  • Familiarity with cloud storage, managed data-processing services, data warehouses, and orchestration tools.
  • Exposure to cloud-native CI/CD, infrastructure automation, and containerized data workloads.
  • Understanding of processing optimization, resource utilization, and sustainable data-engineering practices.

Frameworks and Libraries

Essential

  • Python.
  • PySpark and Spark.
  • Pandas.
  • Oracle SQL.
  • Hadoop.
  • Hive.
  • MapReduce.
  • SQL and NoSQL database technologies.
  • Jupyter Notebook.
  • ETL and data-pipeline development frameworks.
  • CI/CD tools and automated deployment workflows.
  • Data validation, testing, monitoring, and troubleshooting utilities.

Preferred

  • Workflow orchestration and scheduling frameworks.
  • Data-quality, reconciliation, lineage, and metadata-management tools.
  • Pipeline observability, logging, metrics, and alerting frameworks.
  • Libraries supporting performance profiling, distributed processing, and automated testing.
  • Tools supporting cloud-based data integration and analytics.

Development Tools and Methodologies

Essential

  • Experience across the data-engineering lifecycle, including requirements analysis, design, development, testing, UAT support, deployment, monitoring, troubleshooting, and production support.
  • Experience with ETL development and data-mart implementation.
  • Familiarity with version control, code reviews, coding standards, technical documentation, and collaborative development practices.
  • Experience implementing data validation, unit testing, integration testing, regression testing, and pipeline testing.
  • Experience supporting CI/CD implementation for data applications and deployment workflows.
  • Ability to create data mappings, transformation specifications, test plans, support procedures, and operational runbooks.
  • Familiarity with Agile or iterative delivery practices.
  • Experience performing root-cause analysis and resolving production data-pipeline issues.

Preferred

  • Experience with automated data-pipeline deployment and release management.
  • Familiarity with infrastructure as code, containers, and DevOps practices.
  • Experience with pipeline monitoring, operational dashboards, alerting, and service-level reporting.
  • Exposure to automated rollback, recovery, retry, and failure-handling patterns.
  • Experience supporting continuous improvement and process standardization.

Security Protocols

Essential

  • Understanding of secure data access, authentication, authorization, and least-privilege principles.
  • Ability to protect sensitive business, customer, financial, and operational data.
  • Understanding of secure database connections, credential management, data masking, encryption, and controlled data handling.
  • Ability to follow security, privacy, compliance, change-management, and access-control requirements.
  • Awareness of risks related to unauthorized access, data leakage, insecure scripts, exposed credentials, and uncontrolled data movement.
  • Ability to maintain auditability and traceability across data transformations, deployments, and production changes.

Preferred

  • Experience supporting data-security reviews, audit activities, access reviews, and compliance controls.
  • Familiarity with secrets management, encryption in transit and at rest, and security monitoring.
  • Experience integrating security and quality checks into CI/CD pipelines.
  • Knowledge of security requirements for banking and financial-services data.

Experience Requirements

  • Overall experience: Minimum 5 years of professional experience in data engineering, ETL development, data warehousing, big-data engineering, or a related technical role.
  • Data-engineering experience: Strong hands-on experience building data marts, ETL pipelines, data transformations, and large-scale data-processing solutions.
  • Technical experience: Strong experience with Python, PySpark, Spark, Pandas, SQL, Oracle SQL, Hadoop, Hive, MapReduce, and SQL and NoSQL databases.
  • Data-warehouse experience: Experience with data-warehouse concepts, data modelling, data marts, source-to-target mapping, data transformations, and data-quality controls.
  • SDLC experience: Experience across development, UAT support, bug fixing, production deployment, monitoring, and post-production support.
  • Production experience: Experience troubleshooting data-pipeline failures, query issues, data-quality defects, performance problems, and production incidents.
  • Domain preference: Banking and financial-services experience is highly preferred.
  • Industry preference: Experience supporting enterprise-scale, high-volume, regulated, or business-critical data environments is preferred.
  • Alternative experience pathways: Equivalent combinations of relevant data-engineering experience, technical training, substantial pipeline implementations, open-source contributions, or demonstrated production outcomes may be considered.
  • Outcome evidence: Candidates should demonstrate contributions to reliable data delivery, improved pipeline performance, enhanced data quality, reduced processing time, successful production deployments, or reduced recurring support issues.

Day-to-Day Activities

  • Design, develop, test, and optimize Python, PySpark, Spark, SQL, and ETL solutions supporting data marts and large-scale data processing.
  • Analyze source data, debug Oracle SQL and PySpark issues, validate transformations, support UAT, resolve defects, and monitor production pipelines.
  • Collaborate with data, application, testing, operations, and business teams through planning sessions, requirement discussions, technical reviews, release meetings, and issue-resolution activities.
  • Deliver production-quality code, data mappings, test results, validation reports, deployment artifacts, technical documentation, and recommendations within the assigned scope.

Qualifications

  • Bachelor’s degree in Computer Science, Information Technology, Engineering, Data Science, or a related field; equivalent relevant experience may be considered.
  • Minimum 5 years of hands-on experience in data engineering, ETL development, data warehousing, big-data processing, or a related technical role.
  • Demonstrated experience with Python, PySpark, Spark, Pandas, Oracle SQL, ETL development, data marts, data warehousing, Hadoop, Hive, MapReduce, and SQL and NoSQL databases.
  • Experience with Jupyter Notebook, CI/CD pipelines, testing, data validation, SDLC activities, production deployment, and post-production support is required.
  • Commitment to continuous professional development through relevant training, technical experimentation, knowledge sharing, and ongoing learning in data engineering, distributed processing, cloud technologies, data quality, and automation.

Professional Competencies

  • Applies structured analytical thinking to investigate data-quality issues, pipeline failures, query performance concerns, processing errors, and production incidents.
  • Collaborates effectively with data, application, testing, operations, and business teams to deliver reliable data solutions and resolve issues.
  • Communicates technical findings, data limitations, defects, delivery status, risks, and recommendations clearly to technical and non-technical stakeholders.
  • Adapts to evolving data sources, processing technologies, business requirements, delivery priorities, and platform constraints through continuous learning.
  • Identifies practical opportunities to improve pipeline performance, data quality, automation, maintainability, reliability, resource efficiency, and sustainability.
  • Manages time and priorities effectively across development, testing, UAT support, production deployment, incident resolution, documentation, and continuous improvement.

S​YNECHRON’S DIVERSITY & INCLUSION STATEMENT
 

Diversity & Inclusion are fundamental to our culture, and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity, Equity, and Inclusion (DEI) initiative ‘Same Difference’ is committed to fostering an inclusive culture – promoting equality, diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger, successful businesses as a global company. We encourage applicants from across diverse backgrounds, race, ethnicities, religion, age, marital status, gender, sexual orientations, or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements, mentoring, internal mobility, learning and development programs, and more.

All employment decisions at Synechron are based on business needs, job requirements and individual qualifications, without regard to the applicant’s gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law.

Candidate Application Notice

Skills Required

  • Bachelor's degree in Computer Science, Information Technology, Engineering, Data Science, or a related field, or equivalent relevant experience
  • At least 5 years of professional experience in data engineering, ETL development, data warehousing, big-data processing, or a related technical role
  • Hands-on experience building data marts, ETL pipelines, data transformations, and large-scale data-processing solutions
  • Strong experience with Python, PySpark, Spark, Pandas, SQL, Oracle SQL, Hadoop, Hive, and MapReduce
  • Experience with SQL and NoSQL databases and structured, semi-structured, and unstructured data
  • Understanding of data warehousing, data modeling, dimensional modeling, source-to-target mapping, fact tables, dimensions, keys, joins, aggregations, and transformations
  • Experience with data quality, validation, reconciliation, completeness, accuracy, and consistency checks
  • Experience across development, testing, UAT support, bug fixing, production deployment, monitoring, and post-production support
  • Experience troubleshooting data-pipeline failures, query issues, data-quality defects, performance problems, and production incidents
  • Experience with Jupyter Notebook, CI/CD pipelines, automated deployment workflows, testing, and data validation
  • Familiarity with version control, coding standards, code reviews, technical documentation, and collaborative development practices
  • Understanding of secure data access, authentication, authorization, encryption, data masking, credential management, and least-privilege principles
  • Banking or financial-services data and reporting experience
  • Experience with cloud-based data platforms and distributed processing services
  • Familiarity with workflow scheduling, orchestration, monitoring, alerting, data lineage, metadata management, and automated data-quality tools
  • Exposure to infrastructure automation, containerization, DevOps practices, and cloud-native CI/CD
  • Familiarity with Agile or iterative delivery practices

Synechron Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Synechron and has not been reviewed or approved by Synechron.

  • Fair & Transparent Compensation — Pay is frequently characterized as competitive, particularly relative to large service-consulting peers and in certain in-demand skill areas. Compensation sentiment appears strongest when staffing is stable on strong client engagements and for market-aligned roles in major hubs.
  • Healthcare Strength — Healthcare coverage is often portrayed as a strong point in the U.S., with broad coverage and relatively favorable out-of-pocket experiences. Core medical, dental, and vision options are consistently described as meeting or exceeding a baseline expectation for consulting roles.
  • Equity Value & Accessibility — Equity was made broadly accessible through a company-wide RSU grant tied to a major revenue milestone. This is positioned as a notable upside even if it is framed as a one-time recognition event rather than an ongoing program.

Synechron Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: New York, New York
12,827 Employees
Year Founded: 2001

What We Do

At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron’s progressive technologies and optimization strategies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud & DevOps, Data, and Software Engineering, servicing an array of noteworthy financial services and technology firms. Through research and development initiatives in our FinLabs we develop solutions for modernization, from Artificial Intelligence and Blockchain to Data Science models, Digital Underwriting, mobile-first applications and more. Over the last 20+ years, our company has been honored with multiple employer awards, recognizing our commitment to our talented teams. With top clients to boast about, Synechron has a global workforce of 14,700+, and has 48 offices in 19 countries within key global markets. For more information on the company, please visit our website: www.synechron.com.

Similar Jobs

Micron Technology Logo Micron Technology

MTS, EDG System Architecture

Artificial Intelligence • Hardware • Information Technology • Machine Learning
In-Office
2 Locations
45000 Employees

Capital One Logo Capital One

Executive Assistant

Fintech • Machine Learning • Payments • Software • Financial Services
Hybrid
Bengaluru, Bengaluru Urban, Karnataka, IND
55000 Employees

Optum Logo Optum

Consultant

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office
Bengaluru, Bengaluru Urban, Karnataka, IND
160000 Employees
Hybrid
2 Locations
289097 Employees

Similar Companies Hiring

Hanover Park Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
42 Employees
Kepler  Thumbnail
Artificial Intelligence • Fintech • Software
New York, New York
9 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
85 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account