We are seeking an experienced Senior Databricks Developer with 5+ years of experience in designing and implementing scalable data engineering solutions. The ideal candidate will have strong expertise in big data processing, data pipeline development, and cloud-based data platforms, with hands-on experience using Databricks and distributed data processing frameworks like Apache Spark.
The role involves building robust data pipelines, optimizing large-scale data processing workloads, and collaborating with data engineers, analysts, and business stakeholders to enable data-driven decision-making.
Design, develop, and maintain scalable data pipelines using Databricks.
Build and optimize large-scale data processing solutions using Apache Spark.
Develop and manage ETL/ELT workflows for ingesting, transforming, and processing structured and unstructured data.
Write efficient data processing scripts using Python, Scala, and SQL.
Implement and manage data lake and data warehouse architectures.
Optimize performance of data pipelines and Spark jobs.
Integrate Databricks with cloud data services and storage platforms.
Collaborate with data scientists, analysts, and engineering teams to deliver reliable data solutions.
Implement best practices for data governance, data quality, and security.
Troubleshoot data pipeline issues and ensure system reliability and scalability.
Document data architecture, workflows, and operational processes.
5+ years of experience in data engineering or big data development.
Strong hands-on experience with Databricks.
Extensive experience with distributed data processing using Apache Spark.
Proficiency in Python, Scala, and SQL.
Strong experience building scalable ETL/ELT data pipelines.
Experience working with large datasets in data lake or data warehouse environments.
Experience with cloud platforms such as Microsoft Azure, Amazon Web Services, or Google Cloud Platform.
Strong knowledge of data modeling, data architecture, and data integration concepts.
Excellent analytical, troubleshooting, and performance optimization skills.
Skills Required
- 5+ years of experience in data engineering or big data development
- Hands-on experience with Databricks
- Extensive experience with distributed data processing using Apache Spark
- Proficiency in Python, Scala, and SQL
- Experience building scalable ETL/ELT data pipelines
- Experience working with large datasets in data lake or data warehouse environments
- Experience with Microsoft Azure, Amazon Web Services, or Google Cloud Platform
- Knowledge of data modeling, data architecture, and data integration concepts
- Strong analytical, troubleshooting, and performance optimization skills
What We Do
Kumaran Systems is an IT Services Company with imprints in three countries. The past two decades has seen us provide our global clientele with high-end IT services that include migration support, system integration and infrastructure management solutions, providing one-stop-solution to all your IT demands. Our expertise and in-depth knowledge of businesses help us cater to a variety of industries. Our team strives to know your industry better, by observing current trends and the way it works, to tailor-make our solutions to your needs.With over 20 years of customer orientation, over 2000 engagements across Telecom, Education and Banking & Financial services spread across the globe, Kumaran stands as a key advisor to some of the largest Fortune 500 companies in their business driven technology enablement drive. Kumaran System's Customer Orientation is driven by a global delivery business model giving its customers to choose between an Onshore-Offshore mix. The delivery models enable multilevel touch points between the client, partner networks and Kumaran thereby enabling business driven customer sensitivity and agility. ENGAGE, EMERGE AND EXCEL Involving ourselves in a purposeful action, using our abilities to its maximum, helps us feel positive about ourselves. Such positivity gives raise to positive thinking and positive ideas. Ideas evolve into vision and set our aim! Our aspirations engage us with the incentive of sweet success! Every endeavor has its own pitfalls. Perseverance helps us emerge a successful entrepreneur after any deterrent. In our attempt to achieve our aim, we keep upgrading our knowledge and skill, to surface successful! A genius mind does not work to excel, but excels in work! We challenge ourselves with every next step! We change every deterrent to a stepping stone to success. We essentially, engage, emerge and excel
