Data Engineer/Developer
Role Summary
Design, build, and optimize Azure data pipelines and lakehouse solutions.
Deliver secure, reliable datasets with strong governance, automation, and
documentation. Collaborate across teams and contribute to standards in an Agile
setting.
Must-Have (Day 1)
- Experience:
4–6 years in data engineering
- Core
Platform: Databricks with Python, Spark, Pandas (notebooks and modular
code)
- Orchestration:
Azure Data Factory (pipelines, integration runtimes); ingest from diverse
sources
- Lakehouse:
Delta Lake fundamentals; Medallion architecture (bronze/silver/gold) in
production
- Storage/SQL/Performance:
Azure Data Lake Storage (ADLS); strong SQL; performance-aware design
- Data
Patterns: ETL/ELT; data modeling (e.g., dimensional/star schema)
- DevOps
& Security: CI/CD for data projects (Azure DevOps or GitHub
Enterprise); familiarity with Azure Entra ID for SSO/RBAC; secure
workspace/data access
- Quality
& Observability: Data validation/testing, code reviews, and basic
monitoring/alerting for jobs/pipelines
- Ways of
Working: Agile/Scrum (Jira/Confluence); clear pipeline and data contract
documentation
- Collaboration:
Effective stakeholder engagement; support/mentor junior team members;
clear communication
- Generative
AI (Day 1):
- Prompt
design for data tasks (ingestion, transformations, documentation) with
clear objectives and constraints
- Use of
Copilot/ChatGPT to scaffold notebooks/jobs, generate tests, and optimize
SQL/Spark—validates outputs before merging
Nice-to-Have (Train within 60–90 days)
- Unity
Catalog migration (Hive to Unity) and permissions/governance
- Databricks
DevOps (cluster configuration, secret management, workspace automation)
- Azure
Functions (C# or Python) for orchestration/integration
- Synapse
dedicated SQL pools or dbt; Delta Live Tables
- Financial
services domain exposure
Shared Expectations
- Work
independently with minimal supervision while contributing to team outcomes
- Commitment
to secure practices and production-grade reliability
- Continuous
improvement mindset and willingness to learn new tools/technologies
- Willingness
to work within regulated environment controls and policies
- Use
Generative AI responsibly to improve velocity and quality (simple,
structured prompts; guardrails; validate AI-assisted outputs before
adoption)
Skills Required
- 4-6 years of experience in data engineering
- Databricks experience with Python, Spark, and Pandas
- Azure Data Factory experience, including pipelines and integration runtimes
- Experience ingesting data from diverse sources
- Delta Lake fundamentals and production experience with Medallion architecture
- Azure Data Lake Storage experience
- Strong SQL skills and performance-aware design experience
- ETL and ELT experience
- Data modeling experience, such as dimensional or star schema modeling
- CI/CD experience for data projects using Azure DevOps or GitHub Enterprise
- Familiarity with Azure Entra ID for SSO and RBAC
- Experience with secure workspace and data access practices
- Data validation and testing experience
- Code review experience
- Basic monitoring and alerting for jobs and pipelines
- Agile/Scrum experience with Jira and Confluence
- Ability to document pipelines and data contracts clearly
- Effective stakeholder engagement and communication
- Ability to support or mentor junior team members
- Prompt design for data ingestion, transformations, and documentation
- Experience using Copilot or ChatGPT to scaffold notebooks/jobs, generate tests, or optimize SQL/Spark
- Ability to validate Generative AI-assisted outputs before merging or adoption
- Unity Catalog migration and permissions/governance experience
- Databricks DevOps experience, including cluster configuration, secret management, and workspace automation
- Azure Functions experience using C# or Python
- Synapse dedicated SQL pools, dbt, or Delta Live Tables experience
- Financial services domain exposure
- Ability to work independently with minimal supervision
- Commitment to secure practices and production-grade reliability
- Willingness to work within regulated environment controls and policies
What We Do
Kumaran Systems is an IT Services Company with imprints in three countries. The past two decades has seen us provide our global clientele with high-end IT services that include migration support, system integration and infrastructure management solutions, providing one-stop-solution to all your IT demands. Our expertise and in-depth knowledge of businesses help us cater to a variety of industries. Our team strives to know your industry better, by observing current trends and the way it works, to tailor-make our solutions to your needs.With over 20 years of customer orientation, over 2000 engagements across Telecom, Education and Banking & Financial services spread across the globe, Kumaran stands as a key advisor to some of the largest Fortune 500 companies in their business driven technology enablement drive. Kumaran System's Customer Orientation is driven by a global delivery business model giving its customers to choose between an Onshore-Offshore mix. The delivery models enable multilevel touch points between the client, partner networks and Kumaran thereby enabling business driven customer sensitivity and agility. ENGAGE, EMERGE AND EXCEL Involving ourselves in a purposeful action, using our abilities to its maximum, helps us feel positive about ourselves. Such positivity gives raise to positive thinking and positive ideas. Ideas evolve into vision and set our aim! Our aspirations engage us with the incentive of sweet success! Every endeavor has its own pitfalls. Perseverance helps us emerge a successful entrepreneur after any deterrent. In our attempt to achieve our aim, we keep upgrading our knowledge and skill, to surface successful! A genius mind does not work to excel, but excels in work! We challenge ourselves with every next step! We change every deterrent to a stepping stone to success. We essentially, engage, emerge and excel
