Data Engineer

Posted 15 Days Ago
Be an Early Applicant
Hyderābād, Telangāna, IND
In-Office
Senior level
Information Technology • Consulting
The Role
Designs, builds, and optimizes Azure data pipelines and lakehouse solutions using Databricks, Python, Spark, Azure Data Factory, Delta Lake, and ADLS. Develops secure, governed, reliable datasets; applies ETL/ELT, dimensional modeling, testing, monitoring, CI/CD, and performance optimization. Collaborates in Agile teams, documents data contracts and pipelines, mentors junior staff, and responsibly uses generative AI to accelerate development while validating outputs.
Summary Generated by Built In

Data Engineer/Developer

Role Summary
Design, build, and optimize Azure data pipelines and lakehouse solutions. Deliver secure, reliable datasets with strong governance, automation, and documentation. Collaborate across teams and contribute to standards in an Agile setting.

Must-Have (Day 1)

  • Experience: 4–6 years in data engineering
  • Core Platform: Databricks with Python, Spark, Pandas (notebooks and modular code)
  • Orchestration: Azure Data Factory (pipelines, integration runtimes); ingest from diverse sources
  • Lakehouse: Delta Lake fundamentals; Medallion architecture (bronze/silver/gold) in production
  • Storage/SQL/Performance: Azure Data Lake Storage (ADLS); strong SQL; performance-aware design
  • Data Patterns: ETL/ELT; data modeling (e.g., dimensional/star schema)
  • DevOps & Security: CI/CD for data projects (Azure DevOps or GitHub Enterprise); familiarity with Azure Entra ID for SSO/RBAC; secure workspace/data access
  • Quality & Observability: Data validation/testing, code reviews, and basic monitoring/alerting for jobs/pipelines
  • Ways of Working: Agile/Scrum (Jira/Confluence); clear pipeline and data contract documentation
  • Collaboration: Effective stakeholder engagement; support/mentor junior team members; clear communication
  • Generative AI (Day 1):
    • Prompt design for data tasks (ingestion, transformations, documentation) with clear objectives and constraints
    • Use of Copilot/ChatGPT to scaffold notebooks/jobs, generate tests, and optimize SQL/Spark—validates outputs before merging

Nice-to-Have (Train within 60–90 days)

  • Unity Catalog migration (Hive to Unity) and permissions/governance
  • Databricks DevOps (cluster configuration, secret management, workspace automation)
  • Azure Functions (C# or Python) for orchestration/integration
  • Synapse dedicated SQL pools or dbt; Delta Live Tables
  • Financial services domain exposure

Shared Expectations

  • Work independently with minimal supervision while contributing to team outcomes
  • Commitment to secure practices and production-grade reliability
  • Continuous improvement mindset and willingness to learn new tools/technologies
  • Willingness to work within regulated environment controls and policies
  • Use Generative AI responsibly to improve velocity and quality (simple, structured prompts; guardrails; validate AI-assisted outputs before adoption)


Skills Required

  • 4-6 years of experience in data engineering
  • Production experience with Databricks, Python, Spark, and Pandas
  • Experience building Azure Data Factory pipelines and integration runtimes
  • Experience ingesting data from diverse sources
  • Production experience with Delta Lake and Medallion architecture
  • Experience with Azure Data Lake Storage and strong SQL
  • Experience with performance-aware data design
  • Experience with ETL/ELT and dimensional or star-schema data modeling
  • Experience implementing CI/CD for data projects using Azure DevOps or GitHub Enterprise
  • Familiarity with Microsoft Entra ID, SSO/RBAC, and secure workspace and data access
  • Experience with data validation, testing, code reviews, and basic monitoring and alerting
  • Experience working in Agile/Scrum environments, including Jira or Confluence
  • Ability to document pipelines and data contracts clearly
  • Effective stakeholder engagement and communication skills
  • Ability to support or mentor junior team members
  • Prompt design for data ingestion, transformation, and documentation tasks
  • Experience using Copilot or ChatGPT to scaffold notebooks or jobs, generate tests, and optimize SQL or Spark, with output validation
  • Unity Catalog migration and permissions or governance experience
  • Databricks DevOps experience, including cluster configuration, secret management, and workspace automation
  • Azure Functions experience using C# or Python
  • Experience with Synapse dedicated SQL pools, dbt, or Delta Live Tables
  • Financial services domain experience
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Toronto, ON
1,078 Employees
Year Founded: 1990

What We Do

Kumaran Systems is an IT Services Company with imprints in three countries. The past two decades has seen us provide our global clientele with high-end IT services that include migration support, system integration and infrastructure management solutions, providing one-stop-solution to all your IT demands. Our expertise and in-depth knowledge of businesses help us cater to a variety of industries. Our team strives to know your industry better, by observing current trends and the way it works, to tailor-make our solutions to your needs.With over 20 years of customer orientation, over 2000 engagements across Telecom, Education and Banking & Financial services spread across the globe, Kumaran stands as a key advisor to some of the largest Fortune 500 companies in their business driven technology enablement drive. Kumaran System's Customer Orientation is driven by a global delivery business model giving its customers to choose between an Onshore-Offshore mix. The delivery models enable multilevel touch points between the client, partner networks and Kumaran thereby enabling business driven customer sensitivity and agility. ENGAGE, EMERGE AND EXCEL Involving ourselves in a purposeful action, using our abilities to its maximum, helps us feel positive about ourselves. Such positivity gives raise to positive thinking and positive ideas. Ideas evolve into vision and set our aim! Our aspirations engage us with the incentive of sweet success! Every endeavor has its own pitfalls. Perseverance helps us emerge a successful entrepreneur after any deterrent. In our attempt to achieve our aim, we keep upgrading our knowledge and skill, to surface successful! A genius mind does not work to excel, but excels in work! We challenge ourselves with every next step! We change every deterrent to a stepping stone to success. We essentially, engage, emerge and excel

Similar Jobs

Zensar Technologies Logo Zensar Technologies

Data Engineer

Information Technology
In-Office or Remote
2 Locations
10000 Employees

Zensar Technologies Logo Zensar Technologies

Data Engineer

Information Technology
In-Office or Remote
2 Locations
10000 Employees

Akamai Technologies Logo Akamai Technologies

Data Engineer

Cloud • Security • Software • Cybersecurity
In-Office or Remote
2 Locations
10285 Employees

Akamai Technologies Logo Akamai Technologies

Data Engineer

Cloud • Security • Software • Cybersecurity
In-Office or Remote
2 Locations
10285 Employees

Similar Companies Hiring

NODA AI Thumbnail
Artificial Intelligence • Information Technology • Software • Cybersecurity • Defense • Defense Technology
Austin, TX
54 Employees
Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
567 Employees
RapidScale Thumbnail
Cloud • Information Technology • Consulting
Raleigh, NC
600 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account