Role Overview
We are looking for an ETL QA Engineer with strong experience in
Databricks and PySpark, combined with hands-on exposure to AI-assisted
development and prompt engineering. The ideal candidate will be responsible for
validating large-scale data pipelines, ensuring data quality, and leveraging
tools like GitHub Copilot, GitHub Actions/Runners, and AI agents to improve
testing efficiency and coverage. You will also contribute to building and
validating AI-powered knowledge bases.
Required Skills & Experience
- Core
Technical Skills
- Strong hands-on experience in Databricks (notebooks, jobs, clusters, Delta Lake, SQL).
- Expert-level PySpark skills, including transformations, actions, window functions, and
optimization techniques.
- Solid understanding of ETL/ELT
concepts, data warehousing, and data modeling (star/snowflake
schemas, dimension/fact tables).
- Proven experience in data
validation, data quality checks, and reconciliation techniques.
- Practical experience with GitHub, GitHub Copilot, and GitHub Actions/Runners for CI/CD
pipelines.
- AI &
Prompt Engineering
- Experience crafting effective
prompts for large language models (LLMs) for coding, testing, and
documentation.
- Exposure to agent-based
architectures or tools (e.g., workflow/agent frameworks that
orchestrate multi-step AI tasks).
- Familiarity with AI-based
tools for code analysis, test generation, or knowledge management.
- Knowledge
Base & Documentation
- Experience building AI-driven
knowledge bases or documentation systems (e.g., using vector search,
embeddings, or LLM-based retrieval).
- Strong documentation skills,
with the ability to translate complex data/QA concepts into clear
knowledge articles.
- Testing & QA Practices
- Strong understanding of QA
methodologies: test planning, test design, defect management, and
regression testing.
- Experience with automated
testing frameworks (Python-based testing libraries such as pytest, unittest, or similar).
- Familiarity with data quality
tools/practices (e.g., validation rules, thresholds, anomaly detection).
Key Attributes
- Strong analytical and
problem-solving skills with a detail-oriented mindset.
- Passion for data quality,
automation, and continuous improvement.
- Ability to work in an agile,
fast-paced environment and collaborate across multiple teams like Business
and operations team and AD team.
- Curiosity and openness to adopting new AI tools and
practices for QA.
Skills Required
- Hands-on experience with Databricks, including notebooks, jobs, clusters, Delta Lake, and SQL
- Expert-level PySpark experience, including transformations, actions, window functions, and optimization
- Understanding of ETL/ELT concepts, data warehousing, and star/snowflake data modeling
- Experience with data validation, data quality checks, and reconciliation techniques
- Experience with GitHub, GitHub Copilot, and GitHub Actions/Runners for CI/CD
- Experience crafting prompts for large language models for coding, testing, and documentation
- Exposure to agent-based architectures or workflow and agent frameworks
- Familiarity with AI tools for code analysis, test generation, or knowledge management
- Experience building AI-driven knowledge bases or documentation systems using vector search, embeddings, or LLM retrieval
- Strong documentation skills for translating complex data and QA concepts into knowledge articles
- Understanding of QA methodologies, including test planning, test design, defect management, and regression testing
- Experience with automated Python testing frameworks such as pytest or unittest
- Familiarity with data quality practices including validation rules, thresholds, and anomaly detection
What We Do
Kumaran Systems is an IT Services Company with imprints in three countries. The past two decades has seen us provide our global clientele with high-end IT services that include migration support, system integration and infrastructure management solutions, providing one-stop-solution to all your IT demands. Our expertise and in-depth knowledge of businesses help us cater to a variety of industries. Our team strives to know your industry better, by observing current trends and the way it works, to tailor-make our solutions to your needs.With over 20 years of customer orientation, over 2000 engagements across Telecom, Education and Banking & Financial services spread across the globe, Kumaran stands as a key advisor to some of the largest Fortune 500 companies in their business driven technology enablement drive. Kumaran System's Customer Orientation is driven by a global delivery business model giving its customers to choose between an Onshore-Offshore mix. The delivery models enable multilevel touch points between the client, partner networks and Kumaran thereby enabling business driven customer sensitivity and agility. ENGAGE, EMERGE AND EXCEL Involving ourselves in a purposeful action, using our abilities to its maximum, helps us feel positive about ourselves. Such positivity gives raise to positive thinking and positive ideas. Ideas evolve into vision and set our aim! Our aspirations engage us with the incentive of sweet success! Every endeavor has its own pitfalls. Perseverance helps us emerge a successful entrepreneur after any deterrent. In our attempt to achieve our aim, we keep upgrading our knowledge and skill, to surface successful! A genius mind does not work to excel, but excels in work! We challenge ourselves with every next step! We change every deterrent to a stepping stone to success. We essentially, engage, emerge and excel
