Lead Data Engineer

Posted 9 Days Ago
Be an Early Applicant
Bangalore, Bengaluru Urban, Karnataka, IND
In-Office
Senior level
HR Tech • Information Technology • Software
The Role
Own and evolve HackerRank's data platform (StarRocks, Apache Hudi, Trino, Spark, Ranger), build AI-optimized data layer for natural language querying, deliver in-product data features and self-serve pipelines, enforce data security, lead technical design reviews, and partner with PMs to scope AI-enabled use cases.
Summary Generated by Built In

HackerRank helps companies like NVIDIA, Amazon, and Microsoft hire and upskill the next generation of developers based on skills, not pedigree. Our platform is trusted by over 2,500 of the world’s most innovative companies to build strong engineering teams ready for what’s next.
Software has entered an era where humans and AI build side by side. As this shift accelerates, the definition of strong technical talent is changing. We give companies better ways to identify and invest in next-generation skills.
People at HackerRank care deeply about the impact of their work and sweat the small details so our customers can be wildly successful with products they genuinely love to use. We move with urgency and believe great outcomes come from high standards.

About the role

HackerRank's data platform is at an inflection point. We've completed a multi-year modernisation - migrating from Redshift to StarRocks + Apache Hudi - and cut export latencies from 25 seconds to under 5 seconds. The infrastructure groundwork is done. Now we're building the AI-native data layer that will power revenue-generating features like natural language querying for HackerRank for Work customers.

As Lead Data Engineer, you'll be a senior individual contributor at the heart of the data organisation - owning complex platform decisions, collaborating cross-functionally with AI, product, and go-to-market teams, and shipping data-driven features that directly drive revenue. This is a greenfield opportunity to shape the next phase of data at HackerRank.

What you will do
  • Own and evolve the data platform - StarRocks (OLAP), Apache Hudi (Data Lake), Trino, Spark, and Apache Ranger - ensuring performance, reliability, and security at scale.
  • Build the next-gen AI-optimised data layer: clean, structured datasets that power natural language querying and AI add-on features for HackerRank for Work customers.
  • Own in-product data features - exports, insights dashboards, interview analytics, and the self-serve Custom Reports interface.
  • Enable self-service pipelines for internal teams (AI platform, analytics, go-to-market), reducing ad-hoc data requests and scaling data access across the org.
  • Enforce robust data security - access controls, Apache Ranger policies, and confidence-scoring guardrails for AI-generated outputs.
  • Lead technical design reviews and define engineering standards for the data team.
  • Partner with PMs and business stakeholders to proactively identify and scope AI-enabled data use cases.
Who you are
  • 6+ years of data engineering experience, with at least 2 years in a senior or lead capacity.
  • Deep hands-on expertise with OLAP databases - StarRocks, ClickHouse, Druid, or similar.
  • Strong experience with data lake technologies - Apache Hudi, Iceberg, or Delta Lake.
  • Proficient with distributed query engines (Trino / Presto) and batch/streaming compute with Apache Spark.
  • Solid understanding of data security, RBAC, and access control tools like Apache Ranger.
  • Comfortable working in a hybrid AWS + open-source self-managed environment.
  • Strong communicator who can translate technical decisions for non-technical stakeholders and drive cross-functional projects independently.
Even better if you have
  • Hands-on experience with AI/LLM-adjacent data work - confidence scoring, agentic pipelines, RAG architectures, or vector stores.
  • Prior exposure to agentic workflows and understanding how to operationalise emerging AI concepts at production scale.
  • Experience scaling data infrastructure at a SaaS or B2B product company.
  • Familiarity with natural language querying interfaces or building data products for end-customer consumption.
You will thrive in this role if
  • You're energised by working on a platform that's both technically mature and still has enormous greenfield ahead of it.
  • You don't wait for a PM to hand you a roadmap - you proactively connect data capabilities to business outcomes.
  • You care as much about how other teams use data as you do about the pipelines that produce it.
  • You're genuinely curious about AI and want to be close to where data and intelligence intersect.
  • You thrive in lean, cross-functional environments where your decisions have visible, company-wide impact.

Want to learn more about HackerRank? Check out HackerRank.com to explore our products, solutions and resources, and dive into our story and mission here.

HackerRank is a proud equal employment opportunity and affirmative action employer. We provide equal opportunity to everyone for employment based on individual performance and qualification. We never discriminate based on race, religion, national origin, gender identity or expression, sexual orientation, age, marital, veteran, or disability status. All your information will be kept confidential according to EEO guidelines. 

Linkedin | X | Blog | Instagram | Life@HackerRank

Notice to prospective HackerRank job applicants:

  • Our Recruiters use @hackerrank.com email addresses.
  • We never ask for payment or credit check information to apply, interview, or work here.

Skills Required

  • 6+ years of data engineering experience, with at least 2 years in a senior or lead capacity.
  • Hands-on expertise with OLAP databases (StarRocks, ClickHouse, Druid, or similar).
  • Experience with data lake technologies (Apache Hudi, Iceberg, or Delta Lake).
  • Proficient with distributed query engines (Trino / Presto) and batch/streaming compute with Apache Spark.
  • Solid understanding of data security, RBAC, and access control tools like Apache Ranger.
  • Comfortable working in a hybrid AWS plus open-source self-managed environment.
  • Strong communication and cross-functional collaboration skills to translate technical decisions for non-technical stakeholders.
  • Hands-on experience with AI/LLM-adjacent data work (confidence scoring, agentic pipelines, RAG architectures, vector stores).
  • Experience scaling data infrastructure at a SaaS or B2B product company.
  • Familiarity with natural language querying interfaces or building data products for end-customer consumption.

HackerRank Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about HackerRank and has not been reviewed or approved by HackerRank.

  • Fair & Transparent Compensation Pay is considered competitive for many U.S. roles, with engineering and sales ranges aligning to market snapshots for base and total compensation. Overall sentiment positions compensation as above average relative to other categories.
  • Healthcare Strength Core coverage includes medical, dental, vision, mental-health/EAP, and life and disability insurance. Healthcare elements are consistently highlighted as solid within the package.
  • Flexible Benefits Flexible PTO, remote-work support, and home-office stipends are offered alongside equity and an annual learning budget. Flexibility in where and how work happens is emphasized across materials.

HackerRank Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Mountain View, CA
1,053 Employees
Year Founded: 2012

What We Do

HackerRank is a technology hiring platform that is the standard for assessing developer skills for over 2600 companies around the world. HackerRank helps companies hire skilled developers and innovate faster by enabling tech recruiters and hiring managers to objectively evaluate talent at every stage of the recruiting process.

Similar Jobs

In-Office
2 Locations
100000 Employees

Weekday, Inc. Logo Weekday, Inc.

Lead Data Engineer

Artificial Intelligence • HR Tech • Professional Services • Software
In-Office
4 Locations
1M-3M Annually

Weekday, Inc. Logo Weekday, Inc.

Lead Data Engineer

Artificial Intelligence • HR Tech • Professional Services • Software
In-Office
4 Locations
2M-3M Annually

Brillio Logo Brillio

Lead Data Engineer

Information Technology
Hybrid
Bangalore, Bengaluru Urban, Karnataka, IND
2676 Employees

Similar Companies Hiring

Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
178 Employees
Kepler  Thumbnail
Fintech • Software
New York, New York
6 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account