Lead Data Engineer

Posted 2 Days Ago
Be an Early Applicant
2 Locations
In-Office
Senior level
Information Technology • Consulting
The Role
Lead data engineering initiatives by designing governed ETL/ELT pipelines, developing Databricks lakehouse solutions, integrating REST APIs and enterprise data, and maintaining data quality. The role requires expertise in Python, PySpark, SQL, Spark, Unity Catalog, Delta Lake, CI/CD, and Git. Preferred experience includes Planisware, agentic AI frameworks, knowledge graphs, RAG, vector search, pipeline monitoring, and project portfolio management datasets.
Summary Generated by Built In
Required Skills & Experience
    • 7+ years of experience in Data Engineering, Data Platform Development or related technical roles.
    • Strong hands-on experience with Databricks.
    • Strong programming experience in Python and PySpark.
    • Advanced SQL skills, including complex queries, joins, aggregations and data transformations.
    • Strong experience with Apache Spark and distributed data processing.
    • Hands-on experience designing and developing ETL/ELT pipelines.
    • Experience with REST APIs and API-based data integrations.
    • Working knowledge of Databricks Unity Catalog, including data governance, access management and metadata.
    • Understanding of Delta Lake and lakehouse architecture.
    • Hands on experience in Databricks AI Genie / or any Agentic AI framework.
    • Working knowledge of CI/CD practices and Git-based development workflows.
    • Experience designing governed data pipelines and maintaining enterprise data quality.
Preferred Skills
    • Hands-on experience integrating enterprise data with Planisware.
    • Experience with Databricks Genie and Databricks-based AI/BI capabilities.
    • Experience with LangGraph, LangChain, Semantic Kernel, CrewAI or similar agentic frameworks.
    • Experience building knowledge graphs using Neo4j or other graph technologies.
    • Experience with RAG, vector search and enterprise data retrieval patterns.
    • Experience implementing automated data quality and pipeline monitoring.
    • Exposure to enterprise project portfolio management (PPM), resource management or financial planning datasets.

Skills Required

  • 7+ years of experience in data engineering, data platform development, or related technical roles
  • Strong hands-on experience with Databricks
  • Strong programming experience in Python and PySpark
  • Advanced SQL skills, including complex queries, joins, aggregations, and data transformations
  • Strong experience with Apache Spark and distributed data processing
  • Hands-on experience designing and developing ETL/ELT pipelines
  • Experience with REST APIs and API-based data integrations
  • Working knowledge of Databricks Unity Catalog, including data governance, access management, and metadata
  • Understanding of Delta Lake and lakehouse architecture
  • Hands-on experience with Databricks AI Genie or an agentic AI framework
  • Working knowledge of CI/CD practices and Git-based development workflows
  • Experience designing governed data pipelines and maintaining enterprise data quality
  • Hands-on experience integrating enterprise data with Planisware
  • Experience with Databricks Genie and Databricks-based AI/BI capabilities
  • Experience with LangGraph, LangChain, Semantic Kernel, CrewAI, or similar agentic frameworks
  • Experience building knowledge graphs using Neo4j or other graph technologies
  • Experience with RAG, vector search, and enterprise data retrieval patterns
  • Experience implementing automated data quality and pipeline monitoring
  • Exposure to enterprise project portfolio management, resource management, or financial planning datasets
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Vaughan, Ontario
345 Employees
Year Founded: 2007

What We Do

@TechBlocks we power the software defined industries (SDI) of today and tomorrow. We are a software engineering and consulting firm. We build modern digital value chains and businesses reimagined to create frictionless experiences for innovative monetization methods and drive unforeseen efficiencies. We are known to build world class custom platforms and products that are cloud native for some of the worlds largest brands. We are the go to technology partners for born in digital businesses that grew with us from "Concept to Commercialization" and have revenues between $100M - $10B. We help modern businesses transition just from a technology outsourcing mentality to help create globally distributed digital COEs and mature them. Our converged COEs that we create in partnership with our clients help power software factories that are extremely dynamic. We have created modern digital COEs and factories that are created with a single minded goal to future proof our clients businesses. Everything we do is centred around two philosophies and practices - Design Thinking and Lean Engineering. Whether it is building digital commerce platforms, marketplace for worlds largest retailers or smart utilities applications and products or digital health products/platforms that power wearables, patches or devices across healthcare landscape; we do it all with speed and sophistication that is unmatched in the industry

Similar Jobs

Shuru Logo Shuru

Lead Data Engineer

Information Technology
In-Office or Remote
Gurgaon, Haryana, IND
50 Employees

EXL Logo EXL

Lead Data Engineer

Information Technology • Database • Consulting
Hybrid
Gurugram, Haryana, IND
30246 Employees
In-Office
Gurugram, Haryana, IND
956 Employees

Stryker Logo Stryker

Lead Engineer, EDT Data & Analytics - Data Engineering

Healthtech • Other • Robotics • Biotech • Manufacturing
In-Office
Gurugram, Haryana, IND
51000 Employees

Similar Companies Hiring

NODA AI Thumbnail
Artificial Intelligence • Information Technology • Software • Cybersecurity • Defense • Defense Technology
Austin, TX
54 Employees
Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
567 Employees
RapidScale Thumbnail
Cloud • Information Technology • Consulting
Raleigh, NC
600 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account