The Role
Designs Databricks data ingestion pipelines using PySpark and PySQL, develops and deploys Databricks Apps, and configures permissions, tokens, and service principals. Builds AI applications with FastAPI, React, Claude APIs, prompt engineering, vision processing, retrieval augmented generation, guardrails, graph analysis, and caching. Performs scenario-based testing, translates user requirements into code changes, and integrates custom notifications through APIs and webhooks.
Summary Generated by Built In
Databricks
- Design and build pipelines for ingesting data into Databricks from various sources like SAP, website scraping. Core technical skill will be PySpark and PySQL
- Write and do scenario-based testing, edge cases and discuss with the stake holders to finalize the necessary code changes and acceptance criteria.
- Understanding of the Databricks Unity Catalog permissions model and how to use OBO tokens and configure Service Principals for Databricks Agents,Genie Spaces, Machine Learning and Foundation Models.
- Knowledge of Databricks Apps. Ability to build and Deploy Databricks Apps using Visual Studio Code.
- Sending custom notifications from Databricks using APIs to custom Team Channel Webhooks
AI:
- Ability to use FastAPI /React front end and build minimal Single Page Applications(SPAs) using Chat UI interfaces and Claude Models/APIs
- Ability to use defined System Prompts, User Prompts etc..(i..e,) skills in prompt engineering.
- Basic Knowledge of Token Economics and ability use rule enforcement in prompts in addition to standard NLP language.
- Ability to reengineer code and see what it does and map it to the requirement specifications from users
- Ability to use Graphs to detect inter dependencies between rules and circular references.
- Ability to use both Claude Vision API in addition to standard text processing.
- Understand and develop Retrieval Augmented Generation specifically to Databricks like AI Search indexes, Hierarchical Chunking etc..
- Know how to implement basic guard rails related to AI safety
- Understand various caching mechanisms
Tools:
- Ability to work with Claude Code and Genie.
Skills Required
- Experience designing and building data ingestion pipelines in Databricks from sources such as SAP and website scraping
- Proficiency with PySpark and PySQL
- Experience with scenario-based testing, edge-case testing, stakeholder discussions, code changes, and acceptance criteria
- Knowledge of Databricks Unity Catalog permissions, OBO tokens, and Service Principal configuration
- Experience configuring Databricks Agents, Genie Spaces, Machine Learning, and Foundation Models
- Ability to build and deploy Databricks Apps using Visual Studio Code
- Experience sending custom Databricks notifications through APIs and Team Channel Webhooks
- Ability to build minimal FastAPI and React single-page chat applications
- Experience working with Claude models and APIs, including Claude Vision API
- Knowledge of system prompts, user prompts, prompt engineering, token economics, and prompt-based rule enforcement
- Ability to reengineer code and map its behavior to user requirements and specifications
- Ability to use graphs to identify rule dependencies and circular references
- Experience developing Retrieval Augmented Generation solutions using Databricks AI Search indexes and hierarchical chunking
- Knowledge of AI safety guardrails and caching mechanisms
- Ability to work with Claude Code and Genie
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Enable Data specializes in advanced data, application and cloud solutions working on cutting-edge projects driving innovation and transformation for our customers. We empower our customers to leverage modern solutions to deliver increased value across their business ecosystem. We help clients by providing consulting, managed project and staff augmentation services working with clients across various industries including healthcare, financial, media, insurance, and manufacturing.
.jpg)
