Databricks Engineer – Data Engineering (AWS) will build build scalable data pipelines and contribute to modern data platform development using AWS and Lakehouse architecture.
The candidate has solid hands-on Databricks experience and showed good engagement, passion, and familiarity with batch/streaming concepts, clustering, vacuuming, and cost-modeling considerations.
However, we do not recommend moving forward. The main concern is that the candidate frequently gave lengthy, indirect answers that did not address the specific questions asked, even after redirection. There were also notable gaps in modern data engineering practices, especially around CI/CD, unit testing, security scanning, and practical use of AI in development workflows.
Given the architect-level expectations for this role, the candidate may struggle to operate independently, provide clear guidance, and drive engineering standards effectively.
Skills Required
- Hands-on Databricks experience
- Experience building scalable data pipelines on AWS Lakehouse architectures
- Familiarity with batch and streaming data concepts
- Experience with Apache Spark and Delta Lake operations (clustering, vacuuming)
- Cluster cost modeling and optimization experience
- Experience implementing CI/CD for data engineering workflows
- Experience writing and maintaining unit tests for data pipelines
- Experience with security scanning and pipeline security practices
- Practical use of AI in development workflows
- Ability to operate at architect level, provide clear guidance, and drive engineering standards
emids Compensation & Benefits Highlights
The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about emids and has not been reviewed or approved by emids.
-
Fair & Transparent Compensation — Pay is considered market‑matched in certain U.S. roles, with experienced technical and leadership tracks described as competitive.
-
Leave & Time Off Breadth — PTO, paid holidays and sick time, and sabbatical leave are highlighted as part of the standard package.
-
Flexible Benefits — A remote‑work program and flexibility options are referenced as part of the offering.
emids Insights
What We Do
emids is healthcare's digital transformation leader, delivering business and tech solutions that help payers, providers and tech-enablers maximize technology to deliver care better. As a global partner headquartered in Nashville, TN, emids helps bridge the critical gaps in accessible, affordable, high-quality healthcare by providing advisory consulting services, custom application development, and data solutions. Services include EHR application deployment and management, analytics, data integration and governance, software development and testing, and business intelligence.
