Databricks Data Engineer

Posted Yesterday
Be an Early Applicant
Hiring Remotely in USA
Remote
113K-188K Annually
Mid level
Consulting
The Role
Design, build, and maintain scalable batch and streaming data pipelines on Databricks using PySpark, Delta Lake, Unity Catalog, and cloud platforms. Troubleshoot performance and quality issues, implement data governance, collaborate with cross-functional teams, and document pipelines and data models to deliver reliable, production-ready data products.
Summary Generated by Built In

Job Family:

Data Science & Analysis, Data Science Consulting


Travel Required:

None


Clearance Required:

None

Guidehouse is seeking a Databricks Data Engineer to join our AI & Data team to support client projects involving large-scale data pipelines, data transformation, platform modernization, and advanced analytics and AI solution delivery. This role focuses on designing, building, and maintaining scalable data solutions using Databricks, Spark, SQL, Python, Delta Lake, and related cloud-native technologies. The role requires hands-on technical expertise, strong problem-solving skills, and the ability to collaborate with clients and cross-functional teams to deliver reliable, secure, and well-documented data engineering solutions.

This position offers virtual work flexibility within the United States. While remote candidates will be considered, preference will be given to candidates located near a Guidehouse office in one of the following markets: Arlington, VA; Washington, DC; New York, NY; Chicago, IL; Austin, TX; Atlanta, GA; Boston, MA; and Boulder, CO.

Please note, this requisition supports hiring across multiple levels to support our Databricks Data Engineer Teams. The posted salary range represents a range of potential compensation and will vary based on the selected candidate’s experience, qualifications, location, and the level at which the position is filled.

What You Will Do: 

  • Design, build, and maintain scalable data pipelines using Databricks, PySpark, SQL, Delta Lake, and related cloud-native data engineering tools.

  • Develop and support batch and streaming data workflows, including ingestion, transformation, validation, and publishing of curated data products.

  • Write, optimize, and maintain Python, PySpark, and SQL code for data processing, orchestration, and performance tuning.

  • Work with large-scale datasets using Databricks, Spark, Delta Lake, Unity Catalog, and cloud storage services.

  • Troubleshoot and resolve data pipeline failures, performance issues, data quality issues, and workflow bottlenecks.

  • Translate business and technical requirements into data engineering designs, processing scripts, and job orchestration workflows.

  • Perform data validation, quality checks, code reviews, and issue resolution to support accurate and reliable data products.

  • Collaborate with cross-functional teams including solution architects, data scientists, analysts, DevOps engineers, and client stakeholders.

  • Communicate technical concepts, delivery impacts, and data engineering considerations to technical and non-technical audiences.

  • Document pipelines, datasets, data models, workflows, and engineering decisions to support maintainability, transparency, and reuse.

  • Follow data governance, security, lineage, and compliance standards within the Databricks platform.

What You Will Need: 

  • Bachelor’s degree in computer science, engineering, mathematics, statistics, or another relevant field.

  • 3-8 years of relevant experience in data engineering, data architecture, or cloud data platform implementation.

  • Strong experience with Python, PySpark, and SQL for data transformation, pipeline development, and data processing.

  • Experience with Databricks, Spark, Delta Lake, Unity Catalog, or similar cloud-native data platforms.

  • Experience developing batch or streaming data pipelines, ETL/ELT workflows, and reusable data assets.

  • Experience with data modeling, data warehousing, data quality validation, and large-scale data processing concepts.

  • Ability to troubleshoot technical issues, communicate engineering recommendations clearly, and work effectively in team-based delivery environments.

  • Experience applying data governance, access control, lineage, and security practices within cloud data platforms.

What Would Be Nice to Have: 

  • 2+ years of hands-on experience with the Databricks platform.

  • Experience using Unity Catalog for data governance, schema design, data modeling standards, access controls, lineage, and secure management of enterprise data assets.

  • Experience designing and building standard medallion data architectures for scalable ingestion, transformation, quality validation, and curated data delivery.

  • Active Databricks Data Engineer Associate, Databricks Data Engineer Professional, or related certification.

  • Experience with CI/CD tools and practices for notebooks, workflows, orchestration jobs, and infrastructure deployment.

  • Experience with cloud platforms such as Azure, AWS, or GCP.

  • Experience working in project-based or consulting delivery environments.

  • Familiarity with streaming frameworks, orchestration tools, Terraform, or related data platform administration capabilities.

The annual salary range for this position is $113,000.00-$188,000.00. Compensation decisions depend on a wide range of factors, including but not limited to skill sets, experience and training, security clearances, licensure and certifications, and other business and organizational needs.

What We Offer:

Guidehouse offers a comprehensive, total rewards package that includes competitive compensation and a flexible benefits package that reflects our commitment to creating a diverse and supportive workplace.

About Guidehouse

Guidehouse is an Equal Opportunity Employer–Protected Veterans, Individuals with Disabilities or any other basis protected by law, ordinance, or regulation.

Guidehouse will consider for employment qualified applicants with criminal histories in a manner consistent with the requirements of applicable law or ordinance including the Fair Chance Ordinance of Los Angeles and San Francisco.

If you have visited our website for information about employment opportunities, or to apply for a position, and you require an accommodation, please contact Guidehouse Recruiting at 1-571-633-1711 or via email at [email protected]. All information you provide will be kept confidential and will be used only to the extent required to provide needed reasonable accommodation.

All communication regarding recruitment for a Guidehouse position will be sent from Guidehouse email domains including @guidehouse.com or [email protected].  Correspondence received by an applicant from any other domain should be considered unauthorized and will not be honored by Guidehouse.  Note that Guidehouse will never charge a fee or require a money transfer at any stage of the recruitment process and does not collect fees from educational institutions for participation in a recruitment event. Never provide your banking information to a third party purporting to need that information to proceed in the hiring process.

If any person or organization demands money related to a job opportunity with Guidehouse, please report the matter to Guidehouse’s Ethics Hotline. If you want to check the validity of correspondence you have received, please contact [email protected]. Guidehouse is not responsible for losses incurred (monetary or otherwise) from an applicant’s dealings with unauthorized third parties.

Guidehouse does not accept unsolicited resumes through or from search firms or staffing agencies. All unsolicited resumes will be considered the property of Guidehouse and Guidehouse will not be obligated to pay a placement fee.

Skills Required

  • Bachelor's degree in computer science, engineering, mathematics, statistics, or relevant field.
  • 3-8 years relevant experience in data engineering, data architecture, or cloud data platform implementation.
  • Strong experience with Python, PySpark, and SQL for data transformation and pipeline development.
  • Experience with Databricks and Spark.
  • Experience with Delta Lake and Unity Catalog or similar cloud-native data platforms.
  • Experience developing batch or streaming data pipelines, ETL/ELT workflows, and reusable data assets.
  • Experience with data modeling, data warehousing, data quality validation, and large-scale data processing concepts.
  • Ability to troubleshoot technical issues, communicate recommendations clearly, and work effectively in team-based delivery environments.
  • Experience applying data governance, access control, lineage, and security practices within cloud data platforms.

Guidehouse Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Guidehouse and has not been reviewed or approved by Guidehouse.

  • Healthcare Strength Feedback suggests the benefits package includes broad medical, dental, vision, prescription, life, and disability coverage, which is often seen as a strong foundational offering. Access to HSA/FSA options further supports day-to-day healthcare and dependent-care needs.
  • Parental & Family Support Feedback suggests parental leave and adoption assistance are available, alongside an emergency back-up childcare program. These offerings indicate meaningful support for employees managing family responsibilities.
  • Wellbeing & Lifestyle Benefits Feedback suggests flexible work options and counseling/EAP-style support are part of the broader rewards mix. Additional lifestyle-oriented perks like community events and referral programs are also described as available in some contexts.

Guidehouse Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: McLean, VA
12,000 Employees
Year Founded: 2018

What We Do

Guidehouse is a leading global provider of consulting services to the public sector and commercial markets, with broad capabilities in management, technology, and risk consulting. By combining our public and private sector expertise, we help clients address their most complex challenges and navigate significant regulatory pressures focusing on transformational change, business resiliency, and technology-driven innovation. Across a range of advisory, consulting, outsourcing, and digital services, we create scalable, innovative solutions that help our clients outwit complexity and position them for future growth and success. The company has more than 12,000 professionals in over 50 locations globally. Guidehouse is a Veritas Capital portfolio company, led by seasoned professionals with proven and diverse expertise in traditional and emerging technologies, markets, and agenda-setting issues driving national and global economies.

Similar Jobs

EXL Logo EXL

Data Engineer

Information Technology • Database • Consulting
Remote or Hybrid
United States
30246 Employees
65K-87K Annually

Sparq Logo Sparq

Senior Data Engineer

Artificial Intelligence • Information Technology • Software
Remote
United States
723 Employees

Velera Logo Velera

Data Engineer

Fintech • Payments • Financial Services
Remote
USA
4405 Employees
110K-143K Annually
Remote
USA
228 Employees
115K-145K Annually

Similar Companies Hiring

Energy CX Thumbnail
Greentech • Professional Services • Business Intelligence • Consulting • Energy • Financial Services • Utilities
Chicago, IL
108 Employees
Northslope Thumbnail
Artificial Intelligence • Information Technology • Software • Analytics • Consulting • Generative AI
London, GB
100 Employees
Amplify Platform Thumbnail
Fintech • Financial Services • Consulting • Cloud • Business Intelligence • Big Data Analytics
Scottsdale, AZ
62 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account