Databricks Engineer

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

About Quantiphi:

Quantiphi is an award-winning, AI-First global digital engineering company that helps the world’s leading Fortune 1000 organizations transform bold ideas into measurable business impact. We go beyond building innovative AI technologies, we solve the problems that matter most to our clients.

Since our founding in 2013, Quantiphi has built a proven track record of turning complex challenges into meaningful outcomes across industries.

Headquartered in Boston, with more than 4,000 professionals worldwide, we partner with global enterprises to deliver large-scale digital, cloud, and AI-driven transformation. #SolvingWhatMatters

We are an Elite and Premier partner to Google Cloud, AWS, NVIDIA, Snowflake, and other leading technology platforms, and our work has been recognized across the industry, including:

• 21 Google Cloud Partner of the Year awards in the past 10 years

• 3 AWS AI/ML Partner of the Year awards

• 3 NVIDIA Partner of the Year awards

• 3 Snowflake Partner of the Year awards

• Rated Leaders by Gartner, Forrester, IDC, ISG, Everest Group and other leading analyst firms

Quantiphi delivers First-in-class AI solutions across  Life Sciences, Healthcare, Banking, Financial Services, CPG, Manufacturing, Energy, High-Tech, Telecommunications, etc., powered by cutting-edge Generative AI and Agentic AI accelerators.

We are also proud to be certified as a Great Place to Work, reflecting our commitment to our people and our culture.

For more details, visit: Website or LinkedIn Page.

Role: Databricks Data Engineer

Experience Level:  7+ years of experience

Employment type: Full time

Work location: Remote Canada

What you’ll do:

We are looking for an experienced Databricks Data Engineer to join our Insights & Analytics team on a contract basis. In this role, you will design, build, and optimize scalable data pipelines using the Databricks platform while leveraging modern software engineering practices. You'll work closely with cross-functional teams to deliver high-quality data solutions that support analytics, reporting, and AI-driven initiatives.

The ideal candidate has deep expertise in Databricks, Spark, and cloud-native data engineering, along with experience using AI-assisted development tools responsibly to improve engineering productivity and code quality.

Role & Responsibilities:

• Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.

• Build and optimize ETL/ELT workflows to ingest, transform, and curate large datasets.

• Implement data solutions following Medallion Architecture (Bronze, Silver, Gold).

• Develop high-performance PySpark and Spark SQL transformations.

• Optimize Delta Lake tables using partitioning, optimization, vacuuming, and time travel capabilities.

• Orchestrate and monitor Databricks Workflows and scheduled jobs.

• Collaborate with analytics, engineering, and product teams to deliver reliable data products.

• Apply software engineering best practices, including version control, automated testing, and CI/CD.

• Utilize AI-assisted development tools for code generation, debugging, documentation, and productivity while validating outputs to ensure quality and accuracy.

• Contribute to data quality, performance tuning, and operational excellence across the platform.

Essential Skills/Qualifications:

• Bachelor's degree in Computer Science, Software Engineering, or a related field (or equivalent work experience)

• Hands-on experience with Databricks Workspaces, Clusters, Jobs, and Unity Catalog

• Strong knowledge of Delta Lake, including ACID transactions, Time Travel, OPTIMIZE, and VACUUM

• Experience building and managing Databricks Workflows

• Strong experience with PySpark, Spark SQL, and Scala

• Expertise designing and building ETL/ELT pipelines

• Experience implementing Medallion Architecture (Bronze/Silver/Gold)

• Knowledge of incremental data loading patterns and pipeline optimization

• Programming Languages: Python (Advanced) SQL (Advanced), Scala (Strong working knowledge)

• Experience with at least one of the following cloud platform: AWS (Amazon S3, AWS Glue, Redshift), Azure (Azure Data Lake Storage Gen2 (ADLS Gen2),Azure Data Factory (ADF), Azure DevOps)

• Experience using AI-powered developer tools for coding, debugging, testing, and documentation

• Understanding of responsible AI usage, code validation, and quality assurance

• DevOps & Engineering Practices: Git version control, CI/CD pipelines, Unit testing and automated validation for data transformations, Strong software engineering and code quality practices

Nice to have Skils:

• Terraform or Databricks Asset Bundles

• Experience building streaming data pipelines

• dbt

• Data quality and observability frameworks

• MLflow

• Java experience

• Experience working in Agile/Scrum environments

What is in it for you:

• Join one of the world’s fastest-growing AI-first digital engineering companies and make a real impact at scale.

• Lead and collaborate with a high-energy team of talented, driven individuals solving complex, meaningful challenges.

• Work with Fortune 500 companies and disruptive innovators in a research-driven environment with 60+ patents.

• Stay ahead of the curve by gaining hands-on experience with cutting-edge AI, ML, data, and cloud technologies while continuously upskilling.