While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!
- Architect and design end-to-end data platform solutions using Azure Databricks as the primary compute engine.
- Define data lake strategies and structures leveraging Azure Data Lake Storage Gen2 for raw, curated, and consumer zones.
- Design and optimize ETL/ELT pipelines using Azure Data Factory (ADF) and Azure Databricks notebooks/jobs.
- Lead platform-level security design by integrating with Azure Active Directory (AAD) and applying role-based access control (RBAC), Unity Catalog (if applicable), and data masking policies.
- Define standards for Delta Lake implementation, partitioning, schema evolution, and cost optimization in Spark jobs.
- Guide teams on the use of CI/CD, infrastructure-as-code, and automated deployment patterns for data workloads.
- Ensure data solutions meet governance, quality, and compliance requirements across environments.
- Collaborate with business, data governance, and engineering teams to align architecture with enterprise data strategies.
- Provide architecture oversight for performance tuning, scalability, disaster recovery, and observability setup.
- 10+ years of experience in data engineering or architecture, with at least:
- 3–5 years of hands-on experience with Azure Databricks and Apache Spark.
- Strong exposure to Azure Data Lake Storage Gen2.
- Proficiency in Azure Data Factory for pipeline orchestration.
- Good understanding of Azure Active Directory, AAD groups, service principals, and access control models.
- Expertise in designing multi-zone lakehouse architectures and distributed processing solutions.
- Strong in Python (PySpark), SQL, and Spark job performance tuning.
- Experience with Delta Lake, schema evolution, and versioning.
- Proven ability to lead and mentor engineering teams on architecture and technical decisions.
- Knowledge of Unity Catalog for data governance and metadata management.
- Experience with Azure Synapse Analytics, Event Hubs, or Purview.
- Familiarity with data mesh, domain-driven data ownership, or MDM integration.
- Understanding of data quality frameworks and observability tooling.
- Databricks Certified Data Engineer – Professional
- Azure Data Engineer Associate
If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!
Top Skills
What We Do
Quantiphi is an award-winning AI-first digital engineering company driven by the desire to solve transformational problems at the heart of business.
Quantiphi solves the toughest and complex business problems by combining deep industry experience, disciplined cloud, and data-engineering practices, and cutting-edge artificial intelligence research to achieve quantifiable business impact at unprecedented speed.
