XPT Software Australia Pty Ltd

Senior Data Engineer

Posted 7 Days Ago

Be an Early Applicant

Sydney, New South Wales, AUS

In-Office

Senior level

Information Technology • Software • Consulting

The Role

Designs, builds, and operates scalable, secure, high-performance data platforms and pipelines supporting analytics, reporting, machine learning, and real-time insights. Collaborates with analytics, data science, product, and business teams; mentors engineers; optimizes Databricks workloads; and drives architecture, engineering standards, governance, and cloud cost efficiency.

Summary Generated by Built In

Job Description:

Must Have Skills:

 Strong hands‑on experience with Azure Data Lake Gen2, Synapse, Databricks

 Solid experience in PySpark / Spark SQL, Databricks performance tuning & job optimization

 Proficient in Complex SQL (window functions, CTEs, execution plans)

 Strong understanding in Mentoring junior and mid‑level engineers, Communicating trade‑offs to architects and product teams

 Experience working with Azure DevOps or GitHub Actions

Nice to Have Skills:

 Experience with Streaming & Real‑Time Data Processing on Designing low‑latency pipelines.

 Exposure to Data Governance & Metadata Management in Data classification & sensitivity labels

 Familiarity with Advanced Python Engineering Practices on Performance‑aware Python design

 Experience working in Analytics & BI Awareness

 Knowledge of Cloud Cost Optimization & FinOps

Detailed Job Description

The Senior Data Engineer will be responsible for designing, building, and operating scalable, secure, and high‑performance data platforms. The role involves critical data pipelines and architectures that power analytics, reporting, and advanced data use cases such as machine learning and real‑time insights.

The candidate will collaborate closely with analytics, data science, product, and business teams while mentoring engineers and driving engineering best practices across the data organization.

The role combines technical leadership, architecture ownership, and business impact.

Skills Required

Strong hands-on experience with Azure Data Lake Gen2, Azure Synapse, and Databricks
Solid experience with PySpark and Spark SQL
Experience with Databricks performance tuning and job optimization
Proficiency in complex SQL, including window functions, CTEs, and execution plans
Experience mentoring junior and mid-level engineers
Ability to communicate technical trade-offs to architects and product teams
Experience with Azure DevOps or GitHub Actions
Experience designing low-latency streaming and real-time data pipelines
Exposure to data governance, metadata management, data classification, and sensitivity labels
Familiarity with performance-aware Python engineering practices
Analytics and business intelligence awareness
Knowledge of cloud cost optimization and FinOps

View all jobs at XPT Software Australia Pty Ltd

View XPT Software Australia Pty Ltd Profile

Report Job

Am I A Good Fit?

beta

Get Personalized Job Insights.

Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company

120 Employees

Year Founded: 2016

What We Do

XPT Software Australia Pty Ltd is a technology consulting and software services company serving clients across banking, financial services and insurance, telecommunications, mining, retail, energy, and manufacturing. It provides software development, IT management consulting, business analysis, project and program management, infrastructure support, and offshore development through onsite-offshore delivery. The company also works with cloud computing, big data, mobile applications, UI/UX, algorithms, and IoT.