Data Engineer with GCP

Posted 7 Days Ago
Be an Early Applicant
Sydney, New South Wales, AUS
In-Office
Mid level
Information Technology • Software • Consulting
The Role
Design, develop, and maintain scalable batch and real-time data pipelines on Google Cloud. Build ETL/ELT workflows, data lakes, and warehouses using BigQuery, Dataflow, Composer, Pub/Sub, Python, PySpark, and Apache Beam. Perform data modeling, quality validation, monitoring, performance and cost optimization, while supporting security, governance, and collaboration with analytics and business teams.
Summary Generated by Built In
Key Responsibilities
  • Design, develop, and maintain scalable data pipelines using GCP services.

  • Build and optimise ETL/ELT workflows for batch and real-time data processing.

  • Work with Google BigQuery for data warehousing, SQL development, and query optimisation.

  • Develop data processing solutions using Python, PySpark, and Apache Beam.

  • Implement data integration using Cloud Dataflow, Cloud Composer, and Pub/Sub.

  • Build and manage data lakes and data warehouses on Google Cloud.

  • Implement data quality checks, data validation, and monitoring.

  • Optimise pipeline performance, scalability, and cost efficiency.

  • Collaborate with data analysts, data scientists, architects, and business teams.

  • Ensure data security, governance, and compliance best practices.

Mandatory Technical Skills
  1. Strong experience in Google Cloud Platform (GCP).

  2. Hands-on experience with BigQuery.

  3. Strong programming skills in Python and SQL.

  4. Experience developing ETL/ELT data pipelines.

  5. Experience with Cloud Dataflow and/or Apache Beam.

  6. Knowledge of Cloud Storage (GCS).

  7. Experience with data modelling and data warehousing.

  8. Experience with batch and real-time data processing.



Skills Required

  • Strong experience with Google Cloud Platform
  • Hands-on experience with BigQuery
  • Strong programming skills in Python and SQL
  • Experience developing ETL/ELT data pipelines
  • Experience with Cloud Dataflow and/or Apache Beam
  • Knowledge of Cloud Storage
  • Experience with data modeling and data warehousing
  • Experience with batch and real-time data processing
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Melbourne
120 Employees
Year Founded: 2016

What We Do

XPT Software Australia Pty Ltd is a technology consulting and software services company serving clients across banking, financial services and insurance, telecommunications, mining, retail, energy, and manufacturing. It provides software development, IT management consulting, business analysis, project and program management, infrastructure support, and offshore development through onsite-offshore delivery. The company also works with cloud computing, big data, mobile applications, UI/UX, algorithms, and IoT.

Similar Jobs

NCS Group Australia Logo NCS Group Australia

Data Engineer

Artificial Intelligence • Cloud • Information Technology • Consulting • Cybersecurity • Big Data Analytics
Hybrid
Sydney, New South Wales, AUS
1358 Employees

XPT Software Australia Pty Ltd Logo XPT Software Australia Pty Ltd

Data Engineer

Information Technology • Software • Consulting
In-Office
Sydney, New South Wales, AUS
120 Employees

NCS Group Australia Logo NCS Group Australia

Data Engineer

Artificial Intelligence • Cloud • Information Technology • Consulting • Cybersecurity • Big Data Analytics
Hybrid
Sydney, New South Wales, AUS
1358 Employees

Datadog Logo Datadog

Sales Development Representative

Artificial Intelligence • Cloud • Security • Software • Cybersecurity
Easy Apply
Hybrid
Sydney, New South Wales, AUS
6500 Employees

Similar Companies Hiring

Revel Thumbnail
Aerospace • Hardware • Robotics • Software
Marina Del Rey, California
70 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
33 Employees
RapidScale Thumbnail
Cloud • Information Technology • Consulting
Raleigh, NC
600 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account