XPT Software Australia Pty Ltd

Data Engineer-GCP

Posted Yesterday

Be an Early Applicant

Sydney, New South Wales, AUS

In-Office

Entry level

Information Technology • Software • Consulting

The Role

Develop and maintain scalable batch and streaming data pipelines using native GCP services. Design data warehouses and models in BigQuery, write Python and SQL, and use Apache Beam or Spark for data processing. Preferred experience includes AWS or Azure integration, Kafka, containerization with Docker and Kubernetes, and GCP data governance, security, IAM, and compliance.

Summary Generated by Built In

Job Details:

Must Have Skills:

 Strong GCP Data Services Expertise - Hands-on experience with BigQuery, Dataflow, Pub/Sub, Cloud Storage, Ability to design scalable data solutions using native GCP tools.

 Experience in Data Pipeline Development (Batch & Streaming) - Building and maintaining ETL/ELT pipelines, Experience with real-time streaming and batch processing.

 Strong Programming & Query Skills - Proficiency in Python and SQL, Experience with data processing frameworks like Apache Beam or Spark

 Understanding of Data Modeling & Warehousing - Strong understanding of data modeling (star/snowflake schemas), Experience designing and optimizing data warehouses (BigQuery)

Nice to Have Skills:

 Experience in Multi-Cloud Exposure : Understanding of cross-cloud data integration and migration, Experience working with AWS or Azure alongside GCP

 Knowledge of Streaming & Messaging Frameworks : Experience with Kafka or advanced Pub/Sub patterns, Knowledge of real-time data architectures

 Basic Understanding of Containerization & Orchestration : Hands-on with Docker and Kubernetes (GKE), Ability to deploy and manage data workloads in containerized environments

 Data Governance & Security : Familiarity with data lineage, cataloging, and governance tools, Understanding of IAM roles, data security, and compliance in GCP.

Skills Required

Hands-on expertise with GCP data services, including BigQuery, Dataflow, Pub/Sub, and Cloud Storage.
Ability to design scalable data solutions using native GCP tools.
Experience building and maintaining batch and streaming ETL or ELT data pipelines.
Experience with real-time streaming and batch processing.
Proficiency in Python and SQL.
Experience with Apache Beam or Spark.
Strong understanding of data modeling, including star and snowflake schemas.
Experience designing and optimizing BigQuery data warehouses.
Experience with cross-cloud data integration or migration involving AWS or Azure.
Experience with Kafka or advanced Pub/Sub patterns and real-time data architectures.
Hands-on experience with Docker and Kubernetes, including GKE.
Familiarity with data lineage, cataloging, governance tools, IAM roles, data security, and GCP compliance.

View all jobs at XPT Software Australia Pty Ltd

View XPT Software Australia Pty Ltd Profile

Report Job

Am I A Good Fit?

beta

Get Personalized Job Insights.

Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company

120 Employees

Year Founded: 2016

What We Do

XPT Software Australia Pty Ltd is a technology consulting and software services company serving clients across banking, financial services and insurance, telecommunications, mining, retail, energy, and manufacturing. It provides software development, IT management consulting, business analysis, project and program management, infrastructure support, and offshore development through onsite-offshore delivery. The company also works with cloud computing, big data, mobile applications, UI/UX, algorithms, and IoT.