Senior Data Engineer (Big Data and Cloud)

Senior Data Engineer (Big Data and Cloud)

See yourself in our team 

You’ll be joining our CTO Data Engineering team. We're seeking passionate, cloud savvy, data engineers across different levels to help drive our commitments in our Data Strategy and achieve our vision for Commbank.data, a unified, world-class data platform enabling the next generation of AI enabled applications. You'll collaborate with industry leaders to drive forward our mission of revolutionising the fintech landscape. As part of our team, you'll be empowered to deliver world-class, competitive banking products and services with unparalleled levels of service and reliability.  

We support our people with the flexibility to balance where work is done with at least half your time each month connecting in office. We also have many other flexible working options available including changing start and finish times, part-time arrangements and job share to name a few. Talk to us about how these arrangements might work for you.  

We’re interested in hearing from people who: 

• Proficient in Python, with experience in SQL and DBT. 

• Experienced with AWS services such as S3, RDS, Redshift, Glue, and Lambda. Familiarity with Databricks and Apache Spark is highly desirable. 

• Knowledgeable in data ingestion, transformation pipelines, integration, and visualization. 

• Experienced in large-scale data processing. 

• Provides code quality control through peer programming, code review, and automated pipeline release management. Experienced in CI/CD and managing data release pipelines. 

• Are capable of mentoring junior engineers and sharing knowledge to enhance software development practices. 

Technical skills: 

This is a highly technical role, requiring a broad range of tools, languages, and frameworks. You’ll be a good match if you have previous experience in: 

• Developed for data warehouses or data lakes. 

• AWS certification with exposure to any of the following AWS Lambda, S3, Redshift, DynamoDB. 

• Infrastructure as code using CloudFormation or Terraform (desirable) 

• Exposed to CI/CD tools (i.e, GitHub Action) 

• Candidates must have experience in ensuring scalability and reliability in their data solutions. Additionally, proficiency in local testing environments and practices is essential. 

Developed knowledge/understanding of (good to have): 

• Experience with AWS services such as SageMaker, or EMR. 

• Containerization (Docker, Kubernetes) 

• Observability tools. 

• Scripting using Scala, Go. 

• Familiarity with technologies like Apache Kafka, AWS Kinesis, or similar. 

• Familiarity with visualization tools such as PowerBI, Tableau, or similar. 

Working with us 

Whether you’re passionate about customer service, driven by data, or called by creativity, a career with us is for you. 

Our people bring their diverse backgrounds and unique perspectives to build a respectful, inclusive, and flexible workplace with flexible work locations. We’re looking for people who truly live our values, care, courage and commitment and we’ll offer you exceptional opportunities to develop your career with us.