Data Engineer

As a data engineer you will be enabling the business by providing data, reporting and analytics solutions, through developing and supporting Data Ingestion and transformation in the Azure Enterprise Data & Analytics Platform (EDAP).

What will you be doing?

• Designing, developing, and maintaining scalable data pipelines using Databricks, ensuring robust data processing workflows for cleaning, transforming, and aggregating data.
• Implementing stringent measures to ensure data quality, governance, and compliance within the data lake, leveraging Databricks lifecycle management to maintain predictive models.
• Deploying and managing data solutions using Azure cloud platforms, optimising data and platform performance for maximum efficiency.
• Working closely with various business units, data analysts, and scientists to understand and meet diverse data needs, facilitating the integration and deployment of AI and ML models.
• Demonstrating the ability to anticipate and balance the needs of multiple stakeholders, building partnerships and working collaboratively to achieve shared objectives.
• Developing new and innovative strategies to enhance organisational success, handle complex and sometimes contradictory information to effectively solve problems, and focus on building member relationships while delivering customer-centric solutions.

What are we looking for in you?

• Exposure to modern data engineering tool set, preferably databricks with an interest in developing your expertise in building and maintaining data pipelines, operational processes, and predictive models. A solid grounding in data architecture, data modelling, and data lake concepts with a certification in Databricks will be highly regarded.
• Understanding and usage of Unity Catalog and Delta Tables, particularly in managing secure data access and optimising performance within Databricks environments.
• Familiarity with modern data and AI technologies, Synapse, MLOps, LLM frameworks or Generative AI. Willingness to learn new tools and work effectively in cross functional team is essential.
• Strong foundation in SQL, Python, and PySpark, with experience in apply concept to BAU and Project activities.
• Familiarity with various cloud platforms, such as GCP, Azure, and AWS, is preferred. Experience with Azure services, including Data Lake, Blob Storage, and SQL-based solutions, is also desirable.
• Experience using GitHub or similar for version control, with some exposure to CI/CD pipelines, ETL processes, or real-time data integration.
• Understanding of Data Quality and Data Governance practices and willingness to expand capability in these area.
• Experience with business intelligence and reporting tools, preferably Power BI, as well as exposure to APIs and tools such as Postman for testing and integration.
• A proactive and collaborative mindset, with strong problem-solving skills and a keen eye for detail.

What can we offer you?

• Working in an environment that is embracing a continuous improvement culture using experimentation to support learning.
• Time dedicated to your personal development time with access to training, development, and certification programs to ensure you can upskill.
• Being part of empowered, cross-functional agile delivery teams that works with the business to build systems that solve problems.
• Complete flexibility to choose between working from the home or office with flexible hours for a better work life balance.
• Building digital systems that will shape the future for the RAC and a better WA.
• Gain RAC Staff Benefits including free Roadside Assistance and 25% discount on your Insurances, Social Club and access Fitness Passport for cheaper gym memberships.