We are looking for an experienced Principal Data Engineer to join our growing team building our strategic data
platform as we look to adopt best in class data governance and democratise data around the business. The
successful candidate will play a pivotal role in shaping and executing our data strategy, leveraging our data
assets to the full, elevating our decision-making, streamlining processes, and drive innovation within our
organisation.
You will be part of the long-standing Data Platform team using Agile methods to deliver solutions on top of our
cloud platform. Starting with Snowflake as a foundational component, we will look to integrate other
technologies and grow capability to meet the demands of the business. You will work closely with a variety of
business areas and be involved in all aspects of the Secure System Development Lifecycle (S-SDLC).
We operate in a collaborative and inclusive environment, where each team member's expertise is valued. Our
culture emphasizes open communication, mutual support, and a shared commitment to delivering exceptional
results. We foster a learning culture, providing opportunities for growth and innovation while maintaining high
standards of professionalism and integrity.
Role Responsibilities
- Contribute to the solution design and implementation of a robust and scalable data platform using
- Snowflake with complementary tooling to meet the organisation's needs.
- Help evolve the Insight Data Strategy, communicate outcomes from a leadership through to team level.
- Set the firm-wide standards and conventions for ingesting, governing, transforming, persisting, and democratising data.
- Provide expertise in ELT pipeline performance optimisation
- Enable the adoption of advanced analytics & AI by configuring best-in-class tooling.
- Evolve the usage of infrastructure-as-code in collaboration with cloud engineering.
- Align technical designs with internal data, privacy, security and compliance policies.
- Stay abreast of industry trends and emerging technologies related to Snowflake and big data processing, recommending relevant advancements for consideration.
Experience Required
- Snowflake (Warehouses, Tasks/Streams, Snowpipe, Snowpark, Distributed UDF/Sproc, Time Travel, Cloning)
- Demonstratable experience of delivering end to end pipelines in Snowpark (transferable frameworks such as Pyspark, Polars or Dask are also welcome).
- DBT including Jinja templating or equivalent transformation frameworks
- Python including build tools, test automation frameworks and engineering best practices
- SQL (including Snowflake-optimised SQL)
- Experience with CI/CD pipelines (GitHub Actions / TeamCity / Artifactory).
- Demonstrate initiative and independence to solve problems. § Be able to communicate abstract or complex ideas or solutions effectively. § Stand by software engineering principles around quality and doing things the right way.
- Experience working in and with Agile teams performing iterative development. § Ability to collaborate with end users, stakeholders, teams, and team members. § Have a constant desire to improve knowledge and drive adoption of new tech or approaches within the teams. § Own mistakes and work with your colleagues to redress them, learn from them and make improvements for the long-term.
- Docker
- Java / Scala
- Event streaming technologies such as Kafka, Pulsar, or Azure Event Hub
- Integration with Collibra
- Apache Iceberg
- Apache Arrow
- Pyspark
- AI engineering (Snowflake AI, Langchain, Llamaindex, Pinecone, etc)
Insight is committed to being an inclusive employer and encourages applications from all suitably qualified applicants irrespective of background, circumstances, age, disability, gender identity, ethnicity, religion or belief, sexual orientation or other factors protected by federal, state and/or local laws. If you are a candidate with a disability, or are assisting a candidate with a disability, and require an accommodation to apply for one of our jobs, please email us at [email protected]
About Insight Investment
Insight Investment is a leading asset manager focused on designing investment solutions to meet its clients' needs. Founded in 2002, Insight's collaborative approach has delivered both investment performance and growth in assets under management. Insight manages assets across its core liability-driven investment, risk management, full-spectrum fixed income, currency and absolute return capabilities. Insight has a global network of operations in the UK, Ireland, Germany, US, Japan and Australia. More information about Insight Investment can be found at: https://www.insightinvestment.com/corporate/
Skills Required
- Snowflake expertise, including warehouses, tasks and streams, Snowpipe, Snowpark, distributed UDFs and stored procedures, Time Travel, and cloning
- End-to-end pipeline delivery using Snowpark
- Experience with transferable frameworks such as PySpark, Polars, or Dask
- dbt with Jinja templating or equivalent transformation frameworks
- Python, including build tools, test automation frameworks, and engineering best practices
- SQL, including Snowflake-optimized SQL
- CI/CD pipeline experience with GitHub Actions, TeamCity, or Artifactory
- Initiative and independence in problem solving
- Ability to communicate abstract or complex ideas effectively
- Commitment to software engineering quality principles
- Experience working in Agile teams using iterative development
- Ability to collaborate with end users, stakeholders, teams, and colleagues
- Docker experience
- Java or Scala experience
- Experience with event streaming technologies such as Kafka, Pulsar, or Azure Event Hub
- Integration experience with Collibra
- Apache Iceberg experience
- Apache Arrow experience
- PySpark experience
- AI engineering experience with technologies such as Snowflake AI, LangChain, LlamaIndex, or Pinecone
- Willingness to continuously develop technical knowledge and drive adoption of new technologies or approaches
What We Do
Insight* is one of Europe’s largest investment managers** responsible for £626.2bn/€757.3bn*** in assets under management (AUM) across fixed income, risk management strategies including liability-driven investment (LDI) and currency risk management, absolute return and multi-asset capabilities. Based in London, Insight is a specialist asset manager with a global network of operations in the UK, Ireland, Germany, US, Japan and Australia. Our clients include pension funds, corporates, local authorities, insurers, sovereign wealth funds, wealth managers, financial institutions and supranationals. Insight's mission is to offer investors a different approach to achieving their investment goals; one that prioritises the certainty of meeting their chosen objectives in contrast to the traditional focus on maximising return and minimising volatility. Read our mission statement here: www.insightinvestment.com/uk/introducing-insight/our-mission-statement/ We believe responsible investment means helping our clients achieve their desired outcomes and reflecting their priorities. We aim to identify, assess and manage factors that we deem to be financially material, while also reflecting our clients’ sustainability preferences. We were a founding signatory to the UN-supported Principles for Responsible Investment (PRI) in 2006. For more on Insight’s approach to responsible investment, visit https://www.insightinvestment.com/investing-responsibly Posts are intended for UK/EU professional investors only. Capital at risk. *Insight is the corporate brand for certain companies operated by Insight Investment Management Limited (IIML). **IPE, ‘Top 500 Asset Managers’ survey, June 2024. Insight is ranked fifth out of the top 120 European institutional managers by total AUM for external Europe-domiciled institutional clients. ***As at 31 December 2024. AUM are represented by the value of cash securities and other economic exposure managed for clients.
