Lead Cloud Data Engineer

Senior Engineer

We are looking to hire an experienced Sr. Cloud Data Engineer Lead to work with LSEG (LONDON STOCK EXCHANGE GROUP) Azure data platform initiative to build applications on the newly launched Fabric and Purview Service. The person must have skills to design, implement and test solutions.  The person will also work closely with other data engineers, data analysts, data scientists, and business stake holders to understand their data needs and deliver scalable and efficient data solutions. This role is accountable for technology solution delivery and leading team. 

Responsibilities

• Work with Customers, stakeholders, and Suppliers to establish Solution Intent; help establish the solution intent information models and documentation requirements

• Influence and drive adoption common modelling, design, and coding practices at the program level

• Synchronize solutions for data security and quality; Infrastructure; scalability, performance and other NFRS

• Identifies, prioritizes, and executes tasks in the software development life cycle

• Develop tools and applications by producing clean and efficient code

• Automates tasks through appropriate tools and scripting

• Design, Builds and oversee DevOps and CI/CD process

• Collaborates with internal teams (including Product Owners / Businesses) and vendors to fix and improve products

• Mentors Developers by providing advice and coaching.

• Provide partnership and support to SME’s and Tech Leads to ensure delivery on commitments.

• Build and maintain secure and compliant production data processing pipelines by using different tools and techniques)

• Ensure that data pipelines and data stores are highly performant, efficient, organized, and reliable, given a set of business requirements and constraints. They deal with unanticipated issues swiftly, and they minimize data loss.

• Design, implement, monitor, and optimize data platforms to meet the data pipelines needs.

• Design and implement data engineering, ingestion, and curation functions on Azure cloud using Azure native or custom programming. Create data transformation and loading workflows.

• Responsible for data-related implementation tasks that include provisioning data storage services, ingesting streaming, and batch data, transforming data, implementing security requirements, implementing data retention policies, identifying performance bottlenecks, and accessing external data sources.

• Implement data solutions that use the following Azure services: Fabric, Azure Synapse Analytics, Azure Data Factory, Azure Stream Analytics, Azure Databricks, and Azure Blob storage, Microsoft Purview etc.

• Responsible for integrating, transforming, and consolidating data from various structured and unstructured data systems into a structure that is suitable for building analytics solutions.

• Contributes to and develops high-quality software design and architecture

Knowledge & Behaviours

Technical/ job functional knowledge

• Strong experience with Fabric, Purview, Snowflake, Azure: Microsoft Purview, Fabric/Synapse Analytics/ Databricks, Data Factory, Data Lake, Azure Monitor, ADLS, Azure Functions, Log Analytics, Solidatus

• Proficient in data processing languages such as SQL, Python, having PySpark knowledge is a bonus.

• Strong notions of security standard processes (e.g., using Azure Key Vault, Microsoft Entra ID, RBAC, Azure Monitor etc.).

• Good understanding of Azure Networking concepts.

• Good understanding of UML.

• Good understanding and working experience of Devops tools, Terraform

• Good understanding of REST APIs.

• Agile development processes (SCRUM).

• Good understanding of AI concepts, LLM, MCP, Ontology and Knowledge Graph is a bonus.

Certification & development experience

• 10 + years of IT experience with 5 + years of relevant experience in Data Platforms (in Financial Services industry) and Azure’s PaaS offerings (Fabric, Purview, ADF, Azure Data Lake Storage etc.)

• Ability to lead team.

• Ability to work independently as well as in a team.

• Working experience with agile teams in different locations.

• Certification Azure – DP-203: Data Engineering on Microsoft Azure and DP-900: Microsoft Azure Data Fundamentals certification preferred.

Personal skills & capabilities

·        Resilient and not fazed by an exciting environment.

·        Ability to prioritise work.

·        Strong communication, presentation, documentation, and interpersonal skills.

·        Able to self-manage and work independently in a fast-paced environment with dynamic requirements and priorities.

• Self-motivated and proactive with demonstrated critical thinking and creative problem-solving skills.

• Build and Mentors team by providing advice and coaching.

Career Stage:

London Stock Exchange Group (LSEG) Information:

Join us and be part of a team that values innovation, quality, and continuous improvement. If you're ready to take your career to the next level and make a significant impact, we'd love to hear from you.

LSEG is a leading global financial markets infrastructure and data provider. Our purpose is driving financial stability, empowering economies and enabling customers to create sustainable growth.

Our purpose is the foundation on which our culture is built. Our values of Integrity, Partnership, Excellence and Change underpin our purpose and set the standard for everything we do, every day. They go to the heart of who we are and guide our decision making and everyday actions.

Working with us means that you will be part of a dynamic organisation of 25,000 people across 65 countries. However, we will value your individuality and enable you to bring your true self to work so you can help enrich our diverse workforce.

We are proud to be an equal opportunities employer. This means that we do not discriminate on the basis of anyone’s race, religion, colour, national origin, gender, sexual orientation, gender identity, gender expression, age, marital status, veteran status, pregnancy or disability, or any other basis protected under applicable law. Conforming with applicable law, we can reasonably accommodate applicants' and employees' religious practices and beliefs, as well as mental health or physical disability needs.

You will be part of a collaborative and creative culture where we encourage new ideas. We are committed to sustainability across our global business and we are proud to partner with our customers to help them meet their sustainability objectives. Our charity, the LSEG Foundation provides charitable grants to community groups that help people access economic opportunities and build a secure future with financial independence. Colleagues can get involved through fundraising and volunteering.

LSEG offers a range of tailored benefits and support, including healthcare, retirement planning, paid volunteering days and wellbeing initiatives.

Please take a moment to read this privacy notice carefully, as it describes what personal information London Stock Exchange Group (LSEG) (we) may hold about you, what it’s used for, and how it’s obtained, your rights and how to contact us as a data subject.

If you are submitting as a Recruitment Agency Partner, it is essential and your responsibility to ensure that candidates applying to LSEG are aware of this privacy notice.