The Senior Data Engineer will collaborate with our team to build, enhance, and maintain high-performance data systems and machine learning models, ensuring their scalability and reliability.
Responsibilities include designing and maintaining high-throughput data pipelines, optimizing real-time and batch data processing using Scala, and integrating advanced data solutions to improve performance. The role involves working with large-scale distributed systems, handling terabytes of data per second and processing millions of writes per second.
Our team is at the forefront of AdTech innovation, enabling cutting-edge solutions for data-driven advertising. We value hard work and flexibility, offering an environment where engineers can experiment, innovate, and push technological boundaries.
Our client is a leading AdTech company specializing in high-performance data systems and machine learning models, processing terabytes of data per second and millions of writes per second.
It is a large-scale AdTech platform designed to process massive amounts of data in real time, optimizing data-driven advertising with advanced machine learning models. It ensures scalability and reliability, enabling businesses to maximize efficiency and performance while leveraging cutting-edge data solutions.
Job Description- Work with Apache Spark batch and real-time streaming to process data at scale
- Work with Scala microservices hosted on K8s (GKE) to support our products
- Work with MlLib and TensorFlow on Vertex AI to solve prediction and simulation issues, as well as a variety of machine learning challenges
- Deploy new models and engines for new ideas and analysis.
- 5+ years of experience as a Data Engineer
- Advanced knowledge of different non-relational schema models (Column Family, Graph, Document, and Object)
- Experience with advanced topologies and architecture design, such as Kappa and Lambda architectures
- GCP Dataflow experience
- Experience in Scala
- Experience in Python
- Strong background in working with Large Language Models (LLMs)
- Experience with high-throughput systems
- Strong communication skills and at least C1 level of English
WOULD BE A PLUS
- Experience in the AdTech industry
PERSONAL PROFILE
- Problem-solving mindset
- Ability to work closely with cross-functional teams
- Willingness to experiment with new technologies and approaches
Similar Jobs
What We Do
Sigma Software Group, an award-winning and trusted IT partner, has been serving customers for over 21 years, providing comprehensive IT solutions to various businesses, ranging from startups to established software product houses. As one of Europe's substantial IT consultancies, it brings together a dedicated workforce of over 2,100 professionals in 40 offices across 19 countries. With a diverse client base, including more than 300 enterprises, including Fortune 500 stalwarts, Sigma Software Group is a preferred choice for developing solutions that help businesses create cutting-edge products while meeting their unique needs.
Sigma Software Group operates as a dynamic ecosystem of tech companies, offering 25 ready-to-implement innovative products and 40+ value-added services. Furthermore, Sigma Software Group is committed to fostering innovation through initiatives such as the Sigma Software Labs business incubator, Sigma Software University, the SID Venture Partners VC Fund, UA Tech Network, Techosystem, the European Business Association, and other collaborative efforts.
Since 2015, Sigma Software Group has consistently earned recognition on the IAOP's prestigious World's Top 100 Outsourcing list. The company's accomplishments have also been acknowledged by prominent global media outlets such as Forbes, CNBC, The Times, and Reuters
