Company Description
Are you passionate about data and eager to make a significant impact in the ad tech industry? We're seeking a Senior Data Engineer to be a key player in our dynamic team. Our focus is on transforming and optimizing high-velocity ad log streams, creating a unified data lake for seamless downstream data consumption.
Our client processes billions of ad events daily, providing both real-time and batch data to deliver comprehensive reporting and troubleshooting solutions. As a Senior Data Engineer, you'll be at the forefront of developing, deploying, and maintaining robust data pipelines and infrastructure. You'll work closely with cross-functional teams to design, implement, and refine data solutions that enhance the efficiency of data processing, storage, and analysis.
If you're excited about working with cutting-edge technology in a fast-paced environment, we’d love to have you on board. Don’t hesitate—join our stellar team today!
CUSTOMER
Our client has superior end-to-end technology, a premium marketplace, and best-in-market advisory services that power the world’s advertising businesses of the largest media and entertainment companies. For example, Fox, NBC Universal, Viacom in the USA, Sky, Channel 4, RTE, and Mediaset in Europe.
Job Description
- Design, implement, and maintain scalable data pipelines to ingest and transform large volumes of data from various sources
- Optimize streaming and batch data processing and storage solutions for performance and scalability
- Monitor system performance, troubleshoot issues, and implement solutions to ensure high availability and reliability
- Collaborate with product managers and other stakeholders to understand data requirements and deliver solutions that meet business needs
- Communicate technical concepts and solutions effectively to non-technical stakeholders
- Stay abreast of emerging technologies and best practices in data engineering. Identify opportunities to improve data processes, tools, and infrastructure to enhance efficiency and effectiveness
- Document data pipelines, processes, and systems to ensure clarity and maintainability
- Share knowledge and best practices with team members to foster a culture of learning and collaboration
- Work with and mentor junior Data Engineers
- Other duties and responsibilities as assigned
Qualifications
- 5+ years of hands-on experience in the Software Development field and/or Big Data
- 2+ years of hands-on experience in building and operating large-scale data processing system
- Solid programming skills, fluent with Scala
- Familiar with big data processing Framework/tools Spark, Spark Streaming, Databricks, Flink
- Experiences with Data Lakehouse technics Delta Lake and Iceberg
- Solid SQL skills with experience in writing complex queries, stored procedures, and optimizing querying performance
- Experiences with data warehouse, data modeling techniques, ETL processes, and relational databases (MySQL, PostgreSQL)
- Familiarity with AWS (EC2, S3, etc.), and proficiency in managing cloud-based data solutions (Snowflake)
- Upper-Intermediate level of English
Additional Information
PERSONAL PROFILE
- Excellent problem-solving skills and attention to detail
- Strong communication skills, with the ability to collaborate effectively in a team environment
- Proven ability to deliver results in a fast-paced environment
Top Skills
What We Do
Sigma Software Group, an award-winning and trusted IT partner, has been serving customers for over 21 years, providing comprehensive IT solutions to various businesses, ranging from startups to established software product houses. As one of Europe's substantial IT consultancies, it brings together a dedicated workforce of over 2,100 professionals in 40 offices across 19 countries. With a diverse client base, including more than 300 enterprises, including Fortune 500 stalwarts, Sigma Software Group is a preferred choice for developing solutions that help businesses create cutting-edge products while meeting their unique needs.
Sigma Software Group operates as a dynamic ecosystem of tech companies, offering 25 ready-to-implement innovative products and 40+ value-added services. Furthermore, Sigma Software Group is committed to fostering innovation through initiatives such as the Sigma Software Labs business incubator, Sigma Software University, the SID Venture Partners VC Fund, UA Tech Network, Techosystem, the European Business Association, and other collaborative efforts.
Since 2015, Sigma Software Group has consistently earned recognition on the IAOP's prestigious World's Top 100 Outsourcing list. The company's accomplishments have also been acknowledged by prominent global media outlets such as Forbes, CNBC, The Times, and Reuters