Senior Data Engineer

We are seeking a skilled Data Engineer to join our dynamic team. The ideal candidate will have a strong background in big data technologies, ETL/ELT processes, and data modeling. This role will focus on developing and optimizing data pipelines, ensuring data integrity, and supporting our analytics initiatives.

We are seeking a skilled Data Engineer with 5 to 7+ years of relevant experience to join our dynamic team. The ideal candidate will have a strong background in big data technologies, ETL/ELT processes, and data modeling. This role will focus on developing and optimizing data pipelines, ensuring data integrity, and supporting our analytics initiatives.Key Responsibilities: • Design, develop, and maintain scalable ETL/ELT pipelines using PySpark and Databricks to support data ingestion and processing.

• Implement and optimize data streaming solutions for real-time data processing.

• Optimize Spark jobs for performance, including memory management, partitioning strategies, and efficient data storage formats.

• Collaborate with data scientists and analysts to understand data requirements and deliver reliable datasets for analysis.

• Write and optimize complex SQL queries for data extraction, transformation, and analysis.

• Ensure data quality and integrity through automated testing and validation processes.

• Document data workflows and maintain metadata for data governance purposes.

• Explore and implement new data engineering technologies and methodologies to improve efficiency and scalability.

Mandatory Skills:
• PySpark: Proficient in PySpark for data processing and ETL workflows.

• Databricks: Experience with the Databricks platform, including setting up and managing clusters.

• Data Streaming: Knowledge of streaming data processing with frameworks like Spark Streaming.

• Python: Strong Python programming skills for scripting and automation.

• SQL: Advanced SQL skills for querying and managing relational databases.

• Spark Optimization: Experience in optimizing Spark applications for performance improvements.

Optional Skills:
• Snowflake: Familiarity with Snowflake for data warehousing and query optimization.

• Cloud Platforms: Knowledge of cloud services (AWS, Azure, GCP) for data storage and processing.

• ETL/ELT Concepts: Understanding of ETL/ELT processes, data modeling, and warehousing best practices.

• Big Data Tools: Familiarity with tools and frameworks such as Kafka, Hadoop, and Hive.

• CI/CD Practices: Knowledge of CI/CD for automated deployment and version control using Git, Jenkins, etc.

• Flexible working hours (self-managed)
• Competitive salary
• Annual bonus, subject to company performance
• Access to Udemy online training and opportunities to learn and grow within the role

At Mindera we use technology to build products we are proud of, with people we love.

Software Engineering Applications, including Web and Mobile, are at the core of what we do at Mindera.

We partner with our clients, to understand their products and deliver high-performance, resilient and scalable software systems that create an impact on their users and businesses across the world.

You get to work with a bunch of great people, and the whole team owns the project together.

Our culture reflects our lean and self-organisation attitude.

We encourage our colleagues to take risks, make decisions, work in a collaborative way and talk to everyone to enhance communication. We are proud of our work and we love to learn all and everything while navigating through an Agile, Lean and collaborative environment.

Check out our Blog: and our Handbook:

Our offices are located: Porto, Portugal | Aveiro, Portugal | Coimbra, Portugal | Leicester, UK | San Diego, USA | Chennai, India | Bengaluru, India