The Role
Design, develop, and optimize scalable batch and streaming data pipelines using Python, PySpark, and SQL. Build data models, process large datasets, implement data quality and validation controls, troubleshoot pipeline issues, and improve performance and reliability. Collaborate with engineering, architecture, and business teams in Agile environments while applying modern cloud data engineering and software development practices.
Summary Generated by Built In
As a Data Engineer with PySpark, you will:
- Design, develop, and maintain scalable and reliable data processing solutions using PySpark.
- Build and manage robust batch and streaming data pipelines.
- Develop efficient data transformation and processing solutions using Python, PySpark, and SQL.
- Design, develop, and optimize data models to support scalable and high-performance data processing.
- Work with large datasets and complex data processing workloads.
- Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and scalability.
- Implement data quality, validation, and error-handling mechanisms across data pipelines.
- Collaborate with engineering, architecture, and business teams to deliver reliable data solutions.
- Contribute to Agile development practices and continuously improve data engineering standards and processes.
What You Bring to the Table:
- 5+ years of professional experience in Data Engineering or a related field.
- Strong hands-on experience with PySpark as a core data processing technology.
- Strong proficiency in Python, PySpark, and SQL.
- Hands-on experience developing and maintaining batch and streaming data pipelines.
- Experience working with large-scale data processing and transformation.
- Good understanding of data modeling, data integration, and data optimization.
- Experience with cloud-based data platforms and modern data engineering architectures.
- Experience with data quality, validation, troubleshooting, and performance optimization.
- Familiarity with CI/CD and modern software engineering practices.
- Experience working in Agile development and delivery environments.
- Strong communication, collaboration, analytical, and problem-solving skills.
You Should Possess the Ability to:
- Develop scalable and high-performance data pipelines using PySpark.
- Build efficient data transformations using Python, PySpark, and SQL.
- Process and manage large volumes of structured and unstructured data.
- Optimize Spark jobs, data processing performance, and resource utilization.
- Troubleshoot complex data pipeline and processing issues.
- Design reliable and maintainable data engineering solutions.
- Implement data quality and validation processes.
- Work effectively with technical and business stakeholders.
- Apply best practices for code quality, scalability, maintainability, and reliability.
- Continuously improve data engineering processes and solutions.
What We Bring to the Table:
- Opportunity to work on enterprise-scale data engineering initiatives in Spain.
- Exposure to modern data platforms, PySpark, cloud technologies, and large-scale data processing.
- A collaborative Agile environment focused on technical excellence and innovation.
- Opportunities to work on complex data pipelines and data transformation solutions.
- Continuous learning and opportunities for technical and professional growth.
- A culture focused on quality, ownership, scalability, and sustainable data engineering solutions.
Let’s Connect:
Want to discuss this opportunity in more detail? Feel free to reach out.
Recruiter: Hema Murali
Phone: +31 20 369 0609 ; Extn :148
LinkedIn: https://www.linkedin.com/in/hema-murali-315999329/
Skills Required
- 5+ years of professional experience in Data Engineering or a related field
- Strong hands-on experience with PySpark
- Strong proficiency in Python, PySpark, and SQL
- Experience developing and maintaining batch and streaming data pipelines
- Experience with large-scale data processing and transformation
- Understanding of data modeling, data integration, and data optimization
- Experience with cloud-based data platforms and modern data engineering architectures
- Experience with data quality, validation, troubleshooting, and performance optimization
- Familiarity with CI/CD and modern software engineering practices
- Experience working in Agile development and delivery environments
- Strong communication, collaboration, analytical, and problem-solving skills
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
STAFIDE is a Netherlands-based niche technology talent consulting company operating across Europe. It helps organizations identify, recruit, and deploy technology professionals in areas including cybersecurity, cloud engineering, software development, data analytics, ERP, infrastructure, and digital transformation. Its services include recruitment, workforce engagement, secondment, onboarding support, workforce deployment, and workforce analytics that support technology hiring and expansion.
.png){kind=logo}
.jpg)
