We are looking for a highly skilled Senior Data Engineer with strong expertise in PySpark, Python, and Big Data technologies to design, build, and optimize scalable data platforms and pipelines. The ideal candidate will work closely with Data Scientists, Business Analysts, and Analytics Delivery teams to develop high-performance analytics solutions and support advanced data science initiatives.
RequirementsKey Responsibilities
- Collaborate with Analytics Delivery Leads and Lead Data Engineers to understand business requirements and deliver impactful data solutions.
- Work closely with Data Scientists and cross-functional teams to solve complex business problems through data engineering solutions.
- Manage data onboarding, data access, and stakeholder coordination for analytics initiatives.
- Design, develop, and maintain scalable, secure, and high-performance data pipelines.
- Acquire, ingest, process, and transform large-scale structured and unstructured datasets.
- Implement data engineering best practices for building reliable and production-ready data platforms.
- Perform data wrangling, cleansing, transformation, and feature engineering for machine learning and analytics use cases.
- Design and develop modular data pipelines to generate reusable features and modeling datasets.
- Build and optimize data architectures supporting advanced analytics and machine learning workloads.
- Contribute to data platform design by selecting appropriate technologies across Big Data, SQL, and NoSQL ecosystems.
- Ensure data quality, integrity, governance, security, and scalability across the data lifecycle.
- Participate in Agile squads and collaborate effectively with stakeholders across business and technology teams.
- Contribute to enterprise data architecture strategy and roadmap aligned with business objectives.
- Strong experience in Data Engineering and Big Data solutions.
- Expert-level proficiency in PySpark.
- Strong programming experience in Python.
- Experience building large-scale distributed data processing pipelines.
- Data Ingestion
- Data Transformation
- Data Wrangling
- Data Preparation
- Data Modeling
- Feature Engineering
- ETL/ELT Development
- Batch Processing
- Data Pipeline Development
- Apache Spark / PySpark
- Distributed Data Processing Frameworks
- Big Data Ecosystems
- Strong SQL expertise
- Experience with Relational Databases
- Experience with NoSQL Databases
- Data Warehousing Concepts
- Azure Data Platform
- AWS Data Services
- Google Cloud Data Services
- Databricks
- Hadoop Ecosystem
- Feature Engineering for ML Models
- Data Preparation for Analytics and AI Use Cases
- Building Modeling and Feature Tables
- Supporting Data Science Workloads
- Software Engineering Principles
- Data Pipeline Optimization
- Data Quality Management
- Performance Tuning
- Scalability & Security
- Version Control (Git)
- CI/CD Concepts
- Documentation Standards
- Strong stakeholder management and communication skills.
- Experience working with Business Analysts, Data Scientists, and Product Teams.
- Ability to translate business requirements into scalable data solutions.
- Strong analytical and problem-solving skills.
- Experience working in Agile/Scrum environments.
- Ability to work independently and within cross-functional teams.
- Experience working in enterprise-scale data platforms.
- Experience supporting machine learning and advanced analytics initiatives.
- Exposure to cloud-native data engineering solutions.
- Experience designing modern data architectures and data lake environments.
Skills Required
- Strong experience in Data Engineering and Big Data solutions.
- Expert-level proficiency in PySpark.
- Strong programming experience in Python.
- Experience building large-scale distributed data processing pipelines.
- Strong SQL expertise.
- Experience with Relational Databases.
- Experience with NoSQL Databases.
What We Do
Global Software Solutions Group Veracious product line is a series of robust banking platforms that provide core banking, payment systems, custom process automation, and document management solutions for banks and financial institutions in Middle East & Africa. This cutting-edge product line features the Veracious Payments Hub, Digital Banking and the DMS, all built on the Torus Lowcode development platform software. Global Software Solutions Group is a software solutions provider that aims to solve mission-critical problems that financial institutions face today. Our software solutions bring together our Low Code platform, the payments product line and customized service offerings to solve mission-critical statements in core banking, payments, process automation, and document management. The Payments Hub is GSS's flagship product.
