Data Engineer
We are seeking a Data Integrations / Big Data Engineer with experience in Apache NiFi, Kafka, and enterprise systems, to design and implement secure, scalable, and real-time data pipelines for a high-profile client implementation project. The ideal candidate will have expertise in managing structured and semi-structured data, integrating legacy and modern systems, and automating data flows across cloud and on-prem environments.
Key Responsibilities:
- Design, develop, and maintain data pipelines for ingesting and transforming data from various sources (including APIs, flat files, DBs).
- Integrate messaging systems like Kafka and RabbitMQ into the pipeline for event streaming and system decoupling.
- Work closely with Business Analysts and BPEs to translate client requirements into technical data flows.
- Implement and manage connectors for REST APIs, and file-based batch processes.
- Ensure data lineage, auditability, and performance tuning of NiFi flows.
- Deploy and monitor data flows in distributed environments using Zookeeper, Kafka, and ElasticSearch for logging and observability.
- Strong experience in designing and implementing data integration solutions for real-time streaming data.
Required Skills:
- 3–5+ years of experience in data integration/engineering roles.
- Experience with Apache NiFi (flow design, templates, version control, custom processors).
- Proficient in Kafka, RabbitMQ, and event-driven architectures.
- Experience integrating with legacy systems ((BAPIs, IDocs, OData).
- Scripting and transformation in Python, Groovy, or Java.
- Familiarity with distributed systems (Zookeeper, ElasticSearch, etc.).
- Experience with CI/CD for NiFi or container-based deployments (Docker, Kubernetes).
Nice to Have:
- Knowledge of security practices (TLS, role-based access in NiFi).
- Hands-on with monitoring tools (Prometheus, Grafana).
- Familiarity with cloud platforms (AWS/GCP/Azure) for data storage or processing.
Other Requirements:
- Eligibility for Top-Level Security Clearance:
- Candidates must be eligible to obtain and maintain security clearance at the highest level, in accordance with applicable national security regulations.
- On-Site Work Requirement:
- This role requires full-time, on-site presence at the client’s premises located in Pretoria. Remote or hybrid work arrangements are not applicable.
Top Skills
What We Do
Mi-C3 International Limited is a Malta based company that was established in 2014 by the CEO and founder, Glen Scott.
The history and the philosophy of Mi-C3 is in its name which is a combination of two acronyms. The genesis of the technology was in supporting Command and Control for organisations that needed technology to improve their situational awareness to manage and secure their facilities. However, our technology enabled Command and Control to be enhanced with Collaboration (C3) as organisations were empowered with multi-directional communications that took command and control to a new level as more decision-making could be automated and information flows organised in a way that reams of data are transformed into meaningful intelligence (Mi).
Generations of the Mi-C3 software started as a project in the early 2000’s as a solution to support a large oil and gas company operating across multiple countries to help manage its complex operation and considerable security risk spread over a wide and poorly accessible geographical area before the Internet of Things (IoT) was widely understood, and has subsequently grown to over 30,000 sites, and monitoring more than 1 million end points.
Over the years, this solution has evolved into our award winning* flagship product, AFFECTLI, and has delivered significant value to clients through the use of a combination of cutting-edge technologies in the fields of Digital Process Transformation, smart workspaces, AI, machine learning, data management and analytics.
AFFECTLI is an established, flagship product of Mi-C3 that won the Malta MCA eBusiness Award for Best B2B Application 2018, was shortlisted for a nomination in the World Summit Awards 2018 under the Business & Commerce category, is ISO 9001:2015; ISO/IEC 20000:2011 and SOC 2 Type II compliant.
