AWS Cloud Engineer - MDM (Master Data Management)
Build high-performance, low-latency APIs for:
- Creating and updating data
- Searching and filtering attributes
- Managing identifiers, relationships, and metadata
- Use Amazon API Gateway (HTTP/REST APIs) and AWS Lambda or container-based runtimes.
- Implement caching layers (API Gateway cache / Redis via ElastiCache) for performance.
Data Layer (PostgreSQL)
- Develop normalized & optimized PostgreSQL schema,
- Implement indexing, partitioning, materialized views, and query tuning.
- Create CDC-based sync for downstream systems as needed.
- Ensure ACID consistency for transactional workloads.
- Real-time Ingestion,
Build real-time pipelines using:
- API Gateway + Lambda
- Implement idempotency, deduplication, and real-time validations.
Security & Compliance
- Implement authentication/authorization using:
- AWS IAM
- API Keys / OAuth2
- Ensure encryption, auditing, logging, PII compliance (POPIA/GDPR).
Observability & Performance
- Implement monitoring using CloudWatch dashboards, X-Ray tracing.
- Optimize API performance
- Scale PostgreSQL using Aurora PostgreSQL, read replicas, and connection pooling.
CI/CD & DevOps
- Build secure pipelines using:
- Azure,
- CloudFormation,
- docker
Core Technical Skills
- AWS API Gateway (REST/HTTP), Lambda, Step Functions, EventBridge, SQS, SNS, PostgreSQL,
- PostgreSQL performance tuning & schema design
- js / Python for API development
- AWS serverless and container-based architectures (Fargate/ECS)
- AWS Glue, S3, Glue Catalog, batch ETL
MDM concepts: MUST HAVE
- matching, deduplication, golden record
- Performance tuning at API, DB, and network layers
- Experience in building microservices or APIs using Docker containers
Top Skills
What We Do
We are an IT consulting company specializing in data engineering, data science & advanced analytics, cloud computing consulting services and data pipeline automation. We were established in 2017, headquartered in South Africa and have over 100 professionals on board. Our main differentiation is a flexible approach to constantly changing business requirements and needs. Our highly qualified engineers and data scientists provide insightful expertise which help us deliver real added-value to our clients.
