We are looking for a Data Platform Engineer to join our small, dynamic team at a fast-growing company delivering a cutting-edge video streaming OS. You will play a key role in both maintaining our legacy AWS-based data streaming infrastructure and contributing to the migration to a modern Azure-based platform.
Responsibilities- Maintain & Optimize (AWS Infrastructure).
- Monitor and manage AWS MSK (Managed Streaming for Apache Kafka) clusters: broker health, partition rebalancing, consumer lag, throughput optimization.
- Administer AWS EMR (Elastic MapReduce) clusters running Hadoop, Spark, Hive, and Tez: cluster scaling, node health, resource allocation, job scheduling.
- Build & Automate (Azure Migration).
- Implement infrastructure-as-code using Terraform based on architectural designs provided by the team lead and architect.
- Build and deploy cloud resources on Azure.
- 1-4 years of hands-on experience in data platform engineering, site reliability engineering (SRE), DevOps, or distributed systems administration.
- Cloud infrastructure expertise with AWS (required) and/or Azure: not just using services, but configuring, tuning, and troubleshooting them.
- Kafka/MSK experience: understanding of topics, partitions, consumer groups, replication, broker configurations, and performance tuning.
- Hadoop ecosystem administration: HDFS, YARN, MapReduce, Hive, or Spark cluster management and troubleshooting.
- Linux/Unix system administration: command-line proficiency, shell scripting (Bash), process monitoring, log analysis.
- Infrastructure-as-Code: Terraform (preferred) or similar tools (CloudFormation, ARM templates, Pulumi).
- CI/CD and automation: GitLab CI, Jenkins, or similar; building pipelines for infrastructure deployments.
- Experience with distributed system monitoring and debugging: understanding logs, metrics, traces, resource contention, and performance bottlenecks.
- Comfortable with networking concepts: VPCs, subnets, security groups, load balancers, DNS, firewalls.
- Experience migrating workloads between cloud providers.
- Exposure to Azure Spark deployment options.
- Familiarity with container orchestration.
- Flexible work arrangements.
- 20 working days per year is a Non-Operational Allowance and settled to be use for personal recreation matters and are compensated in full.
- Collaborative and supportive team culture.
- Truly competitive salary.
- Help and support from our caring HR team.
Top Skills
What We Do
Globaldev Group provides end-to-end software development services and builds skillful teams of specialists to fuel your business’ growth. Leveraging our extensive 12-year background and deep industry knowledge, we have established ourselves as a reliable partner for numerous enterprises, SMEs, and startups.
Globaldev Group is an ISO/IEC 27001 certified company.
▪️️ Global Teams ▪️
Globaldev offers R&D team extensions to both emerging technology companies and global corporations. Empowered by our global talent pool, we excel at connecting our clients with high-skilled specialists.
Hiring, onboarding, and continuous management are all included in our comprehensive set of services, enabling our clients to quickly scale their teams and achieve their product development and implementation goals.
▪️ Global Engineering ▪️
- Obtain a custom-designed solution from scratch
- Scale or enhance your current software infrastructure
- Stay ahead with the latest cutting-edge technologies
- Access a comprehensive range of services from a single vendor
As a full-stack development company, we are equipped to fulfill all your business requirements. Whether it's product discovery, proof of concept, UX design, or development, Globaldev serves as your all-in-one solution provider.
