Copy of Senior DataOps Engineer

N-iX is looking for Senior DataOps Engineer to join the team

Client Overview:
Our client is an Azerbaijani telecommunications company, the largest mobile network operator in Azerbaijan. The main products are: Fixed telephony, Mobile telephony, Internet services, Wireless broadband, and Value-added services.

Project Objectives:
The primary goal is to accelerate the client’s Data & AI initiatives via a secure, hybrid cloud foundation on AWS while systematically modernizing the IT estate as part of the cloud migration.

Key Project Objectives include:

• Cloud Foundation & Landing Zone: Deploy target hybrid network architectures, establishing a secure Landing Zone and hybrid Data/AI platforms on AWS.
• Security, Compliance & Governance: Operationalize on-prem tokenization (achieving zero raw PII in the cloud), resolve policy blockers to include AWS in the ISMS, and establish a Cloud Center of Excellence (CCoE) to govern Cloud adoption.
• AI Chatbot & Voicebot Design & Implementation: Develop and operationalize a flagship Customer Care Chatbot and Voicebot as the first hybrid-setup consumer.

Key Responsibilities:

• Design, implement, and maintain the AWS Data Platform foundation, including Amazon S3 lake layout, Apache Iceberg table format standardization, and AWS Glue Data Catalog integration.
• Build and optimize scalable batch and streaming data pipelines using Amazon EMR (Serverless and EMR-on-EKS), Apache Spark/PySpark, Apache Flink, and Amazon Athena with workgroup cost caps.
• Implement data tokenization and de-identification pipelines on the data side (PA-T) using Protegrity/Spark UDFs for batch, Spark Streaming for micro-batch, and Kafka Connect SMT for streaming PII masking prior to cloud transit.
• Build and manage real-time streaming architectures with Amazon MSK (Managed Streaming for Apache Kafka), MSK Replicator/MirrorMaker2, and Schema Registry integration.
• Implement Medallion Architecture (Bronze, Silver, Gold layers) for lakehouse data modeling, automating Iceberg table registration and backfill frameworks.
• Execute data migration waves across non-PII and PII datasets using AWS DataSync, automated register steps, and streaming migration configs.
• Configure data access control and governance models using AWS Lake Formation, cross-engine authorization, AWS Macie for PII detection, and Informatica Data Catalog (Axon, EDC, BDQ) integration.
• Build automated schema-drift identification components, GitLab CI/CD pipeline integration for dataset synchronization, and automated data contracts/circuit breakers.
• Implement DataOps observability, centralizing logging via Amazon CloudWatch, configuring FinOps cost & anomaly monitoring, and setting up automated alerts.
• Author technical documentation, operational runbooks, disaster recovery (DR) procedures, and cutover/rollback playbooks for data platform hardening.

Requirements:

Mandatory Technical Skills:

• 4+ years of hands-on experience as a Data Engineer or DataOps Engineer building enterprise-grade data platforms and pipelines.
• Strong expertise with AWS Data Analytics stack: Amazon S3, AWS Glue Data Catalog, AWS Lake Formation, Amazon EMR (EMR-on-EKS / Serverless), Amazon Athena, AWS DataSync, and Amazon MSK.
• Deep experience with Apache Iceberg table format, cataloging, compaction, and schema evolution.
• Proficient in Apache Spark / PySpark and Spark Streaming for batch, micro-batch, and real-time data processing.
• Strong experience in Medallion Lakehouse Architecture design and implementation (Bronze, Silver, Gold layers).
• Practical experience in implementing data tokenization and encryption at scale (e.g., Protegrity, Thales, FPE, or Spark UDF-based de-identification pipelines).
• Solid knowledge of event-driven architectures & streaming: Apache Kafka / Amazon MSK, Kafka Connect (SMT), Schema Registry, and MirrorMaker2 / MSK Replicator.
• Expertise with Relational Databases (Amazon RDS, PostgreSQL, Oracle) and data synchronization techniques.
• Hands-on experience with DataOps CI/CD & Automation: GitLab CI/CD, Infrastructure-as-Code (Terraform / AWS CDK), schema-drift detection, and data contract validation.
• Familiarity with data governance tools and enterprise data catalogs (e.g., Informatica Axon/EDC, AWS Lake Formation).

Strong Plus (Nice-to-Have Skills):

• AWS Certified Data Analytics – Specialty or AWS Certified Data Engineer – Associate.
• Experience in telecom domain data models, CDR processing, and high-throughput real-time telemetry.
• Experience with cloud-side tokenization/detokenization via Athena UDFs / AWS Lambda.
• Familiarity with containerization (Docker, EKS, Kubernetes) for big data runtimes.
• Experience with AWS Macie and FinOps cost-allocation/anomaly-detection frameworks.

Soft Skills & Team Fit:

• Strong critical thinking, problem-solving, and analytical skills.
• Excellent communication and collaboration skills to work closely with cross-functional teams (Data Science, Cloud/Platform, Security, Governance).
• Results-oriented, proactive mindset with strong ownership of deliverables within an Agile / Scrum framework.
• Upper-Intermediate+ English level (written and spoken).