Role: Lead Data Platform Engineer (AWS | Kafka | OpenSearch)
Preferred Location: Ahmedabad, Pune, Indore
Years of Experience: 08 – 12 years
Notice Period – 0-30 Days (Preferred)
Introduction
Our team builds a data platform that reliably collects, processes, and retrieves large-scale IoT data based on AWS. We are seeking senior engineer who can design unobstructed flow even among billions of data records, push data storage performance to the limit, and technically lead service growth.
Key Responsibilities
- High-availability data pipeline design: AWS MSK (Kafka), end-to-end data streaming architecture design and operation using Logstash
- Storage optimization and sharding strategies: Optimizing large-scale search performance through OpenSearch's index design, sharding, and lifecycle (ISM) management
- Data Modeling and Partitioning: Table partitioning and data modeling in PostgreSQL/MySQL, architecture guide for efficient querying
- Performance Tuning and Guidance: Leading SQL optimization across services and spreading technical best practices to backend development teams
- Release Management: Be the gatekeeper during key database / data pipeline release activities, create review checklist, review deployment details and validates details before deployment to ensure zero-error deployments without roll back needs.
- Infrastructure automation: DB schema versioning (such as Flyway) and deployment process automation systems
Requirements
- Over 7 years of experience in data engineering or related backend development
- Extensive experience in operating infrastructure handling large-scale traffic in AWS cloud environments
- Strong capabilities in building real-time data pipelines and troubleshooting using Kafka
- Experience in OpenSearch (Elasticsearch) cluster design and sharding optimization is essential.
- Deep understanding of the internal structure of RDBMS (PostgreSQL/MySQL) and ability to tune complex queries
Preferred Qualifications
- Seamless technical collaboration capabilities: Communication skills to discuss infrastructure optimization with DevOps engineers and provide efficient data modeling and query guides to backend developers
- Global collaboration skills: Able to share technical documents with engineers across the globe and conduct work via video conferences during collaborative work sessions
- IoT Domain Experience: Experience in handling large-scale device messages and building data pipelines
- Ensuring data stability: Experience in designing logic to prevent data loss during failures and ensure consistency verification
- Nice to have è IaC Proficiency: Nice to have Experience coding and automated operations of data infrastructure using Terraform, Ansible, and others
- 4 years of bachelor’s degree or higher in Computer Science or equivalent discipline
What We Offer:
- Competitive Salary
- Work Life Balance
- Professional Development
- Innovative Work Environment.
- Recognition and Rewards
Skills Required
- 7+ years experience in data engineering or related backend development
- Extensive experience operating large-scale infrastructure in AWS
- Strong experience building real-time data pipelines and troubleshooting using Kafka/AWS MSK
- Experience in OpenSearch/Elasticsearch cluster design and sharding optimization
- Deep understanding of RDBMS (PostgreSQL/MySQL) and complex query tuning
- Experience with release management, deployment checklists, and validation for zero-error deployments
- Experience with DB schema versioning and deployment automation (e.g., Flyway)
- Bachelor's degree (4 years) or higher in Computer Science or equivalent
- Communication and cross-team collaboration skills (infrastructure optimization guidance)
- Experience in IoT domain handling large-scale device messages
- Experience designing data loss prevention and consistency verification mechanisms
- IaC proficiency (Terraform, Ansible) and automated data infrastructure operations
Arrow Electronics, Inc. Compensation & Benefits Highlights
The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Arrow Electronics, Inc. and has not been reviewed or approved by Arrow Electronics, Inc..
-
Healthcare Strength — Healthcare offerings are positioned as robust, with multiple medical plan options and access to telemedicine, EAP, and wellbeing programs. Income-banded premium support is described as helping keep the base plan more affordable for lower earners.
-
Leave & Time Off Breadth — Time-off programs include unlimited PTO for U.S. salaried employees alongside accrual-based vacation and sick programs for hourly staff. Paid parental leave is described as available with a defined fully paid period for new parents.
-
Parental & Family Support — Family-focused supports include subsidized back-up childcare and eldercare days and dependent-care spending options. These offerings add practical value beyond base pay, particularly for caregivers.
Arrow Electronics, Inc. Insights
What We Do
A Fortune 500 company, ranked #133 in 2024, with over 22,000 employees worldwide, Arrow guides innovation forward for over 220,000 leading technology manufacturers and service providers. With 2023 sales of $33 billion, Arrow develops technology solutions that improve business and daily life. Arrow.com is the easiest place for innovators to create, make and manage technology.
Why Work With Us
Arrow is much more than products and services. We are a team of many backgrounds in a global ecosystem, working toward one common goal: to help customers create a better tomorrow, where innovation improves the quality of life and the benefits of technology are more accessible to all. Join us in building a better tomorrow for many!
Gallery
