About the Role
We are seeking a highly skilled Senior Data Engineer to join our team and drive the development of our data platform. This role will focus on building scalable data pipelines, optimizing data storage, and ensuring data security and governance. You will collaborate with product teams and other engineering teams to build efficient, high-performance data solutions.
Key Responsibilities
- Design, develop, and optimize scalable ETL/ELT pipelines using Databricks and Apache Spark for both batch and streaming data processing.
- Develop and maintain data models that support analytical and operational workloads.
- Collaborate with product teams and other engineering teams to understand data requirements and design robust solutions.
- Ensure data security, access control, and governance frameworks are implemented and adhered to.
- Monitor data quality and performance, identifying areas for optimization and improvement.
- Ensure code quality and implement test cases on data pipeline codes.
- Follow the latest trends/updates in the data side across the industry and validates its applications on improving our data platform.
Requirements
Required Skills
- 5+ years of experience in data engineering with a focus on large-scale data processing.
- Expertise in Databricks, Apache Spark, and SQL.
- Strong experience in batch and streaming data processing.
- Strong experience in Python and PySpark for data processing and automation.
- Strong experience in data modeling for both Operational and Analytical data.
- Strong problem-solving skills and ability to optimize performance in distributed data environments.
- Hands-on experience with data security, access control, and governance frameworks.
- Experience with Apache Druid/ MSSQL/ MySQL/ MongoDB/ Oracle/ PostgreSQL.
- Good to have experience with Graph Databases to manage complex relationships within data
- Good to have experience in working with Open-Source community.
Benefits
- Comprehensive Medical Insurance – Covers the entire family, ensuring health and well-being for employees and their loved ones.
- Hybrid Work Culture – Only 3 days in-office per week, offering flexibility and reducing commute stress.
- Healthy Work-Life Balance – Prioritizes employee well-being, allowing time for personal and professional growth.
- Friendly Working Environment – A supportive and collaborative workplace culture that fosters positivity and teamwork.
- No Variable Pay Deductions – Guaranteed full compensation with no unpredictable pay cuts, ensuring financial stability.
- Certification Reimbursement - Company will reimburse the money required to take certificates based on project demands
Skills Required
- 5+ years of experience in data engineering focused on large-scale data processing
- Expertise with Databricks, Apache Spark, and SQL
- Experience with batch and streaming data processing
- Strong experience with Python and PySpark for data processing and automation
- Strong experience with operational and analytical data modeling
- Strong problem-solving skills and ability to optimize distributed data environments
- Hands-on experience with data security, access control, and governance frameworks
- Experience with Apache Druid, MSSQL, MySQL, MongoDB, Oracle, or PostgreSQL
- Experience with graph databases
- Experience working with open-source communities
What We Do
Zero Pixels is a Kochi-based technology company founded in 2021 that provides software development and data analytics services. Its capabilities include data engineering, data warehousing, analytics, cloud infrastructure, platform and product development, QA automation, and related technology solutions. The company focuses on helping organizations achieve digital transformation through unified data platforms, innovative software, and scalable, quality-focused technology services.
