Manager, Engineering - Data Engineering | Big Data | People Management

Job Title:

Manager, Engineering - Data Engineering | Big Data | People Management

About Skyhigh Security:

Skyhigh Security is a dynamic, fast-paced, cloud company that is a leader in the security industry.  Our mission is to protect the world’s data, and because of this, we live and breathe security. We value learning at our core, underpinned by openness and transparency. 

Since 2011, organizations have trusted us to provide them with a complete, market-leading security platform built on a modern cloud stack. Our industry-leading suite of products radically simplifies data security through easy-to-use, cloud-based, Zero Trust solutions that are managed in a single dashboard, powered by hundreds of employees across the world. With offices in Santa Clara, Aylesbury, Paderborn, Bengaluru, Sydney, Tokyo and more, our employees are the heart and soul of our company. 

Skyhigh Security Is more than a company; here, when you invest your career with us, we commit to investing in you. We embrace a hybrid work model, creating the flexibility and freedom you need from your work environment to reach your potential. From our employee recognition program, to our ‘Blast Talks' learning series, and team celebrations (we love to have fun!), we strive to be an interactive and engaging place where you can be your authentic self. 

We are on these too! Follow us on LinkedIn and Twitter@SkyhighSecurity.

Role Overview:

We are seeking an experienced and strategic Data Engineering Manager to lead our team. The ideal candidate will have a strong background in big data architecture, cloud-native services, and team leadership. You will be responsible for building and optimizing our data platforms, ensuring data security and privacy, and integrating emerging technologies like AI/LLMs to enhance our analytics capabilities. This role requires a blend of technical expertise, leadership skills, and a forward-thinking mindset to drive our data initiatives.

Responsibilities:

• Lead, mentor, and grow a team of talented data engineers. Provide technical guidance and conduct code reviews to ensure best practices.

• Architect and develop scalable and robust data pipelines using a blend of big data frameworks (e.g., Spark, Kafka, Flink) and cloud-native services (AWS) to support security analytics use cases.

• Drive CI/CD best practices, infrastructure automation, and performance tuning across distributed environments.

• Evaluate and pilot the use of AI/LLM technologies in data pipelines for tasks like anomaly detection, metadata enrichment, and automation.

• Ensure data security and privacy compliance across all data platforms and processes.

• Evaluate and integrate LLM-based automation and AI-enhanced observability into engineering workflows.

• Collaborate with data scientists, product managers, and business leaders to understand data needs and deliver solutions that drive business value.

• Oversee the design and implementation of various databases, including NoSQL databases like HBase and Cassandra.

What We’re Looking For (Minimum Qualifications)

• 10 to 15 years of experience in big data architecture and engineering, including deep proficiency with the AWS cloud platform, with at least 3-5 years in a leadership or management role.

• Expertise in distributed systems and frameworks such as Apache Spark, Scala, Kafka, and Flink, with experience building production-grade data pipelines.

• Strong programming skills in Java for building scalable data applications.

• Hands-on experience with ETL tools and orchestration systems.

• Solid understanding of data modeling across both relational (PostgreSQL, MySQL) and NoSQL (HBase) databases, as well as performance tuning.

• Demonstrated experience with AWS services including Lambda functions, AWS Step Functions, and CloudFormation (CF).

• Strong interpersonal and communication skills to effectively lead a team and collaborate with a diverse group of stakeholders.

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

What Will Make You Stand Out (Preferred Qualifications)

• Experience integrating AI/ML or LLM frameworks (e.g., LangChain, LlamaIndex) into data workflows.

• Experience implementing CI/CD pipelines with Kubernetes, Docker, and Terraform.

• Knowledge of modern data warehousing (e.g., BigQuery, Snowflake) and data governance principles (GDPR, HIPAA).

Company Benefits and Perks:

We believe that the best solutions are developed by teams who embrace each other's unique experiences, skills, and abilities. We work hard to create a dynamic workforce where we encourage everyone to bring their authentic selves to work every day. We offer a variety of social programs, flexible work hours and family-friendly benefits to all of our employees.

• Retirement Plans

• Medical, Dental and Vision Coverage

• Paid Time Off

• Paid Parental Leave

• Support for Community Involvement

We're serious about our commitment to a workplace where everyone can thrive and contribute to our industry-leading products and customer support, which is why we prohibit discrimination and harassment based on race, color, religion, gender, national origin, age, disability, veteran status, marital status, pregnancy, gender expression or identity, sexual orientation or any other legally protected status.