Lead Data Engineer
At Klaviyo, we value the unique backgrounds, experiences and perspectives each Klaviyo (we call ourselves Klaviyos) brings to our workplace each and every day. We believe everyone deserves a fair shot at success and appreciate the experiences each person brings beyond the traditional job requirements. If you’re a close but not exact match with the description, we hope you’ll still consider applying. Want to learn more about life at Klaviyo? Visit careers.klaviyo.com to see how we empower creators to own their own destiny.
Klaviyo operates a real-time data analytics platform coded primarily in Python that is built for massive scale and hosted on Amazon Web Services (AWS). The event analytics team persists billions of interactions per day to power core, Klaviyo functionality and generate data-driven insights.
We are looking for a Senior Technical Leader with expertise in Data Storage and Data Infrastructure. You will have ownership over defining the evolutionary, technical vision of how Klaviyo models analytical data creating an efficient baseline for powering offline queries and flexible data loading. You will quickly come up to speed on existing pipeline and analytics infrastructure, identify areas of improvement, and adopt industry best-practices for data processing, movement, and management.
What You Will Do
- You will work on a deep analytics system that provides insight into hundreds of terabytes of data.
- You will help architect and design robust and high performance data processing and storage systems leveraging new data models to serve different internal and external use cases. Contribute to open source data processing technologies.
- You will design and develop data pipelines to move data from disparate sources for consolidation and analysis.
- Establish industry leading design patterns around data ingestion, lifecycle management, governance, and consumption.
- Work with other leaders and team members to create a technology roadmap, address technical debt, and put together an execution plan.
- You will design and develop data schemas and implementation strategies to provide fast insight turnaround on petabytes of data.
- You will develop robust monitoring infrastructure and strategy to return real-time visibility into various stages of data handling.
- You will evangelize design and processes for data handling, CI/CD, security, extensibility, maintainability, etc.
- You will be responsible for coaching engineers, managing/reviewing technical documentation and articulating a phased approach to achieving the team's overall technical vision.
What You Will Need
- 7+ years of production experience in designing, creating and maintaining data driven business solutions and solving big data problems using a wide variety of technologies and modern architectures.
- Hands-on experience designing reliable, fault-tolerant, and high performance distributed systems.
- Experience designing systems leveraging:
- Apache Spark
- Cloud-based DFS (eg. AWS S3, Google Storage Buckets, Azure Blob Storage)
- HDFS
- Hive
- You understand data modeling, data access, and data storage, caching, replication, and optimization techniques.
- Experience with automated analysis of data quality and validation.
- Hands-on experience with data transformation, SQL queries, and optimization.
- High proficiency working with large, heterogeneous datasets in building/optimizing data pipelines using ELT, data replication, API access, data virtualization, stream data integration, and emerging technologies.
- Experience partnering with Data Scientists, Engineers, and Product Managers to understand data needs and opportunities.
Get to Know Klaviyo
Klaviyo is a world-leading marketing automation platform dedicated to accelerating revenue and customer connection for online businesses. Klaviyo makes it easy to store, access, analyze and use transactional and behavioral data to power highly-targeted customer and prospect communications. The company's hybrid customer-data and marketing-platform model allows companies to grow by fostering direct relationships with customers, without giving up their valuable data to popular big-tech ad platforms. Over 265,000 innovative companies like Unilever, Custom Ink, Living Proof and Huckberry sell more with Klaviyo. Learn more at www.klaviyo.com.
If you are a Colorado resident and this role is a remote role, you can receive additional information about the compensation and benefits for this role, which we will provide upon request. Requests can be submitted here. Additional information regarding benefits can be found here.
Klaviyo is committed to diversity and to a policy of equal employment opportunity and non-discrimination. We do not discriminate on the basis of race, color, religion, national origin, age, sex, marital status, ancestry, physical or mental disability, veteran status, gender identity, sexual orientation or any other characteristic protected by applicable law.