Senior Data Engineer
BigCommerce’s mission is to help merchants sell more at every stage of growth, from small startups, to mid-market businesses to large enterprises. We focus on being the best ecommerce platform so our customers can focus on what matters most: growing their businesses.
We are equally passionate about growing our employee’s careers and providing them an incredible experience as we rapidly expand across the globe. We are proud to have been recognized numerous times for our product and workplace culture. We empower our people and customers to build, innovate and grow, so together we can redefine the ecommerce industry.
BigCommerce, named a 2021 "Best Place to Work" is Austin is looking for a Senior Data Engineer to join our IT/Ops Data Engineering team. Leveraging your big data skills, you will help design and build a Realtime, centralized data distribution service for our core business systems to achieve faster and bigger results. This involves deciphering value from several systems with a variety of access patterns. You will report to Sr. Director of IT and work with cross-functional teams in a highly visible role. The Senior Data Engineer will mentor junior engineers and communicate with senior leadership.
What you’ll do
- Build scalable near real time data pipelines on AWS and GCP that collects, transforms, loads and curate’s data from various internal and external data sources
- Build a scalable distributed data store that will be central source of truth
- Create scalable and low latency solutions
- Implement machine learning pipelines and algorithms
- Evaluate new technologies and build prototypes for continuous improvements in Data Engineering
- Contribute to setting standards and development principles for the team and influence across the larger organization
- Write automated unit, integration, and acceptance tests to support our continuous integration pipelines
- Build performance and load tests written from scalability and resiliency standpoint
- Participate in peer code reviews and advocate for the best coding practices and principles
- Partners with Infrastructure and Engineering teams to ensure instrumentation, logging and monitoring is in place
Team / Leadership Skills
- Actively participate with our development team in all phases of the software development lifecycle, including requirements gathering, functional and technical design, development, testing and roll-out, and support
- Estimate work items and help maintain a scrum backlog
- Distill and present research findings to both technical and non-technical leaders
- Partners with engineering management to determine hiring needs and helps with recruiting activities
Who you are:
- BS or MS in Computer Science, SWE, CE, MIS or equivalent experience or field
- 5+ years developing data and software solutions
- 4+ years of experience in Java, Scala and/or Python
- 4+ years of experience in building data pipelines in the cloud with tools like S3, Hadoop, Hive; or their equivalents
- 4+ years of experience in streaming technology such as Kafka
- Strong experience with AWS and/or GCP
- Strong experience with SQL, ETL, Data Warehousing and Data Orchestration
- Experience with scheduling frameworks, preferably Airflow
- Experience with monitoring near real time scalable data pipelines
- Experience with of all aspects of data systems including database design, ETL, aggregation strategy, performance optimization, and technology trade-offs
- Experience with Terraform, Puppet, and Jenkins
- Experience with shell scripting
- Focused developer with a strong sense of ownership
- Ability to independently drive projects consisting of many stories from inception through to completion and production deployment
- Capable of working closely across multiple teams to ensure data solutions line up with business initiatives and are of high quality
- Excellent written and verbal communication skills
Diversity, Equity & Inclusion at BigCommerce
Our employees make the difference. At BigCommerce, we believe that celebrating the unique histories, perspectives and abilities of every employee makes a difference for our company, our customers and our community. We are an equal opportunity employer and the inclusive atmosphere we build together will make room for every person to contribute, grow and thrive.
We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the interview process, to perform essential job functions and to receive other benefits and privileges of employment. If you need an accommodation in order to interview at BigCommerce, please let us know during any of your interactions with our recruiting team.
Learn more about the BigCommerce team, culture and benefits at https://careers.bigcommerce.com.
