Join our small and nimble engineering team! We’re looking for an experienced engineer who enjoys working on a variety of different projects and large(ish) data pipelines (100M+ data points collected every month).
Work in our downtown San Francisco office a few times per week
Provide mentorship to your teammates via code and architecture review
Help to improve, maintain and expand our end to end data pipeline
Create and manage data collection/crawling scripts that utilize everything from basic HTTP requests to browser and mobile app automation (deep experience in this area is a huge plus)
Build and maintain automation/scheduling tooling to ensure data is collected on a timely and consistent basis (we’re currently using self hosted Dagster for scheduling)
Create data cleaning and normalization scripts (with some opportunity to integrate ML/LLMs for labeling/cleaning)
Help with miscellaneous DevOps tasks to help manage the infrastructure running all the above
Our codebase is mostly Python with Typescript and Golang used when they make sense. We almost entirely rely on self hosted open source/core services on bare VMs via docker compose.
Proficient (intermediate or higher) in:
Python (e.g. can write a decorator, understand async/await, MRO, contextmanagers)
SQL (e.g. familiar with complex joins, efficiently bulk update 10k+ rows, schema migrations)
Basic understanding of Linux and Docker
Excited to work on a variety of different projects, sometimes many at the same time
Able to execute with minimal supervision
Bonus skills: Web Crawling, Kubernetes, LLM Pipelines, DB Administration, HomeLab/Server Administration, Background in Statistics, Full Stack Web Development, Mobile App Development
Lunch provided when in office
Unlimited PTO
401k
Company paid Platinum PPO health and comparable dental & vision insurance
$160K - $220K salary, 0.5% - 3% equity
15 minute call with CTO about the position, your experience, and career goals
1st 30 minute technical screening call (coding interview)
2nd 30 minute technical screening call (coding interview)
Onsite at our office to further review your skills, learn more about us, and to get a feel for how we’d work together
Offer extended
Skills Required
- Proficient in Python
- Proficient in SQL
- Basic understanding of Linux
- Basic understanding of Docker (including Docker Compose)
- Experience building and maintaining large-scale data pipelines (100M+ data points/month)
- Experience creating and managing data collection and crawling scripts
- Experience building and maintaining automation/scheduling tooling (Dagster)
- Ability to mentor teammates via code and architecture review
- Able to execute with minimal supervision and manage multiple projects
- Familiarity with Typescript
- Familiarity with Golang
- Bonus skills: web crawling/browser/mobile automation, Kubernetes, LLM pipelines, DB administration, server/HomeLab administration, statistics, full-stack or mobile development
What We Do
Spice Data provides enterprise-level restaurant data, licensing this information to leading Fortune 500 companies. Founded in 2019 and based in San Francisco, the company focuses on the restaurant industry, delivering high-quality data solutions designed for large-scale enterprise use to help organizations gain critical insights into the restaurant sector and optimize their business strategies.
