The Role
Looking for a Software Engineer with 3+ years of experience in web-scraping at scale and proficient in Python. The role involves gathering data from the web at scale and working on very large distributed systems.
Summary Generated by Built In
We are looking for people with experience gathering data from the web at scale. You’ll be part of Luma’s infrastructure team, working cross-functionally with Applied AI and Research on mission critical work-streams.
Responsibilities
- Own / lead engineering projects for data acquisition.
- Work with Research and Applied AI to ensure their data and technical needs are met.
- Develop and deploy highly scalable distributed systems capable of handling petabytes of data.
- Build and gather high quality datasets by collaborating closely with the research team.
- Build and maintain backend services for data storage.
- Architect and implement algorithms for data management including indexing & search.
- Conduct and analyze system performance experiments.
Experience
- Proficiency in Python.
- Experience of 5+ years in web-scraping at-scale.
- Experience building and working efficiently in very large distributed systems.
- Experience working with Kubernetes, proxies and other relevant tools.
- Experience working with research datasets.
Top Skills
Python
The Company
What We Do
Luma is a multimedia platform that delivers personalized movie and TV program selections from a range of sources to its viewers.
