Web Scraping Engineer - Python (Mid/Senior)

Posted Yesterday
Be an Early Applicant
Vilnius, Vilniaus miesto savivaldybė, Vilniaus apskritis, LTU
Hybrid
42K-73K Annually
Senior level
Big Data • Information Technology
Oxylabs is a web intelligence collection platform, enabling companies worldwide to unlock the power of big data.
The Role
Build and maintain scalable web scrapers and backend systems, design resilient scraping strategies to bypass blocks, improve observability, and develop features for large-scale scraping and parsing infrastructure.
Summary Generated by Built In
We’re a team of 500+ professionals who develop cutting-edge proxy and web data scraping solutions for thousands of the world’s best known businesses, including Fortune 500 companies. 

What’s in store for you:
You’ll be solving complex challenges and maintaining our own infrastructure with 60PB+ monthly data traffic. Here are its scale and maturity in numbers:

- 6PB+ Ceph storage
- 60PB+ monthly data traffic through our systems
- 300k+ service requests/sec processed
- 500k+ Kafka messages/sec streamed

The team waiting for you: 
 
We run one of the most advanced and largest scraping and parsing products in the world. We serve thousands of requests per second with a very high success rate. Our scrapers and parsers are used by leading e-commerce, market intelligence, and AI industry players making the work challenging and truly global. The team is a blend of different interesting personalities from different walks of life and nationalities. Here you can find people who are experts in gaming, playing guitar, riding bicycles, and other areas. We, as a team, will support you in learning how to build your own scrapers and will share all the tips, tricks and hacks we know to ensure that you are onboard in no time.

In this role, you’ll:

  • Develop scalable scrapers.
  • Define resilient scraping strategies, unblock websites for scraping.
  • Improve observability in the system.
  • Develop back-end solutions for scraping & parsing problems of various magnitudes.
  • Maintain the current system and develop new features related to scraping & parsing.

Your skills & experience:

  • Experience working with Python.
  • Understanding of computer science, including data structures, algorithms, computability and complexity.
  • Version Control skills using Git.
  • Knowledge on how to unblock websites for scraping.
  • Is able to use different scraping techniques & open-source tools to build scrapers.
  • Is comfortable with using Dev Tools.
  • Network (TLS/SSL) knowledge.
  • Worked with browser automations.
  • Knows their way around asynchronous programming.
  •  
     

Nice to have:

  • Web development knowledge.
  • Knows how to use CSS Selectors / XPaths for parsing.
  • Experience working with Go & C++.
  • Worked on browser source code.
  • Knowledge of any front-end framework.
  • Experience working with Pydantic, FastAPI, SQLAlchemy.
  • Has experience working with Redis, MySQL, Docker, Kubernetes, Elasticsearch, Kibana and monitoring tools like Grafana, Prometheus.
  • Experience with machine learning that is scraping domain-specific.
  • Has experience in building scalable systems.

Salary & Benefits:

  • Gross salary: 3500-6100 EUR/month. Keep in mind that we are open to discussing a different salary based on your skills and experience.
  • Growth & Learning: 40+ internal learning options, external conferences, mentorship, and year-round knowledge-sharing.
  • Health & Well-being: Private health insurance, psychotherapy, on-site well-being consultants, 24/7 gym access, and a wellness app.
  • Celebration & Community: Team events, an overseas workation, quarterly team-building budgets, and plenty of ways to mark milestones together.
  • Plus bonus vacation days, paid life-moment days off, barista coffee, and all the tools you need. See the full list of benefits here: https://career.oxylabs.io/blog/our-work-benefits/.

Up for the challenge? Let’s talk!

Skills Required

  • Experience working with Python
  • Understanding of data structures, algorithms, computability and complexity
  • Version control using Git
  • Knowledge of how to unblock websites for scraping
  • Ability to use different scraping techniques and open-source tools
  • Comfortable using browser DevTools
  • Network knowledge (TLS/SSL)
  • Experience with browser automation
  • Familiarity with asynchronous programming
  • Web development knowledge
  • CSS Selectors / XPaths for parsing
  • Experience with Go or C++
  • Experience working on browser source code
  • Knowledge of front-end frameworks
  • Experience with Pydantic, FastAPI, SQLAlchemy
  • Experience with Redis, MySQL, Docker, Kubernetes
  • Experience with Elasticsearch, Kibana and monitoring tools like Grafana, Prometheus
  • Experience with machine learning in scraping domain
  • Experience building scalable systems
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Vilnius, Lithuania
500 Employees
Year Founded: 2015

What We Do

Established in 2015, Oxylabs is a web intelligence acquisition solution and premium proxy provider, enabling companies of all sizes to utilize the power of big data. Constant innovation, an extensive patent portfolio, and a focus on ethics have allowed Oxylabs to become a global leader in the web intelligence collection industry and forge close ties with dozens of Fortune Global 500 companies. In 2022 and 2023, Oxylabs was named Europe's fastest-growing web intelligence acquisition company in the Financial Times’ FT 1000 list. Unlocking the power of web intelligence with Oxylabs! As businesses look for ways to create and optimize revenue streams, internal data is being exhausted, forcing companies to turn outwards. Organizations are now competing over external data insights that are fueling the digital economy's growth. Oxylabs proudly stands as a leading force in the web intelligence collection industry. Our innovative and ethical scraping solutions make web intelligence insights accessible to those that seek to become leaders in their own domain. Why choose us? Industry Leaders: As one of the market-leading companies, we offer cutting-edge web scraping solutions backed by the world's largest ethical proxy network. Partnering for success: We view our clients as partners, working hand in hand to maximize the benefits of our collaboration. Your success is our success. Elevate your business with our services: - Market Research: Gain insights that drive informed decisions. - Ad Verification: Ensure real audiences see your ads. - Brand Protection: Safeguard your online reputation. - Travel Fare Aggregation: Optimize travel-related pricing. - SEO Monitoring: Stay ahead in search engine rankings. - Pricing Intelligence: Make data-driven pricing decisions and more. Curious about how Oxylabs can supercharge your business? We're here to help! Drop us a line at [email protected].

Similar Jobs

Cencora Logo Cencora

Customer Relationship and Performance Analyst with French

Healthtech • Logistics • Pharmaceutical
In-Office
Vilnius, Vilniaus miesto savivaldybė, Vilniaus apskritis, LTU
51000 Employees
2K-3K Annually

Cencora Logo Cencora

Data Visualization Specialist

Healthtech • Logistics • Pharmaceutical
In-Office
Vilnius, Vilniaus miesto savivaldybė, Vilniaus apskritis, LTU
51000 Employees
2K-3K Annually

Airwallex Logo Airwallex

Safeguarding and Capital Adequacy Manager - Airwallex (Lithuania)

Artificial Intelligence • Fintech • Payments • Business Intelligence • Financial Services • Generative AI
In-Office
Vilnius, Vilniaus miesto savivaldybė, Vilniaus apskritis, LTU
2200 Employees

Ericsson Logo Ericsson

Design Engineer

Cloud • Information Technology • Internet of Things • Machine Learning • Software • Cybersecurity • Infrastructure as a Service (IaaS)
In-Office or Remote
109 Locations
88000 Employees

Similar Companies Hiring

Scrunch  Thumbnail
Artificial Intelligence • Information Technology • Marketing Tech • Software • SEO
Salt Lake City, Utah
Standard Template Labs Thumbnail
Artificial Intelligence • Information Technology • Software
New York, NY
25 Employees
Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
178 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account