Alternative Path

Data Engineer - Web Scraping

Posted Yesterday

Be an Early Applicant

Hiring Remotely in Gurgaon, Haryana, IND

In-Office or Remote

Mid level

Information Technology • Consulting

The Role

Design and maintain web scrapers for alternative datasets, clean and transform large data sets using Python and Pandas, and ingest data into databases or warehouses. Build scheduled workflows with Airflow, create quality-control checks, maintain alerting systems, investigate data incidents, and develop tools for the web-scraping platform. Collaborate with analysts and cross-functional teams to deliver reliable datasets and data products.

Summary Generated by Built In

This is a remote position.

Alternative Path is seeking skilled software developers to collaborate on client projects with an asset management firm. In this role, you will collaborate with individuals across various company departments, to shape and innovate new products and features for our platform, enhancing existing ones. You will have a large degree of independence and trust, but you won't be isolated; the support of the Engineering team leads, the Product team leads, and every other technology team member is behind you. This is an opportunity to join a team-first meritocracy and help grow an entrepreneurial group inside Alternative Path. You will be asked to contribute, given ownership, and will be expected to make your voice heard.

Role Summary

Performing Web Scraping using various scraping techniques and then utilizing Python’s Pandas library for data cleaning and manipulation. Then ingesting the data into a Database/Warehouse, and scheduling the scrapers using Airflow or other tools.

Requirements

Role Overview

The Web Scraping Team at Alternative Path is seeking a creative and detail-oriented developer to contribute to client projects. The team develops essential applications, datasets, and alerts for various teams within the client's organization, supporting their daily investment decisions. The mission is to maintain operational excellence by delivering high-quality proprietary datasets, timely notifications, and exceptional service. We are seeking someone who is self-motivated, self-sufficient, with a passion for tinkering and a love for automation.

In this role, you will,

Collaborate with analysts to understand and anticipate requirements.
Design, implement, and maintain Web scrapers for a wide variety of alternative datasets.
Perform data cleaning, exploration, transformation, etc., on scraped data.
Collaborate with cross-functional teams to understand data requirements and implement efficient data processing workflows.
Author QC checks to validate data availability and integrity.
Maintain alerting systems and investigate time-sensitive data incidents to ensure smooth day-to-day operations.
Design and implement products and tools to enhance the Web scraping Platform.

Must Have
Bachelor's/master’s degree in computer science or in any related field.
2-4 years of software development experience
Strong Python and SQL/Database skills
Strong expertise in using the Pandas library (Python) is a must
Experience with web technologies (HTML/JS, APIs, etc.)Proven work experience in working with large data sets for Data cleaning, Data transformation, Data manipulation, and Data replacements.
Excellent verbal and written communication skills
Aptitude for designing infrastructure, data products, and tools for Data Scientists

Preferred
Familiarity with scraping and common scraping tools (Selenium, Scrappy, Fiddler, Postman, XPath)
Experience containerizing workloads with Docker (Kubernetes a plus)
Experience with build automation (Jenkins, GitLab CI/CD)
Experience with AWS technologies like S3, RDS, SNS, SQS, Lambda, etc.

Skills Required

Bachelor's or master's degree in computer science or a related field
2-4 years of software development experience
Strong Python skills
Strong SQL and database skills
Strong expertise using the Pandas library
Experience with web technologies, including HTML, JavaScript, and APIs
Experience working with large data sets for data cleaning, transformation, manipulation, and replacement
Excellent verbal and written communication skills
Ability to design infrastructure, data products, and tools for data scientists
Familiarity with web scraping and tools such as Selenium, Scrapy, Fiddler, Postman, and XPath
Experience containerizing workloads with Docker; Kubernetes familiarity is a plus
Experience with build automation using Jenkins or GitLab CI/CD
Experience with AWS technologies including S3, RDS, SNS, SQS, and Lambda

View all jobs at Alternative Path

View Alternative Path Profile

Report Job

Am I A Good Fit?

beta

Get Personalized Job Insights.

Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company

Gurgaon, Haryana

75 Employees

Year Founded: 2020

What We Do

Alternative Path provides Technology, Business Intelligence, and Financial services. We help small to medium sized enterprises establish their presence in India in the technology and finance space. Our teams' decades of experience in setting up of Centers of Excellence (COE) and smaller teams have helped Organizations harness the full potential of skilled and diverse talent in India. Our implementation teams have delivered numerous projects across Technology functions such as Software Quality Assurance, Application Development, Technology Infrastructure, Application Support, and Data Management and Analysis and Finance functions such as Fund Accounting, Management Reporting, Investor Reporting, Reconciliation, Treasury Workflow and Deal Management, and Record to Report processes. As our name, Alternative Path, suggests we were formed to take a different approach, which is rooted deep in a Quality driven delivery process that is missing in typical service engagements large or small. We were formed in 2020 and are based in Gurgaon, India