Senior Software Developer - Data Forge

Posted 20 Minutes Ago
Be an Early Applicant
Seattle, WA, USA
In-Office
167K-204K Annually
Senior level
Fintech • Real Estate • PropTech
Careers that Help Everyone Home.
The Role
Own production data products and pipeline families end to end, from source extraction and transformation through schemas, SLAs, documentation, and downstream delivery. Build and migrate Airflow, Iceberg, Spark, Snowflake, and cloud-based pipelines; manage legacy systems, schema drift, backfills, reliability, and cost. Develop operational AI tooling to reduce on-call work, support data quality, and improve diagnostics. Partner with ML, analytics, and product teams while mentoring engineers and participating in production on-call.
Summary Generated by Built In
Data Forge owns the data lake and builds the reusable, production-grade datasets that turn raw company signals into reliable data products for ML, search, analytics, and product teams.
We build on shared infrastructure owned by our sister teams. What we own is the lake itself and what lives in it: the storage architecture, catalog, schemas, SLAs, and the lifecycle of every curated dataset we deliver. Our work shows up in conversational search, home recommendations, agent matching, clickstream analytics, and the ML ranking and relevance features behind Redfin's core product.
Three strategic bets define our roadmap. First, Fill the Lake: we are systematically bringing every meaningful raw signal at Redfin into a single, well-governed data store - migrating legacy systems, onboarding new source systems, and expanding coverage so every team builds on the same underlying data rather than maintaining their own copies. Second, unification of our Snowflake and data lake environments: we are converging on a single Iceberg lakehouse where the same tables are queryable through SQL, Spark, and ML tooling - no split schemas, no duplicate copies for different consumers. Third, accessibility: we are reducing the friction between raw lake data and the feature teams who need it, building the data contracts, discovery patterns, and self-service interfaces that let ML engineers and product teams find, trust, and use lake data without needing a Data Forge engineer in the loop.
We are building operational AI tooling to reduce the on-call overhead that comes with owning a large and growing catalog of production datasets.
About the role
  • Own a data product or pipeline family end to end: understand the source system, design the extraction and transformation, own the schema, and deliver to downstream consumers with clear SLAs and change management. Ownership means you are accountable when it breaks and driving it forward when it doesn't.
  • Build data products that feature teams can actually use: user behavior profiles, property data, clickstream features, agent and customer data - delivered with clear schemas, data contracts, and enough documentation that ML engineers and product teams can build on them without asking you first. The goal is not just correctness but accessibility.
  • Migrate legacy systems to production-grade Airflow and Iceberg pipelines: phase the work carefully, run shadow validation before cutover, deliver clean post-migration cleanup, and retire what you replaced. We do these on live systems with downstream consumers watching.
  • Extract data reliably from complex source systems: Postgres replicas under production load, Kafka consumers (KCL), AWS Lambda-based ingest pipelines, and third-party feeds - handling schema drift, timeout tuning, connection reliability, and backfill strategy.
  • Drive unification of our Snowflake and data lake environments: lead the Glue-to-Snowflake catalog migration in your area, design Iceberg tables that serve both SQL analysts and ML consumers from a single source of truth, and reduce the number of places teams have to look to find the same data.
  • Build operational AI tooling for the data platform: extend our alert triage agent, design diagnostic automation, and reduce on-call toil by building tooling that learns from production signals.
  • Partner with ML, analytics, and product teams to deliver on shared milestones: communicate proactively when timelines shift, unblock downstream consumers quickly, and represent data quality as a first-class concern.

About you
  • You have 5+ years building and operating data pipelines or data products at scale. You've owned production datasets - not just written DAGs - and you understand the difference between a pipeline that runs and a pipeline teams can trust.
  • You are fluent across the full stack data engineering requires: Python and SQL for pipeline logic and transformations; Terraform and Kubernetes for infrastructure management; Spark for distributed processing; Airflow for orchestration. You can work across all of these in the same week without needing a specialist for each layer.
  • You have worked with Iceberg, Hive, or Delta Lake in production: you understand compaction strategy, merge-on-read vs. copy-on-write tradeoffs, partition spec design, and how write concurrency affects table health over time.
  • You have extracted data from complex, legacy source systems - Postgres replicas, Kafka consumers (KCL), AWS Lambda ingest pipelines - and built extraction pipelines that handle schema drift, failures, and large backfills without heroic intervention.
  • You think carefully about data modeling and reusability: not just "does this table answer the question" but "how will downstream teams depend on this, and how do I make it reliable enough that they can build on it without asking me first."
  • You approach legacy migrations methodically: shadow validation, phased cutover, explicit cleanup. You know that "it works in test" is not the same as "it is safe to cut over," and you take that seriously when ML models and product features are downstream.
  • You treat resource constraints as first-class design inputs: Spark executor tuning, pod memory requests, namespace quota management, S3 storage costs. When you own pipelines that run every hour, reliability and cost are not afterthoughts.
  • You are comfortable with AI-assisted development and excited to build operational AI for a data platform: alert triage, automated diagnostics, pipeline scaffolding. You think about feedback loops and how to make tooling smarter from production signals.
  • You have operated production pipelines on-call: you can triage a P2 alert, trace a data quality regression to its source system, and write a postmortem worth reading.
  • You are a force multiplier. You raise the people around you through code review, design feedback, and mentorship - and you communicate clearly with the engineers, analysts, and ML practitioners who depend on your data to do their own work.

What you'll get
Our team members fuel our strategy, innovation and growth, so we ensure the health and well-being of not just you, but your family, too! We go above and beyond to give you the support you need on an individual level and offer all sorts of ways to help you live your best life. We are proud to offer eligible team members perks and health benefits that will help you have peace of mind. Simply put: We've got your back. Check out our full list of Benefits and Perks.
On-Call Expectations
This role may include participation in an on-call rotation to support production systems and ensure service reliability. On-call responsibilities may include coverage during nights and weekends. If applicable, frequency and scheduling will be determined by team needs and communicated accordingly.
About us
Redfin is a technology-driven real estate company with the country's most-visited real estate brokerage website. As part of Rocket Companies (NYSE: RKT), Redfin is creating an integrated homeownership platform from search to close to make the dream of homeownership more affordable and accessible for everyone. Redfin's clients can see homes first with on-demand tours, easily apply for a home loan with Rocket Mortgage, and save thousands in fees while working with a top local agent.
This job description is an outline of the primary responsibilities of this position and may be modified at the discretion of the company at any time. Decisions related to employment are not based on race, color, religion, national origin, sex, physical or mental disability, sexual orientation, gender identity or expression, age, military or veteran status or any other characteristic protected by state or federal law. The company provides reasonable accommodations to qualified individuals with disabilities in accordance with applicable state and federal laws. Applicants requiring reasonable accommodations in completing the application and/or participating in the application process should contact a member of the Human Resources team, at [email protected] .
The compensation information below is provided in compliance with all applicable job posting disclosure requirements. The compensation for this position is $166,900.00-$203,900.00 . The position may also be eligible for an annual bonus, incentives, and other employment-related benefits including, but not limited to, medical, dental, and vision benefits, 401K retirement plan, and paid-time off. More information regarding these benefits and others can be found here . The information regarding compensation and other benefits included in this paragraph is the company's current, good faith estimate at the time of posting. [Compensation and benefits are subject to modification from time to time as the Company, in its sole and exclusive discretion, deems appropriate.] The Company may determine during its future reviews of the proposed compensation and benefits provided for this position, that the compensation and benefits for such position should be reduced. In no event will the Company reduce the compensation for the position to a level below the applicable jurisdictional minimum wage rate for the position. Los Angeles County and San Francisco Candidates only: qualified applicants with arrest or conviction records will be considered for employment per the Fair Chance Ordinance and the Fair Chance Initiative for Hiring.

Skills Required

  • 5+ years building and operating data pipelines or data products at scale
  • Production ownership of datasets and data products
  • Fluency in Python and SQL
  • Experience with Terraform and Kubernetes for infrastructure management
  • Experience with Spark for distributed processing
  • Experience with Airflow orchestration
  • Production experience with Iceberg, Hive, or Delta Lake
  • Experience extracting data from PostgreSQL replicas, Kafka consumers, AWS Lambda pipelines, or similar complex source systems
  • Experience handling schema drift, failures, and large backfills
  • Experience with data modeling, reusable data products, and data contracts
  • Experience with legacy system migrations, shadow validation, phased cutovers, and cleanup
  • Experience tuning Spark executors, Kubernetes pod resources, namespace quotas, and cloud storage costs
  • Experience operating production pipelines on-call and responding to data quality incidents
  • Comfort with AI-assisted development and building operational AI tooling
  • Ability to mentor engineers and communicate effectively with engineering, analytics, and ML teams

Redfin Compensation & Benefits Highlights

  • Healthcare Strength — Health, dental, and vision coverage alongside HSAs/FSAs and life/disability insurance are repeatedly highlighted as robust for eligible employees. Feedback suggests these offerings are a dependable cornerstone of the package.
  • Leave & Time Off Breadth — Paid vacation, holidays, sick time, and flexible vacation for some agent programs are described as meaningful components. Feedback suggests these provisions support taking time away from work when needed.
  • Parental & Family Support — Parental leave and fertility benefits are explicitly included in the offerings. Feedback suggests family-related support stands out as a positive element of the package.

Redfin Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Seattle, Washington
5,800 Employees
Year Founded: 2006

What We Do

Launch, pivot, or accelerate your career with an organization invested in your success. Together, we’re transforming the homeownership journey and redefining what it means to serve clients with heart, innovation, and impact. At Redfin, your work will help make the dream of owning a home more accessible and equitable for everyone. You bring the relationships; we handle the rest. Our Real Estate Agents are at the heart of Redfin. They deliver the expert guidance and local knowledge that helps clients win in a competitive market. Redfin agents focus on what matters most — serving clients and building lasting relationships.

Why Work With Us

At Redfin, Powered by Rocket, your work unlocks a more seamless, personalized experience for buyers and sellers alike. Collaborate with industry leaders, leverage innovative technology and be part of a team that sets a new standard for client-first service.

Gallery

Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery

Redfin Offices

Hybrid Workspace

Employees engage in a combination of remote and on-site work.

Typical time on-site: Not Specified
Company Office Image
HQSeattle
Redfin’s headquarters in Seattle, WA is a collaborative hub where product, engineering, design and support teams build and refine the platform that powers the American Dream. Seattle teams focus on scaling technology, driving innovation, and supporting our hybrid workforce.

Similar Jobs

Redfin Logo Redfin

Senior Software Engineer

Fintech • Real Estate • PropTech
In-Office
Seattle, WA, USA
5800 Employees
167K-204K Annually

Redfin Logo Redfin

Software Engineer

Fintech • Real Estate • PropTech
Hybrid
Seattle, WA, USA
5800 Employees
139K-170K Annually

Redfin Logo Redfin

Principal Software Engineer

Fintech • Real Estate • PropTech
In-Office
Seattle, WA, USA
5800 Employees
234K-286K Annually

Redfin Logo Redfin

Senior Data Scientist

Fintech • Real Estate • PropTech
In-Office
Seattle, WA, USA
5800 Employees
128K-161K Annually

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account