Staff Data Engineer (Scala, Spark, & Gen AI)

Reposted 7 Days Ago
Easy Apply
Los Angeles, CA, USA
Hybrid
210K-230K Annually
Senior level
AdTech • Big Data • Cloud • Marketing Tech • Software • Analytics
DISQO is an audience insights platform where members, real people, share information that improves human experience.
The Role
As a Staff Data Engineer at DISQO, you will lead the architecture of data pipelines, integrate Generative AI capabilities, and mentor engineers, focusing on scalable solutions and data quality.
Summary Generated by Built In

DISQO is a leading provider of advertising intelligence, measuring brand and performance outcomes across every media channel to power data-driven marketing decisions. Trusted by 500+ of the world’s largest brands and 150+ agency and media partners, and recognized by Inc., Deloitte, Ad Age, Digiday, Forbes, and Cynopsis, DISQO is redefining the power of measurement in advertising. 

Joining DISQO Nation means being part of a team that moves fast, thinks boldly, and is passionate about solving meaningful problems. Innovation isn't just something we talk about, it's how we operate. We challenge assumptions, embrace new ideas, and continuously push ourselves to build better solutions for our customers and each other.

Our values guide how we work and win together. We believe in winning as one team, fostering a culture of collaboration, trust, and shared accountability. We pursue outsized impact by focusing on opportunities that drive meaningful results for our customers and our business. We champion the customer by putting their needs at the center of every decision and delivering solutions that create real value. And we are relentlessly all in, bringing energy, passion, and commitment to every challenge, every day.

If you're energized by high-growth environments, motivated by innovation, and excited by the opportunity to do impactful work alongside talented, driven teammates, you'll feel right at home at DISQO.

What you will do:

  • Architect and Lead: Design, build, and maintain highly scalable, fault-tolerant data pipelines using expert-level Scala and Apache Spark.

  • Gen AI Integration: Pioneer the use of Generative AI within our data ecosystem—incorporating LLMs to enrich datasets, extract value from unstructured data, automate metadata generation, and build intelligent data products.

  • Cross-Functional Strategy: Partner with Product and Engineering leadership to translate complex business requirements into forward-looking data and AI-augmented architectures.

  • Optimize Systems: Architect and aggressively optimize large-scale ETL/ELT workflows. Dive deep into Spark internals to resolve complex performance bottlenecks, memory issues, and data skew.

  • Modern AI Tooling: Implement and manage infrastructure to support AI integration, including vector databases, embeddings pipelines, and Retrieval-Augmented Generation (RAG) architectures.

  • Set the Standard: Write clean, highly optimized, and maintainable code, while establishing standards for code quality, testing, and system architecture across the organization.

  • Ensure Operational Excellence: Champion data quality, observability, and system health to consistently meet enterprise SLAs and customer commitments.

  • Mentorship: Actively mentor engineers, lead technical design reviews, and foster a culture of continuous learning and technical rigor.

What we're looking for:

  • 8+ years of experience building, architecting, and supporting complex production data pipelines, distributed systems, and backend infrastructure.

  • Expert-Level Scala & Spark: Deep, hands-on expertise in Scala and Apache Spark. You must understand Spark internals, query plans, memory management, and advanced performance tuning for massive-scale batch processing.

  • Applied Generative AI Experience: Proven experience integrating Gen AI / LLMs (e.g., OpenAI APIs, Anthropic, Bedrock) into data products or data engineering workflows. Hands on experience developing with AI dev tools such as Claude code, etc

  • Strong Python Skills: Proficiency in Python specifically to interface with modern AI ecosystems, data APIs, and orchestration tools.

  • Cloud Mastery: Extensive architectural experience within the AWS ecosystem (EMR, Glue, Athena, S3, Bedrock, etc.).

  • Core Data Foundations: Deep understanding of advanced ETL/ELT concepts, complex data modeling, and performance-tuning SQL.

  • Orchestration: Expert-level experience with workflow orchestration tools such as Airflow.

  • Leadership: Proven track record of leading technical initiatives, making architectural decisions, and mentoring teams in an agile, fast-moving environment.

Nice to have:

  • Experience with Snowflake or other modern cloud data warehouses.

  • Deep exposure to streaming or real-time event processing (Kafka, Flink, Kinesis, etc.).

  • Experience utilizing AI for automated data observability, anomaly detection, or data quality tooling.

  • Background in ad tech, measurement, attribution modeling, or specialized analytics platforms.

Why DISQO?

  • Lead the architecture of intelligent data products that directly influence how the world's top brands measure advertising impact.

  • Work with bleeding-edge data and Gen AI infrastructure at a highly meaningful scale.

  • Shape the technical culture and elevate a talented engineering organization while owning massive-scale production systems.

At DISQO, we pride ourselves on having a positive, performance-oriented workplace that includes a flexible hybrid approach, competitive medical benefits, and an amazing vacation policy. Read more about our culture on Glassdoor.
 
You can learn more about what’s happening at DISQO by visiting the DISQO Company Blog.
 
Perks & Benefits:
 
·100% covered Medical/Dental/Vision for employee, competitive dependent coverage
·Stock options
·401K
·Generous PTO policy
·Team offsites, social events & happy hours
·Life Insurance
·Health FSA
·Catered lunch and fully stocked kitchen
·Paid Maternity/Paternity leave
·Disability Insurance
·Travel Assistance Program
·24/7 Counseling Services offered to Employees
 
Note: The benefits noted above are for full time US based employees only.
 
DISQO is an equal opportunity employer. Discovery, innovation, and growth are possible when we open ourselves to new possibilities, perspectives, and approaches. That’s why, at DISQO, we welcome, support, and empower individuals from diverse backgrounds. Exceptional teams are rooted in extraordinary people, each with a unique story and a compelling set of skills. DISQO does not discriminate against employees based on race, color, religion, sex, national origin, gender identity or expression, age, disability, pregnancy (including childbirth, breastfeeding, or related medical condition), genetic information, protected military or veteran status, sexual orientation, or any other characteristic protected by applicable federal, state or local laws.
 
*Recruiting firms that submit resumes to DISQO without first entering into a written contract will not be entitled to any compensation on candidates referred by that firm.

Skills Required

  • 8+ years of experience building and supporting complex production data pipelines
  • Expert-Level Scala & Spark expertise
  • Proven experience integrating Generative AI / LLMs into data products
  • Proficiency in Python
  • Cloud architectural experience within the AWS ecosystem
  • Deep understanding of advanced ETL/ELT concepts and SQL
  • Expert-level experience with workflow orchestration tools such as Airflow

What the Team is Saying

Howard
Vanja
Siran
Drew
Marina
Annie Saekow
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Glendale, CA
272 Employees
Year Founded: 2015

What We Do

DISQO’s mission is to build the world’s most trusted ad measurement platform that fuels brand growth. The world’s largest brands, agencies, and media companies trust DISQO for expert insight and AI-driven intelligence about their advertising performance across all platforms. We capture people’s sentiments and journeys, connecting them with the brands they value and the media they consume. With this identity-based approach, brands gain more accurate and authentic insight so they can create more meaningful interactions. Founded in 2015 and headquartered in Los Angeles, DISQO is recognized as a hyper-growth tech startup and one of the best places to work in the US, with more than 270 team members globally. Follow @DISQO on LinkedIn and Twitter/X.

Why Work With Us

At DISQO, we don’t just hire talent—we champion it. We unlock potential, fuel growth, and raise the bar. Our culture thrives on curiosity, creativity, and courage. Respect is non-negotiable, collaboration is instinctive, and impact is expected. Here, you grow, lead, and redefine what’s possible.

Gallery

Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery

DISQO Offices

Hybrid Workspace

Employees engage in a combination of remote and on-site work.

In 2023, we implemented a structured hybrid model for employees who live within 50 miles of any of our physical offices (Glendale, CA/New York, NY/Yerevan, Armenia). All other employees are encouraged to visit offices.

Typical time on-site: Flexible
HQGlendale, CA
New York, NY
Learn more

Similar Jobs

DISQO Logo DISQO

Business Development Representative

AdTech • Big Data • Cloud • Marketing Tech • Software • Analytics
Easy Apply
Hybrid
Los Angeles, CA, USA
272 Employees
70K-80K Annually

DISQO Logo DISQO

Sr. Insights Manager - Ad Measurement

AdTech • Big Data • Cloud • Marketing Tech • Software • Analytics
Easy Apply
Hybrid
Los Angeles, CA, USA
272 Employees
110K-130K Annually

DISQO Logo DISQO

Senior Security Engineer

AdTech • Big Data • Cloud • Marketing Tech • Software • Analytics
Easy Apply
Hybrid
Los Angeles, CA, USA
272 Employees
180K-200K Annually

DISQO Logo DISQO

Counsel

AdTech • Big Data • Cloud • Marketing Tech • Software • Analytics
Easy Apply
Hybrid
Los Angeles, CA, USA
272 Employees
225K-275K Annually

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account