Senior Data Engineer

Posted Yesterday
Hiring Remotely in USA
Remote
Senior level
AdTech • Digital Media • Marketing Tech
The Role
Design and maintain scalable data pipelines and ETL processes, optimize identity resolution, implement new business features, ensure data quality, mentor junior engineers, and build analytics frameworks. Work with data science to productionize ML models.
Summary Generated by Built In

About PebblePost

PebblePost is the industry leader in next-generation addressable marketing, enabling brands to engage decision-ready consumers across the online and offline moments that matter via Programmatic Direct Mail.

Fueled by billions of shared 1st-party identity, intent, and transaction signals, PebblePost’s platform enables brands to quickly and easily engage addressable audiences with active purchase intent and measure performance across all points of sale with address-level accuracy. With these powerful audiences and analytics on their side, brands can build a sustainable marketing engine, creating impactful ways to engage consumers and fostering profitable growth with full-funnel solutions tuned to their data and goals.

About the role and the team 

Our data engineering team designs, builds, and maintains robust data pipelines to ingest, transform, and store massive volumes of data from diverse data sources. We leverage this data to build the PebblePost Graph, a sophisticated, proprietary identity graph that enables us to connect user behavior and intent to household data. We apply data quality and governance frameworks to ensure all data adheres to our privacy-by-design framework. You will work across all our data products to deliver innovative features while collaborating with our data scientists and product teams to help uncover insights and drive business growth.

We run a cloud based stack with assets in both AWS and GCP. We use Kubernetes, Jenkins, Terraform, for our CICD infrastructure. We leverage React on top of a Springboot stack with PostgresQL as our primary relational database. The data engineering pipelines are based in S3 using Databricks with Scala/Spark being the tool of choice for our ETL pipelines running on Airflow/Jenkins and Kubernetes. In addition, we have real time streaming pipelines built using Kafka (MSK) and spark streaming for processing behavioral intent signals from our brand customers.

Our reporting infrastructure leverages Databricks delta lake via a combination of tools including AWS Athena, Metabase, Domo. We continuously evolve our architecture to match our growing scale and recently incorporated graph databases and streaming pipelines to evolve our Identity solution.

This role reports to the Senior Director of Data Engineering.

You will:

  • Design, develop, and maintain scalable data pipelines and ETL processes
  • Contribute to ongoing efforts to enhance and maintain the identity graph, including optimizing the identity resolution process to ensure its accuracy and scalability
  • Drive cross-team initiatives to implement new business features and functionality that leverage our data platform, including the identity graph. Ensure alignment between technical solutions and business goals by delivering cohesive, integrated solutions
  • Partner with product managers, data scientists, business stakeholders, and fellow engineers to build high quality data assets while managing cross functional dependencies in an agile development framework
  • Ensure data quality, consistency, and security by implementing robust data validation, monitoring, and governance processes. Improve processes and tooling that result in high quality CI/CD pipelines
  • Mentor and guide junior engineers fostering a culture of continuous learning and improvement through a variety of techniques including peer reviews and tech talks
  •  Build reporting and analytics frameworks that can allow large volumes of data aggregated and analyzed for insights to our brands for tracking the performance of their campaigns
  • Work with our data science team to productionize ML models that leverage LLM and Generative AI techniques

You have:

  • 8+ years of professional experience working with a programming language such as Java or Scala in a relevant industry
  • Built software assets in big data ecosystems such as Spark or Tensorflow 
  • Proficiency in all aspects of SDLC including CICD and automated testing
  • Led projects building large scale ETL and ML pipelines including being able to orchestrate more complex pipelines using DAG based schedules such as Airflow, Kubeflow, Mleap or Sagemaker
  • Written maintainable code, automated testing and performed code reviews
  • Experience in the AWS tech stack including use of Kafka, Lambdas, Glue, Athena, IAM and large scale data management formats and frameworks such as Parquet ORC, Delta Lake, Iceberg or Hudi
  • Broken down product requirements into measurable engineering milestones, and owning them to completion with outcome driven six week roadmap planning increments
  • Experience in data governance at scale with expertise ranging from Relational or NoSQL/Graph databases and working closely with analysts to deliver scalable insights to Business Intelligence teams 
  • Bachelor’s degree in Computer Science or related discipline

Our Benefits

  • Remote-friendly team
  • Unlimited PTO policy
  • Comprehensive medical, dental and vision plans
  • Cell phone reimbursement program
  • Flexible spending (FSA), health savings (HSA), and pre-tax commuter accounts
  • Employee-based 401(k) program
  • Additional voluntary benefit programs available such as life, critical illness, disability, employee assistance and additional buy-up options

The salary range is a reasonable estimate based on aggregate data for all US locations. Any offered salary is determined by a wide range of factors including but not limited to; candidate location, cost of labor, market data/ranges, internal equity, internal salary ranges, applicant’s skills, prior relevant experience, certain degrees and certifications (e.g. JD/technology, for example).

Salary range: Low: $175,000 - High: $215,000

PebblePost is an equal opportunity employer. All employment decisions are made without regard to race, color, age, gender, gender identity or expression, sexual orientation, marital status, pregnancy, religion, citizenship, national origin/ancestry, physical/mental disabilities, military status or any other basis prohibited by law.

Top Skills

Java
Scala
The Company
HQ: New York, New York
90 Employees
On-site Workplace
Year Founded: 2014

What We Do

PebblePost is the world’s leading Digital To Direct Mail marketing platform, helping hundreds of brands to reach consumers at home with timely, relevant mail that activates buying decisions and drives conversions everywhere.

We are the inventors of Programmatic Direct Mail®. Our platform captures online interest and intent data, leveraging advanced targeting, algorithmic optimization, attribution and quantitative analysis to send relevant direct mail within 12-24 hours of a user expressing interest in a product or service. With 100% coverage of all US households and up to 70% match rates thanks to the proprietary data asset fueling our platform, the PebblePost Graph, this truly differentiated channel has become one of the most powerful ways for marketers to acquire and retain high-value customers at scale.

Hundreds of brands in retail, travel, financial services, non-profit, education, wellness and more are seeing unmatched results and speaking on the record about how PebblePost is helping them to break through the noise of over-saturated digital channels, and provide a more flexible and modern alternative to traditional direct mail

Similar Jobs

AVM Consulting Logo AVM Consulting

Sr. Data Engineer with AWS experience

Information Technology • Software • Consulting
Remote
Los Angeles, CA, USA
100 Employees
100K-220K Annually

Two Barrels LLC Logo Two Barrels LLC

Senior Data Engineer

eCommerce • Legal Tech • Professional Services • Software • Data Privacy
Remote
Hybrid
Austin, TX, USA
950 Employees
150K-150K Annually

Two Barrels LLC Logo Two Barrels LLC

Senior Data Engineer

eCommerce • Legal Tech • Professional Services • Software • Data Privacy
Remote
Hybrid
Salt Lake City, UT, USA
950 Employees
150K-150K Annually

Bombora Logo Bombora

Sr. Data Engineer (Reno, NV or NYC, NY or Remote, US)

AdTech • Big Data • Information Technology • Marketing Tech • Sales • Software
Easy Apply
Remote
3 Locations
152 Employees

Similar Companies Hiring

Artlist Thumbnail
Social Media • Other • Music • Digital Media
Tel Aviv, IL
450 Employees
bet365 Thumbnail
Software • Gaming • eSports • Digital Media • Automation
Denver, Colorado
9000 Employees
Hedra Thumbnail
Software • News + Entertainment • Marketing Tech • Generative AI • Enterprise Web • Digital Media • Consumer Web
San Francisco, CA
14 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account