Data Engineer

Posted 9 Days Ago
Mountain View, CA
Hybrid
Mid level
Artificial Intelligence • Cloud • Machine Learning • Software • Database
The Role
As a Data Engineer, you will build and optimize production-grade data solutions on AWS or GCP, develop scalable ETL pipelines, and manage data storage and transformation routines. You will work closely with the Machine Learning and Data Platform team and engage with both technical and business stakeholders.
Summary Generated by Built In

The global data management software market is set to reach $137.6 billion by 2026, and we're on a mission to make a significant impact. We're seeking intellectually curious and highly motivated Data Engineers to become foundational members of our Machine Learning and Data Platform team.

Required Qualifications for Ideal candidate

  • 4+ years of professional experience in SaaS/Enterprise companies 
  • Strong experience with data ingestion and connectors
  • Experience in building end-to-end production-grade data solutions on AWS or GCP
  • Experience in building scalable ETL pipelines.
  • Ability to plan effective data storage, security, sharing, and publishing within an organization.
  • Experience in developing batch ingestion and data transformation routines using ETL tools.
  • Familiarity with AWS services such as S3, Kinesis, EMR, Lambda, Athena, Glue, IAM, RDS.
  • Proficiency in several programming languages (Python, Scala, Java).
  • Familiarity with orchestration tools such as Temporal, Airflow, Luigi, etc.
  • Self-starter, motivated, with the ability to structure complex problems and develop solutions.
  • Excellent communication skills and ability to explain data and analytics strengths and weaknesses to both technical and senior business stakeholders.

Preferred Qualifications - good to have

  • Deep familiarity with Spark and/or Hive
  • Understanding of different storage formats like Parquet, Avro, Arrow, and JSON and when to use each
  • Understanding of schema designs like normalization vs. denormalization.
  • Proficiency in Kubernetes, and Terraform.
  • Azure, ADF and/or Databricks skills
  • Experience with integrating, transforming, and consolidating data from various data systems into analytics solutions
  • Good understanding of databases, SQL, ETL tools/techniques, data profiling and modeling
  • Strong communications skills and client engagement

We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

Top Skills

Java
Python
Scala
The Company
HQ: Mountain View, CA
38 Employees
On-site Workplace
Year Founded: 2021

What We Do

Democratizing AI on the Modern Data Stack!

The team behind PyG (PyG.org) is working on a turn-key solution for AI over large scale data warehouses. We believe the future of ML is a seamless integration between modern cloud data warehouses and AI algorithms. Our ML infrastructure massively simplifies the training and deployment of ML models on complex data.

With over 40,000 monthly downloads and nearly 13,000 Github stars, PyG is the ultimate platform for training and development of Graph Neural Network (GNN) architectures. GNNs -- one of the hottest areas of machine learning now -- are a class of deep learning models that generalize Transformer and CNN architectures and enable us to apply the power of deep learning to complex data. GNNs are unique in a sense that they can be applied to data of different shapes and modalities.

Similar Jobs

Atlassian Logo Atlassian

Principal Data Engineer

Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Remote
San Francisco, CA, USA
11000 Employees
136K-218K Annually

Atlassian Logo Atlassian

Data Engineer Intern, 2025 Summer U.S.

Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Remote
San Francisco, CA, USA
11000 Employees

BlackLine Logo BlackLine

Principal Data Engineer - Snowflake

Cloud • Fintech • Information Technology • Machine Learning • Software • App development • Generative AI
Hybrid
Pleasanton, CA, USA
1810 Employees
201K-269K Annually

Snap Inc. Logo Snap Inc.

Data Engineer, 3+ Years of Experience

Artificial Intelligence • Cloud • Machine Learning • Mobile • Software • Virtual Reality • App development
5 Locations
5000 Employees
129K-228K Annually

Similar Companies Hiring

InCommodities Thumbnail
Renewable Energy • Machine Learning • Information Technology • Energy • Automation • Analytics
Austin, TX
234 Employees
RunPod Thumbnail
Software • Infrastructure as a Service (IaaS) • Cloud • Artificial Intelligence
Charlotte, North Carolina
53 Employees
Hedra Thumbnail
Software • News + Entertainment • Marketing Tech • Generative AI • Enterprise Web • Digital Media • Consumer Web
San Francisco, CA
14 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account