Senior Data Engineer

Posted 16 Days Ago
Sunnyvale, CA
175K-234K Annually
Senior level
Big Data • Cloud • Logistics • Machine Learning • Retail
The Role
The Senior Data Engineer will build large-scale real-time streaming data pipelines, optimize data workflows, and support analytics for retail operations. Key tasks include developing applications using Spark and Python, enhancing data processing pipelines, and ensuring data accuracy. The role also involves working with various data sources and maintaining data conformity across models.
Summary Generated by Built In

What you'll do...

Position: Senior Data Engineer

Job Location: 640 W. California Avenue, Sunnyvale, CA 94086

Duties: Build real time, large volume streaming data pipelines with fault tolerant and resilient systems leveraging Spark. Build robust applications using Spark and Python. Leverage understanding of star/dimensional model Kimball Data Model (physical and logical). Analyze processes and procedures and identify and suggest improvements and innovations for query performance improvements. Work closely with the business teams and stakeholders to support retail store operations with data. Build audits to monitor operations, investigate data anomalies, and execute root cause analysis. Build data blocks to support faster analytics and data mappings. Develop data flows, maintain databases, and work with users for reporting requests and cube development. Map source system data to data warehouse models and maintain data conformity rules across data marts. Adapt and enhance data processing pipelines to accommodate changes in source systems, performance requirements, and data accuracy issues. Optimize data orchestrations and job scheduling to produce real-time data analytics. This position does not supervise employees.

Minimum education and experience required: Master’s degree or equivalent in Computer Science, Engineering (Any), or related field and 2 years of experience in large scale enterprise software development environment; OR Bachelor’s degree or equivalent in Computer Science, Engineering (Any), or related field and 5 years of experience in large scale enterprise software development environment.

Skills required: Experience architecting big data applications that can be deployed on multi-cloud platforms, including Azure and Google Cloud Platform. Experience architecting ETL workflows for real-time data processing at scale. Experience authoring scripts to perform various automation activities, including emailing data and reports to end users and scheduling jobs. Experience extracting transactional data from Oracle databases and Microsoft databases for processing and ingesting data into MPP systems. Experience with Hadoop and Data Lakes for Business Intelligence reporting purposes. Experience designing data processing pipelines using Apache Spark and optimizing for performance. Experience designing data models for real-world use cases following Star Schema and Snowflake Schema. Experience using Object Oriented Programming in Python for file/HTML processing. Experience acquiring data from large disparate data sources, including REST APIs, SQL databases, and NoSQL databases. Experience with CI/CD tools, including Git, Looper, and Jenkins. Experience with workflow orchestration tools, including Airflow. Employer will accept any amount of experience with the required skills.

Salary Range: $174,518/year to $234,000/year. Additional compensation includes annual or quarterly performance incentives. Additional compensation for certain positions may also include: Regional Pay Zone (RPZ) (based on location) and Stock equity incentives.

Benefits: At Walmart, we offer competitive pay as well as performance-based incentive awards and other great benefits for a happier mind, body, and wallet. Health benefits include medical, vision and dental coverage. Financial benefits include 401(k), stock purchase and company-paid life insurance. Paid time off benefits include PTO (including sick leave), parental leave, family care leave, bereavement, jury duty and voting. Other benefits include short-term and long-term disability, education assistance with 100% company paid college degrees, company discounts, military service pay, adoption expense reimbursement, and more.

Eligibility requirements apply to some benefits and may depend on your job classification and length of employment. Benefits are subject to change and may be subject to a specific plan or program terms. For information about benefits and eligibility, see One.Walmart.com.

Wal-Mart is an Equal Opportunity Employer.

Top Skills

Python
Spark
The Company
HQ: Bentonville, AR
578,950 Employees
Hybrid Workplace

What We Do

Walmart has a long history of transforming retail and using technology to deliver innovations that improve how the world shops and empower our 2.2 million associates. It began with Sam Walton and continues today with Global Tech associates working together to power Walmart and lead the next retail disruption. We’re a high-performing, primarily virtual workforce that is human-led and tech-empowered. Our world-class software engineers, data scientists and engineers, cybersecurity professionals, product managers and business service professionals work with top talent on cutting-edge technologies that create unique and innovative experiences for our associates, customers and members across Walmart, Sam’s Club and Walmart International. At Walmart Global Tech, one line of code or bold idea can make life easier for hundreds of millions of people – talk about epic impact at a global scale.

Similar Jobs

Anduril Logo Anduril

Senior Data Engineer

Aerospace • Artificial Intelligence • Hardware • Robotics • Security • Software • Defense
Costa Mesa, CA, USA
1400 Employees
150K-225K Annually

Atlassian Logo Atlassian

Senior Data Engineer

Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Remote
San Francisco, CA, USA
11000 Employees
136K-218K Annually

Crunchyroll Logo Crunchyroll

Senior Data Engineer - Platform Engineering

Digital Media • eCommerce • Gaming • Mobile • News + Entertainment
Hybrid
San Francisco, CA, USA
1200 Employees
167K-209K Annually

CoreWeave Logo CoreWeave

Senior Data Engineer - Compute and Storage Infrastructure

Cloud • Information Technology • Machine Learning
2 Locations
806 Employees

Similar Companies Hiring

Jobba Trade Technologies, Inc. Thumbnail
Software • Professional Services • Productivity • Information Technology • Cloud
Chicago, IL
45 Employees
InCommodities Thumbnail
Renewable Energy • Machine Learning • Information Technology • Energy • Automation • Analytics
Austin, TX
234 Employees
RunPod Thumbnail
Software • Infrastructure as a Service (IaaS) • Cloud • Artificial Intelligence
Charlotte, North Carolina
53 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account