Lead Spark Data Engineer

Sorry, this job was removed at 08:49 p.m. (CST) on Tuesday, Sep 24, 2024
Hiring Remotely in USA
Remote
185K-235K Annually
5-7 Years Experience
Artificial Intelligence • eCommerce • Software
The Role

About Us

Wizard is revolutionizing the shopping experience using the power of generative AI and rich messaging technologies to build a personalized shopping assistant for every consumer. We scour the entire internet of products and ratings across brands and retailers to find the best products for every consumer’s personalized needs. Using an effortless text-based interface, Wizard AI is always just a text away. The future of shopping is here. Shop smarter with Wizard.

The Role

We seek a Lead Data Engineer to take charge of our data engineering initiatives, focusing on enhancing data collection, storage, and analysis across all of Wizard's dynamic services. This lead position is pivotal to our data infrastructure, enabling data-driven decision-making and supporting our ambitious growth objectives.

Key Responsibilities:

  • Architect and scale a state-of-the-art data infrastructure capable of handling batch and real-time data processing needs with unparalleled performance.
  • Collaborate closely with the data science team to oversee data systems, ensuring accurate monitoring and insightful analysis of business processes.
  • Design and implement robust ETL (Extract, Transform, Load) data pipelines, optimizing data flow and accessibility.
  • Develop comprehensive backend data solutions to bolster microservices architecture, ensuring seamless data integration and management.
  • Engineer and manage integrations with third-party e-commerce platforms, expanding Wizard's data ecosystem and capabilities.

You

  • A Bachelor's degree in Computer Science or a related field, with a solid foundational knowledge of data engineering principles.
  • 7-10 years of software development experience, significantly focusing on data engineering.
  • Extensive experience in building ETL pipelines using tools such as Apache Spark, Databricks, or Hadoop.
  • Proficiency in Python or Java, with a deep understanding of software engineering best practices.
  • Expertise in distributed computing and data modeling, capable of designing scalable data systems.
  • Proficiency with NoSQL databases, including MongoDB, Cassandra, DynamoDB, and CosmosDB.
  • Proficiency in real-time stream processing systems such as Kafka, AWS Kinesis, or GCP Data Flow.
  • Skilled in utilizing caching and search technologies like Redis, Elasticsearch, or Solr.
  • Familiarity with message queuing systems, including RabbitMQ, AWS SQS, or GCP Cloud Tasks.
  • Experience with Delta Lake, Parquet files, and AWS, GCP, or Azure cloud services.
  • A strong advocate for Test Driven Development (TDD) and experienced in version control using Git platforms like GitHub or Bitbucket.

Additional Preferred Qualifications:

  • Exceptional written and verbal communication skills, capable of articulating complex technical concepts clearly and concisely.
  • A collaborative team player, eager to share knowledge and learn from peers, passionate about mentoring junior team members, and leading by example.

The expected salary for this role is $210,000 - $235,000 depending on skills and experiences.

The Company
HQ: New York City, New York
77 Employees
On-site Workplace
Year Founded: 2021

What We Do

Wizard AI is powering the future of commerce through conversation. Our full-service B2B solution empowers brands to sell, market, and engage their customers directly via text, resulting in conversion rates 10x higher than ecommerce.

Cofounded by Marc Lore, Wizard is backed by a $50M Series A round led by NEA and Accel, and is on a mission to build the dominant technology in the space. We're hiring the best talent and we'd love for you to join us! We’re proud of our commitment to diversity and inclusivity—and we invest in your career by creating a workplace that values transparency and flexibility, and offers remote/hybrid work options. To learn more about Wizard, review our open roles, or request a demo, visit www.wizard.com. We'd love to hear from you!

Jobs at Similar Companies

bet365 Logo bet365

Customer Service Representative

Digital Media • Gaming • Software • eSports • Automation
Denver, CO, USA
6100 Employees

Jobba Trade Technologies, Inc. Logo Jobba Trade Technologies, Inc.

Customer Success Specialist

Cloud • Information Technology • Productivity • Professional Services • Software
Hybrid
Chicago, IL, USA
45 Employees

Similar Companies Hiring

TrainingPeaks (A Peaksware Company) Thumbnail
Software • Fitness
Louisville, CO
69 Employees
bet365 Thumbnail
Software • Gaming • eSports • Digital Media • Automation
Denver, Colorado
6100 Employees
Jobba Trade Technologies, Inc. Thumbnail
Software • Professional Services • Productivity • Information Technology • Cloud
Chicago, IL
45 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account