About Prophecy
Data was the new oil, but getting value out of data has remained out of reach for most organizations. Prophecy is simplifying Data Engineering with a powerful yet simple-to-use platform that fundamentally fixes the longstanding problem of delivering data for Analytics and Machine Learning reliably and with high quality.
Prophecy is trusted by top Fortune 500 firms (including the largest insurer, a top four bank, and two of the largest credit card processors) to be their primary Data Engineering platform as they transition from legacy ETL products to the cloud and Apache Spark. Prophecy is backed by top VCs and is defining a new product category.
Prophecy's Low-Code Platform enables all the data users to build pipelines using a drag-and-drop interface visually. Uniquely, as you do visual programming, Prophecy produces high-quality code, enabling data users with software engineering practices - code on Github, tests, and continuous delivery. This opens up data engineering to a larger set of users who can focus on business value instead of wrestling with the infrastructure.
Position Summary
Prophecy is used by customers to develop data engineering workflows on Apache Spark, Apache Airflow, and Data Warehouses. We're looking for a solid data engineer who can work with our customers, has a product mindset, can write high-quality Spark code, and be articulate in communication.
The role involves the following:
- Be a resident expert in data engineering using Prophecy with Spark & Airflow
- Work hand-in-hand with customer teams as they transition from legacy data systems (on-premise ETL) to Prophecy data engineering in the cloud
- Help our customers (Fortune 500) onboard to the product, answering queries about Prophecy, Spark, and Airflow
- Write high-performance Spark code
- Travel to customer offices occasionally if needed ( 30%)
The impact you will have
Overachiever:
In Prophecy, we encourage and give environments to engineers to overachieve their own limits. Set up your own metrics and improve them iteratively. You are expected to autonomously figure out the solutions to the challenges while supporting your teammates.
Entrepreneurial skills
Own the products end to end as a founder/ceo from the requirement to delivery. We don’t believe in hierarchy or the number of years of experience. Put yourself in the user's shoes, be curious and willing to learn, work in a supportive environment, and create the best product & experience. Manage your own career growth.
What You'll Need
- Must have: Experience in Apache Spark, and you can write Python or Scala code
- Must have: Experience in complex data engineering (ETL) and have solved many performance issues with Spark jobs
- Must have: Experience in a technical customer-facing role - preferably solution engineering, or wanting to do such a role
- Bachelor in CS (preferred)
Benefits and Perks
- Great company culture.
- Competitive compensation.
- Fair and Open Equity awards for everyone.
- Amazing work.
- Private medical insurance.
- Manage your own growth and career.
- Experience building a transformative product.
- High learning opportunity.
- End-to-end ownership of your project.
Our Commitment to Diversity and Inclusion
At Prophecy, we hire for merit and foster an inclusive culture where people from diverse backgrounds can excel and do their best work. We take great care to ensure that our hiring practices are inclusive and meet equal employment opportunity standards. Individuals looking for employment at Prophecy are considered without regard to age, color, disability, ethnicity, family or marital status, gender identity or expression, language, national origin, physical and mental ability, political affiliation, race, religion, sexual orientation, socio-economic status, veteran status, and any other protected characteristics under applicable laws.
Top Skills
What We Do
Prophecy.io is the only low-code data engineering platform. Prophecy democratizes the development and deployment of high-quality data pipelines, uniquely combining visual development with agile software engineering best practices. The developed code is open source and is targeted at Apache Spark & Apache Airflow. Prophecy is headquartered in Silicon Valley.
