Senior Software Engineer - Data Engineering Team

About Solidus Labs

At Solidus, we are shaping the financial markets of tomorrow by providing cutting-edge trade surveillance technology that protects investors, enhances transparency, and ensures regulatory compliance across traditional financial assets and crypto markets. 

With over 20 years of experience in developing Wall Street-grade FinTech, our team delivers innovative solutions that financial institutions and regulators worldwide rely on to detect, investigate, and report market manipulation, financial crime, and fraud. Headquartered in Wall Street, with offices in Singapore, Tel Aviv, and London, we safeguard millions of retail and institutional entities globally, monitoring over a trillion events each day.

The Role

We’re looking for a strong Software Engineer with Data Engineering experience. Someone who is proficient in building robust, scalable, maintainable and thoroughly monitored data pipelines on cloud environments.

As a young and ambitious company in an extremely dynamic space, we pride ourselves on being independent, accountable, and organized, have a self-starter attitude, and be willing to get their hands dirty with day-to-day work that might be out of their official scope, while keeping an eye on their goals and the big picture.

Responsibilities

• Design and develop the data's team micro services - Java services running on K8S.
• Scale our data pipeline to support the processing of billions of events in both low-latency real-time and T+1 batch using advanced technologies like Clickhouse, Spark, and Kafka. 
• Tackle data duplication, velocity, schema adherence (and schema versioning), high availability, data governance, and more.
• Develop and maintain our data pipeline written mostly in Java and running on K8S in a micro-service architecture.
• Plan and communicate integrations with other teams that consume the data and use it for insights creation.
• Ongoing improvement of the way data is stored and served. Improve queries and data formats to make sure the data is optimized for consumption by a variety of clients.
• Query optimizations in our data warehouses, ensuring data completeness and reliability at scale.

Requirements

• BSc. in Computer Sciences from a top university, or equivalent.
• Strong background as a software engineer with at least 3+ years experience with Java.
• 5+ years in data engineering, and data pipeline development on high-volume, low-latency production environments.
• Experience working in low-latency, real-time systems processing billions of events a day
• Experience with data-engineering cloud technologies as Apache Airflow, K8S, Clickhouse, Snowflake, Redis, Spark, Caching technologies and Kafka.
• Proven experience with relational and non-relational DBs. ExpertProficient in SQL and query optimizations - preferably in clickhouse.
• Experience with monitoring systems (Prometheus, Grafana, Zabbix, Datadog).
• Experience in object-oriented development. Should have strong software engineering foundations.
• Curiosity, ability to work independently and proactively identify solutions. 
• Excellent verbal and written communication skills in a remote environment.