SparkCognition catalyzes sustainable growth for our clients throughout the world with proven artificial intelligence (AI) systems, award-winning machine learning technology, and a multinational team of AI thought leaders. Our clients are trusted with advancing lives, infrastructure, sustainability, and financial systems across the globe. They partner with SparkCognition to understand their industry's most pressing challenges, analyze complex data, empower decision-making, and transform human and industrial productivity with scalable AI solutions to solve the problems that matter most. With our leading-edge artificial intelligence products and solutions, our clients can adapt to a rapidly changing digital landscape, accelerate their business strategies, and reduce environmental impact creating a better, smarter, and more sustainable world.
SparkCognition is seeking a Senior Data Architect/Engineer to help develop the next generation of SparkCognition's Data Platform. They will be part of a cross-product team that includes data scientists and software engineers working to create this solution.
If you were working for us today, you would be:

• Architect and develop complex and high-level architecture plans that require the integration
• of multiple modern data technologies
• Working with data scientists to help understand the needs of the algorithms we use
• Work closely with our various products to understand core data issues that need to be
• solved
• Working to continuously improve the code base
• Ownership of work items. Be self-directed and will find out the most important parts of a
• problem to work on first, while overcoming obstacles to drive to completion
• Help design, code, integrate and test new features
• Collaboration with other team members to help put the goals of the team above any
• individual
• Be willing to pick up whatever knowledge you're missing to get the job done
• Be willing to share what you know and to learn what you don't

You may be a fit for our team if you have:

• 7+ years of software development experience architecting and developing production
• systems
• Experience working with both large batch and high volume streaming data sources
• Experience with "scale-out" type data science models
• Significant Experience working with Distributed File Systems both in cloud environments
• (e.g. S3, Azure Storage) as well as in on-premises environments (e.g. Ceph, GlusterFS,
• Minio)
• Experience with non-RDBMS type data warehouses (e.g. Hadoop + Hive, Cassandra)
• Experience with Message based / Event based systems (e.g. Kafka, Pulsar, RabbitMQ)
• Expert with ANSI SQL
• Have a desire to better yourself and those around you
• Enjoys working on a codebase with other individuals

It would be great if you had:

• Experience scaling data science models to hundreds of nodes, TB of data
• Experience with Python (either 2.x or 3.x)
• Experience with Scala (> 2.10.x is another plus)
• Experience with Avro, Thrift, Parquet, Orc
• Worked on teams using an Agile process (Scrum, Kanban, or some variation)

Equal Opportunity Employer/Protected Veterans/Individuals with Disabilities
The contractor will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. However, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by the employer, or (c) consistent with the contractor's legal duty to furnish information. 41 CFR 60-1.35(c)
#Remote