Data Engineer (Scala, Spark) - R01547940

Primary Skills

• MapReduce, HDFS, Nifi, Spark - Pyspark, ETL Fundamentals, SOLR, Model Provisioning: Kubernetes, Kibana, SQL, ELK - LogStash, SQL (Basic + Advanced), Spark - Scala, Python, AWS Elastic Cache, Spark - Java, Data Warehousing, Kafka, Hive, Flume, Storm, Flink, Modern Data Platform Fundamentals, Data Modelling Fundamentals, Confluent Kafka, PLSQL, T-SQL, Stored Procedures, Oozie

Specialization

• Big Data Architecture: Senior Lead Data Engineer

Job requirements

• Join us as we build a next-generation Enterprise Analytics Platform to reengineer a real estate enterprise. As a Senior Platform Engineer, you will be joining our Enterprise Analytics Platform Team to build EAP platform from ground up to serve agents, brokers, home buyers and sellers.  
• Your Experience 
• 5+ years of experience in large scale real-time stream processing using Apache Spark with messaging infrastructure like Kafka/Pulsar.
• 7+ years of experience on Data Pipeline development, ETL and processing of structured and unstructured data.
• 5+ years of experience using NoSQL systems like MongoDB, DynamoDB and SnowFlake /Athena.
• Experience with technologies like EMR, S3, Lambda, AWS Fargate, CloudWatch, DataDog.
• Experience owning and implementing technical/data solutions or pipelines.
• Excellent written and verbal communication skills in English.
• Strong work ethic and entrepreneurial spirit.
• Additional Information - Main skills set that are a must are:1. Spark with Scala or Python
• 2. Apache Kafka
• 3.Snowflake and No-SQL DB experience like MongoDB
• 4. Experience in AWS cloud services.
• 5. Solid understanding of Medallion architecture, Parquet/ Iceberg
• Behavioral Competences 
• Excellent communication ability to communicate on different levels of seniority. 
• Team player; enjoying being part of a cross-functional setup. 
• Ability to perform well on time-critical endeavors and on multiple fronts at the time. 
• Strong dedication to quality and a 'client' mindset. 
• A passion for learning and continuous improvement. 
• Know more about:
• DAI: https://www.brillio.com/services-data-analytics/
• Know what it’s like to work and grow at Brillio: https://www.brillio.com/join-us/
•  
• Equal Employment Opportunity Declaration
• Brillio is an equal opportunity employer to all, regardless of age, ancestry, colour, disability (mental and physical), exercising the right to family care and medical leave, gender, gender expression, gender identity, genetic information, marital status, medical condition, military or veteran status, national origin, political affiliation, race, religious creed, sex (includes pregnancy, childbirth, breastfeeding, and related medical conditions), and sexual orientation.