The Role
The Senior Lead Data Engineer will architect and maintain ELT pipelines, manage data infrastructure, optimize SQL queries, and mentor teammates.
Summary Generated by Built In
Senior Lead Data Engineer
Primary Skills
- Architect, construct, test and maintain ELT pipelines using Hadoop and Spark SQL
Specialization
- Hadoop and Spark SQL
Job requirements
- Key Responsibilities:
- Architect, construct , test and maintain ELT pipelines using Hadoop and Spark SQL
- - Manage and optimize the infrastructure ensuring reliability and performance
- -Design and implement data models and schemas for enterprise data warehouse and data marts
- -Write and maintain scripts in Python for various applications
- -Develop, optimize, troubleshoot complex SQL queries and database structures for relational and non-relational databases
- - Work closely with analysts and product managers to understand data requirements and deliver high-quality data
- -Provide technical guidance and mentorship to other team members fostering best practices in data engineering
- -Monitor system performance, identify bottlenecks, implement solutions to ensure high data availability
- -High proficiency in the Hadoop ecosystem (HDFS, MapReduce, Yarn) and Apache Spark, Scala for large scale data processing
- -Good knowledge in Python for data manipulation and scripting
- -Mastery of advanced SQL and deep experience working with relational databases and datawarehouses
- -Hands-on experience with workflow management tools like Apache Airflow
- -Familiarity with Unix shell scripting for automation and system management
Required Skills:
Top Skills
Apache Airflow
Hadoop
Python
Spark Sql
SQL
Unix
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Brillio is the leader in global digital business transformation, applying technology with a human touch. We help businesses define internal and external transformation objectives, and translate those objectives into actionable market strategies using proprietary technologies. With 2600+ experts and 13 offices worldwide, Brillio is the ideal partner for enterprises that want to quickly increase their core business productivity, and achieve a competitive edge, with the latest digital solutions.
