Data Engineer Lead

Data Engineer Lead

Lead Data Engineer

Role

As a Lead Data Engineer, you will:

• Lead the design, development, and evolution of enterprise-grade data platforms and pipelines supporting credit risk products, decisioning capabilities, and analytics solutions.

• Architect and implement scalable ETL/ELT frameworks utilizing Databricks, Spark, Delta Lake, and cloud-native technologies.

• Establish data quality, lineage, governance, observability, and monitoring capabilities to ensure trusted and compliant data products.

• Drive the migration and modernization of legacy data assets into cloud-based architectures and Data Lakehouse platforms.

• Partner with Risk, Product, Architecture, and Engineering teams to translate business requirements into scalable technical solutions.

• Define and promote engineering standards, coding practices, testing frameworks,Data Quality frameworks, deployment automation, and operational excellence across the data ecosystem.

• Lead technical design reviews and influence architectural direction for data-intensive applications and services.

• Optimize large-scale data processing workloads for performance, reliability, scalability, and cost efficiency.

• Enable AI and advanced analytics initiatives through creation of high-quality, reusable, governed data products.

• Mentor, coach, and raise the technical capability of engineers across the organization by fostering a culture of ownership, continuous learning, accountability, and engineering excellence.

• Shape strategic roadmap planning, technology evaluation, and delivery priorities across the ECR portfolio, balancing business outcomes, engineering feasibility, risk, compliance, and long-term platform sustainability.

• Support regulatory, compliance, security, and audit requirements through robust engineering controls and documentation.

• Own complex problems with dependencies across multiple services and facilitate cross-functional collaboration to drive resolution.

• Conduct technical interviews, assess engineering talent, and contribute to raising the overall performance bar of the organization.

All About You

The ideal candidate for this position should have:

Essential Skills & Experience

• Strong expertise in designing and implementing large-scale data engineering solutions and distributed data processing systems.

• Advanced proficiency with Databricks, Apache Spark, Delta Lake, SQL, Hadoop and Python.

• Experience building and operating cloud-based data platforms on Azure, AWS, or GCP.

• Expertise developing enterprise-grade ETL/ELT pipelines, streaming architectures, and data integration frameworks.

• Strong understanding of data modeling techniques for analytical and operational workloads.

• Experience implementing data quality frameworks, lineage, metadata management, and governance practices.

• Experience with Data formats ( Parquet, Avro, ORC )

• Working knowledge of CI/CD pipelines, infrastructure-as-code, automated testing, and DevOps practices.

• Experience with Workflow orchestration Tools like Airflow

• Strong understanding of security, privacy, and compliance requirements associated with sensitive financial and customer data.

• Proven ability to lead technical initiatives across multiple teams and influence engineering direction without direct authority.

• Excellent communication skills with the ability to collaborate effectively across technical and business functions.

• Demonstrated leadership in aligning engineering teams around shared goals, driving delivery through ambiguity, and creating clarity for stakeholders across product, risk, analytics, architecture, and operations.

• Ability to influence senior technical and business stakeholders, make thoughtful trade-off decisions, and guide teams toward pragmatic solutions that improve credit risk outcomes and operational resilience.

• Knowledge of Java Based application development is a huge Plus.

Leadership Skills

• Lead by influence across engineering, product, risk, analytics, and architecture teams to align priorities and deliver measurable business outcomes.

• Create clarity in complex, ambiguous environments by translating business needs into actionable technical direction and execution plans.

• Develop engineering talent through mentoring, knowledge sharing, design guidance, and constructive feedback.

• Promote a high-accountability culture focused on quality, reliability, security, compliance, and continuous improvement.

• Communicate effectively with senior stakeholders and clearly articulate trade-offs, risks, dependencies, and delivery progress.

Preferred Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or a related STEM discipline or alternative minimum of 10 years of experience in a related field.

Technical Skills

Preferred expertise in:

• Databricks

• Apache Spark / PySpark

• Hadoop

• AirFlow

• Delta Lake

• SQL

• Python

• Airflow

• Azure Data Services

• Kafka/Event Streaming

• GitHub / CI-CD Tooling

REMOTE

ADVANCED ENGLISH