Lead PySpark Engineer

Job Description

This position will follow our hybrid work model, we expect the selected candidate to be in office 2-3 days a week(10-12 days a month ) at the following office location: St. Petersburg, FL.

Job Summary:
This Lead Engineer, Technology will partner closely work with the Compliance and Supervision Business Unit on solutions from inception through ongoing maintenance. The role will focus on building out Supervision and Compliance Surveillance controls within Wealth Management. The ideal candidate will have extensive experience with AWS Glue, PySpark, and Python programming. This position will be responsible for designing, implementing, and developing business controls and collaborating with cross-functional teams to meet business needs.
This hybrid position will be based out of our corporate headquarters in Saint Petersburg, FL.

Responsibilities:

• Design and develop business controls using AWS Glue and PySpark.

• Integrate data from various sources into Amazon Redshift, ensuring data quality and consistency.

• Optimize the performance of data processing jobs and fine tune queries.

• Monitor and troubleshoot data pipelines to ensure reliability and efficiency.

• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements.

• Ensure compliance with security policies and regulations.

• Document data workflows and best practices.

• Mentor and guide junior engineers on the team.

• Continuously explore new AWS services and technologies to improve existing solutions.

Skills:
 

Data Engineering Tools:

• Experience in Airflow

• PySpark and experience with caching, RDD, multithreading , in-memory computations of very large datasets. 

•  Experience in open table format; Ex: Iceberg , Delta Lake, Avro , Parquet

• Solid knowledge and experience with Glue Notebook and EMR/Databricks. 

• Experience with DataLake concept on Distributed object store. 

• Python programming

Exp SQL:

• Advanced experience in writing complex SQL queries

Data Warehousing:

• Solid Knowledge of OLTP and OLAP

• AWS Services Experience minimum 3 years:

• Glue

• EMR

• S3

• Redshift

• Athena

• Lambda

• Step Functions

• Data pipeline and migration

• Terraform

• CI/CD pipeline

Generic domain background:

• Data Analysis and Reporting: Proficiency is SQL 

• Data Analysis using large data sets

• System Integration: Experience with integrating compliance systems with other enterprise systems.

• Financial Industry Experience: Experience working in financial firms, understanding their unique compliance challenges and regulatory environments.

• Knowledge in TFS is a plus.

• Knowledge in Jenkins pipeline and deployment.

Ability to:

• Identify and understand issues, problems and opportunities for automation.

• Good communication and organization skills that clearly convey information and ideas through a variety of media to individuals or groups in a manner that engages the audience and helps them understand and retain the message.

• Use effective approaches for choosing a course of action or developing appropriate solutions; recommend or take action that is consistent with available facts, constraints and probable consequences.

• Demonstrate a satisfactory level of technical and professional skill or knowledge in position-related areas; remains current with developments and trends in areas of expertise.

• Develop and use collaborative relationships to facilitate the accomplishment of work goals.

• Make internal and external clients and their needs a primary focus of actions; develop and sustain productive client relationships.

Education

Bachelor’s: Business Administration, Bachelor’s: Computer and Information Science (Required), Bachelor’s: Computer Engineering, Master's

Work Experience

General Experience - 6 to 10 years

Certifications

Travel

Workstyle

Hybrid

At Raymond James our associates use five guiding behaviors (Develop, Collaborate, Decide, Deliver, Improve) to deliver on the firm's core values of client-first, integrity, independence and a conservative, long-term view. 
We expect our associates at all levels to:
•  Grow professionally and inspire others to do the same
•  Work with and through others to achieve desired outcomes
•  Make prompt, pragmatic choices and act with the client in mind
•  Take ownership and hold themselves and others accountable for delivering results that matter
•  Contribute to the continuous evolution of the firm

At Raymond James – as part of our people-first culture, we honor, value, and respect the uniqueness, experiences, and backgrounds of all of our Associates.  When associates bring their best authentic selves, our organization, clients, and communities thrive. The Company is an equal opportunity employer and makes all employment decisions on the basis of merit and business needs.