Big Data Engineer - Python, PySpark

Sorry, this job was removed at 3:10 p.m. (CST) on Sunday, July 31, 2022
Find out who's hiring in Phoenix, AZ.
See all Data + Analytics jobs in Phoenix, AZ
Apply
By clicking Apply Now you agree to share your profile information with the hiring company.

Description

You Lead the Way. We've Got Your Back.

At American Express, we know that with the right backing, people and businesses have the power to progress in incredible ways. Whether we're supporting our customers' financial confidence to move ahead, taking commerce to new heights, or encouraging people to explore the world, our colleagues are constantly redefining what's possible - and we're proud to back each other every step of the way. When you join #TeamAmex, you become part of a diverse community of over 60,000 colleagues, all with a common goal to deliver an exceptional customer experience every day.

Be part of the exciting transformation world and join a team of creative, curious, and passionate professionals whose mission is to become the future of American Express by staying on the cutting edge and bringing innovation to life! The Enterprise Customer Marketing technology organization leads the technology strategy and delivery of marketing capabilities, products, and customer value streams within American Express.

This Big Data Engineer position will be part of the transformation journey supporting Enterprise Customer Marketing Data Engineering and Business Intelligence organization, within American Express Technology (AET).

If you love to work with data, build data pipelines using next generation technologies, create analytics data marts for generating intelligence and support self-service analytics for business users we'll support you with an open environment and learning culture to grow your career.

Here's some of what you'll be doing:

  • Passionately developing data driven products.
  • Partner with the business and technical partners to design new customer experiences through innovative data & insight products.
  • Prioritizing the product backlog with the business effectively and meeting tight deadlines when required.
  • Balancing complex integrations and simple design
  • Function as member of an agile team by contributing to software builds through consistent development practices (tools, common components, and documentation)
  • Participate in code reviews and automated testing
  • Debug basic software components and identify code defects for remediation
  • Enable the deployment, support, and monitoring of software across test, integration, and production environments
  • Automate deployments in test or production environments
  • Automatically scale applications based on demand projections
  • Demonstrate increased self-reliance to achieve team goals
  • Influence team members with creative changes and improvements by challenging status quo and demonstrating risk taking



Additional skills:

  • Machine learning
  • GCP Cloud/Big Query



High Performance Behaviors:

  • Recognizes opportunities to adopt innovative technologies to enable business capabilities
  • Keeps up to date on current research and technology in the industry
  • Recognizes the importance of collaboration to achieve objectives
  • Clearly communicates ideas and concepts to others
  • Finds opportunities within projects and acts on own initiative without being prompted, including making appropriate decisions as vital
  • Provides feedback to team members in code reviews
  • Drive creative changes & continuous improvements
  • Mentors and guides junior team members to success within the team


Qualifications

Education & Experience:

  • A Bachelor's degree in computer science, computer engineering, other technical discipline, or equivalent work experience.
  • 5+ years of software development experience with proven hands-on experience in Bigdata technologies.
  • Familiar with Agile or other rapid application development methods. Experience with design and coding across one or more platforms and languages as appropriate.
  • Hands-on data engineering experience in big data and familiarity with ETL using tools and programming language
  • Hands-on expertise with application design, software development and automated testing.
  • Ability to effectively interpret technical and business objectives and challenges, and articulate solutions.
  • Ability to think abstractly and deal with ambiguous/under-defined problems. Ability to enable business capabilities through innovation.
  • High energy demonstrated willingness to learn new technologies and takes pride in how fast to develop working software.
  • Experience with distributed (multi-tiered) systems, algorithms, and relational databases.
  • Knack to guide/mentor other team members in both technology and domain.



Required skills:

  • Strong programming knowledge in Java
  • Strong hands-on experience in Big Data Technologies (Hive, Spark, PySpark, Java, Scala, Python etc.) and BI tools like Tableau.
  • Design and implementation of REST APIs, Event Driven Architecture - Kafka.
  • Knowledge/ experience in Distributed NoSQL databases (Cassandra, Redis, HBASE).
  • Good understanding of CI/CD processes leveraging Jenkins, XLR and Maven.
  • Knowledge of Big Data Ingestion/Transformation processes is a plus
  • Good knowledge about the configuration management tools like GitHub.
  • Understanding of various Software Delivery Lifecycle methodologies particularly SAFe Agile and tools such as Confluence, Rally / Jira.
  • Understanding and hands-on experience with UNIX / Shell / Perl / Python scripting
  • Database query optimization for Hive, Spark
  • Knack and natural curiosity for data exploration and analysis
  • Strong analytical ability
  • Excellent communication skill



We back our colleagues with the support they need to thrive, professionally and personally. That's why we have Amex Flex, our enterprise working model that provides greater flexibility to colleagues while ensuring we preserve the important aspects of our unique in-person culture. Depending on role and business needs, colleagues will either work onsite, in a hybrid model (combination of in-office and virtual days) or fully virtually.

If the role you are applying for is designated as hybrid or onsite, you will be required to demonstrate that you have completed your primary COVID-19 vaccination series (i.e., 2 doses for Moderna/Pfizer and 1 dose for J&J) and, for medically eligible* colleagues, a booster shot, in order to work in or visit any of our offices. This requirement is subject to legally required accommodations.

*Booster eligibility: The CDC has established guidelines for when adults are eligible to receive booster shots depending on when they completed their initial vaccine series (currently five months after the Pfizer-BioNTech and Moderna vaccines, and two months after the J&J Vaccine). If you have completed your primary vaccine series but have not yet reached your booster eligibility date, you will be able to come into the office; however, you will need to complete your booster within 30 days of becoming eligible to continue coming in and participating in company-sponsored in-person events.

Employment eligibility to work with American Express in the U.S. is required as the company will not pursue visa sponsorship for these positions.

American Express is an equal opportunity employer and makes employment decisions without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, protected veteran status, disability status, age, or any other status protected by law.

More Information on American Express
American Express operates in the Financial Services industry. The company is located in New York, NY and New York, NY. American Express was founded in 1850. It has 100703 total employees. It offers perks and benefits such as Flexible Spending Account (FSA), Disability insurance, Dental insurance, Vision insurance, Health insurance and Life insurance. To see all 142 open jobs at American Express, click here.
Read Full Job Description
Apply Now
By clicking Apply Now you agree to share your profile information with the hiring company.

Similar Jobs

Apply Now
By clicking Apply Now you agree to share your profile information with the hiring company.
Learn more about American ExpressFind similar jobs