Big Data Architect - Core Infrastructure Technology
Job Description:
Responsible for defining an architectural vision and solution architecture which aligns with the enterprise architecture strategy, technology and platform choices. Describes the solution intent and the associated operating environment, determines the primary systems/subsystems and their interfaces, defining non-functional requirements and architectural runway to support new epics/features. Ensures the solution is fit for purpose and use by working with stakeholders, vendors/service providers, and evaluating the impact of strategic design decisions. Works across business and technology to create the solution intent and architectural vision and evolves it to an emerging backlog. Consults the business to clearly understand the business problems and technology to understand the technology challenges of the solution and finds creative solutions through practical experiments and POCs. Leads rapid shaping of a high level architecture with details filled in with emerging business requirements; ensures architecture is flexible and modular and designed to adapt easily. Utilizes the defined best practices, templates and documentation to create architectural designs; suggests improvements to best practices and templates through practical knowledge. Works with Product Manager/Owner to plan and prioritize technology focused backlog items for the architecture runway to enable business epics/features. Clarifies the architecture for the development teams to support implementation, and provides solution options to resolve any architectural impediments. Performs design and code reviews to ensure all non-functional requirements for a solution are sufficiently met (e.g. security, performance, maintainability, scalability, usability, and reliability). Educates team members on the technology practices, standardization strategies and best practices to create innovative solutions. Individual contributor.
We seek a highly technical and hands on Big Data Platform Architect to join a team of platform architects who are the technical face of our platform. The candidate will support one of our lines of business by partnering with their development teams and architects to solution the appropriate distributed system architecture as per the customers' requirements and the platform's standards.
This candidate will be a part of the larger architecture group that sets the strategy, direction and standards for 100+PB analytics platform which is on a transformation journey to hybrid cloud architecture with self-serve, APIs and containerization as key mandates. The candidate will be in the unique position to work with technical members of the business applications running on the platform while having the ability to influence the platform's technical strategy and direction.
Responsibilities:
Design, architecture and building of data analytics solutions for existing or new tenants of the platform. Work directly with platform tenants to understand requirements, align to platform technology stack and document solution.
- Design and certify new platform components for use by tenants.
- Provide thought leadership as a member of the platform's architecture team and collaborate extensively with platform engineering and operations teams.
- Provide feedback from the tenants back to the larger architecture team in terms of standards, process and technology gaps. Partner with fellow architects to update and create new standards, policies, and best practices.
- Conduct proof of concept work to determine proper configurations, design patterns and general tool fit for the platform tenant
- Participate in regular status meetings to track progress, resolve issues, mitigate risks and escalate concerns in a timely manner
Required Skills:
- 6+ years developing software in C, C++, Java, Scala, Ruby, and/or Python
- 5+ years of hands on architecture experience with technologies such as Hadoop, Hive, Spark, Ranger, Impala
- Experience designing and implementing into production complex solutions for distributed systems
- Experience with Ranger, Atlas, Tez, Hive LLAP, NiFi
- Working knowledge of application and system availability, scalability and distributed data platforms
- Ability to design systems with security focus from the start
- Excellent communication skills, particularly those relating to complex findings and presenting them to ensure audience appeal at various levels of the organization
- Ability to integrate research and best practices into problem avoidance and continuous improvement
- Exercise independent judgment in methods, techniques and evaluation criteria for obtaining results
- Be able to demonstrate analytical and problem solving skills
- Strong leadership presence. Can-do attitude. Positive and proactive leadership style.
Desired Education & Technical Training:
- BS degree in Computer Science/Software Engineering (or related degree) or equivalent experience
Desired Skills:
- Knowledge and experience with Cloud and containerization technologies: Kubernetes, Rancher, OpenShift or Docker
- Experience with public cloud migrations and adoptions, ideally on Azure
- Experience administrating large-scale distributed applications using Hadoop
- Industry certifications in Big Data, Data Science, Cloud
Shift:
1st shift (United States of America)
Hours Per Week:
40