TransUnion's Job Applicant Privacy Notice
What We'll Bring:
We are looking for a highly skilled SRE having experience in Cloud DevOps Operations and Hadoop Administration to join our dynamic team. Within our SRE team, you'll be a master of managing and administering the backbone of our technological analytics infrastructure. This role will work closely with the engineering team and will be responsible for capacity planning, service configuration, cluster expansion, monitoring, tuning, and overall ongoing support of the on-prem and cloud solutions. Responsibility also includes researching and recommending methods and technologies to improve cluster operation and user experience.
What You'll Bring:
Job Description:
Manage the Hadoop distribution on Linux instances, including configuration, capacity planning, expansion, performance tuning and monitoring.
Upgrades of MapR cluster to newer versions.
Expertise in managing Hadoop cluster patching and upgrades.
Administrator experience with NoSQL MapR DB.
Ability to handle and responsible for commissioning and decommissioning of nodes from clusters.
Support OS patching of cluster nodes.
Hands-on experience with Dataproc, Dataflow, GCS, BigQuery,
Hands-on experience with Terraform, Jenkins, Kubernetes.
Work with data engineering team to support deployment of Spark and Hadoop jobs.
Work with delivery teams for Provisioning of users into Hadoop.
Work with end users to troubleshoot and resolve incidents with data accessibility.
Contribute to the architecture design of the cluster to support growing demands and requirements.
Contribute to planning and implementation of software and hardware upgrades.
Responsible for Capacity Planning, Infrastructure Planning based on the workloads and future requirements.
Recommend and implement standards and best practices related to cluster administration.
Research and recommend automated approaches to cluster administration.
Enable Ranger for RBAC (role-based access control) to have a privilege level access to the data as per the security policies.
Enable data encryption at rest and at motion with TLS/SSL to meet the security standards.
Day-to-day operation activities including, incidents, problem and change management.
What You'll Bring:
Bachelor's degree in Computer Science, Information Technology, or related field.
4+ years of administrator experience managing full stack Hadoop distribution (preferably MapR) with technical stack of MapReduce, Spark, Yarn, Hive, HDFS, including installation, configuration, administration, monitoring and optimization.
3+ years experience with implementing and managing Hadoop related security in Linux environments (Kerberos, SSL, Sentry, Ranger, Encryption)
3+ Years of doing data related Benchmarking, Performance Analysis and Tuning.
Experience in monitoring and optimizing SparkSQL and Hive queries, including memory usage.
Strong knowledge of Yarn configuration in a multi-tenancy environment. Candidate should have sound knowledge with Yarn fair scheduler.
Strong knowledge on Active Directory/LDAP security integration with Big Data.
Worked as a team member to provide 24x7 on-call Hadoop administration and support.
Hands-on experience in Monitoring and Reporting on Hadoop resource utilization.
Proficiency in using tools like Splunk, Grafana, etc for performance monitoring.
Moderate experience with Linux or storage including shell scripting, system monitoring, and troubleshooting.
Knowledge of disaster recovery planning.
Knowledge of data protection regulations.
Knowledge of network concepts and experience managing host level network services.
Ability to troubleshoot and resolve production issues.
Ability to develop and maintain SOP and documentation.
Ability to work in a fast-paced collaborative environment and meet deadlines.
Strong communication and collaboration skills.
Excellent problem-solving skills.
Impact You'll Make:
We're also looking for the preferred skills below. Whether you are proficient or could use some brushing up, we're happy to support your development in:
Certification on GCP (preferably, DevOps Engineer)
Experience with on-prem to cloud migration.
Experience with Tableau or Superset.
Linux/shell/python scripting.
Knowledge on Helm, Groovy.
Knowledge of web services, API, REST, and RPC
Basic expeirence of Python.
Knowledge on RDBMS.
Familiarity with containerization (Docker, Kubernetes) and orchestration.
This is a hybrid position and involves regular performance of job responsibilities virtually as well as in-person at an assigned TU office location for a minimum of two days a week.
TransUnion Job Title
Sr Engineer, Database Engineering
Top Skills
What the Team is Saying
What We Do
TransUnion is a global information and insights company that makes trust possible by ensuring that each consumer is reliably and safely represented in the marketplace.
We do this by having an accurate and comprehensive picture of each person.
This picture is grounded in our legacy as a credit reporting agency which enables us to tap into both credit and public record data; our data fusion methodology that helps us link, match and tap into the awesome combined power of that data; and our knowledgeable and passionate team, who stewards the information with expertise, and in accordance with local legislation around the world.
Because of our work, organizations can better understand consumers in order to make more informed decisions, and earn their trust through great, personalized experiences, and the proactive extension of the right opportunities, tools and offers. In turn, consumers can be confident that their data identities will result in the opportunities they deserve.
We make trust possible, so businesses and consumers can transact with confidence and achieve great things. We call this Information for Good®—it’s our purpose, and what drives us every day.
Why Work With Us
Our culture is welcoming, energetic and innovative. There’s an overall synergy that flows throughout TransUnion, creating a sense of unity in knowing that we’re all working to achieve the same overall goal. We’re dedicated to providing opportunities for our people to get involved and stay connected with their colleagues across the globe.
Gallery
TransUnion Teams
TransUnion Offices
Hybrid Workspace
Employees engage in a combination of remote and on-site work.
