TransUnion's Job Applicant Privacy Notice
Team Overview
The team is focused on building and evolving job manager and distributed processing solutions using Apache Spark and cloud technologies. This is a hybrid position and involves regular performance of job responsibilities virtually as well as in-person at an assigned TU office location for a minimum of two days a week.Role Overview And Core Responsibilities
Design scalable and resilient data processing solutions using Apache Spark and modern data platform technologies. Create architecture designs, reference patterns, and technical specifications for distributed data platforms. Guide engineering teams on architecture, design, and implementation decisions. Ensure solutions align with enterprise architecture standards, security requirements, and engineering best practices. Evaluate technology choices, frameworks, and platform capabilities to address evolving business needs. Identify architectural risks and recommend mitigation strategies. Contribute to platform modernization and cloud transformation initiatives.
Provide technical leadership for Apache Spark-based batch and streaming data processing solutions. Guide development teams on Spark architecture, performance optimization, and operational best practices. Define design patterns for scalable, fault-tolerant, and maintainable Spark applications. Assist teams in resolving complex technical challenges involving distributed processing and large-scale data workloads. Drive performance improvements through effective partitioning, query optimization, resource management, caching, and tuning strategies. Establish standards for observability, monitoring, reliability, and operational excellence.
Collaborate with engineering teams throughout the software development lifecycle. Conduct architecture and design reviews for new features, data pipelines, and platform enhancements.
Promote adoption of CI/CD, automated testing, infrastructure-as-code, and DevOps best practices. Contribute to technical decision-making for platform enhancements and modernization initiatives.
Work closely with product managers, engineering leads, data scientists, and platform teams. Translate business requirements into scalable technical solutions and architecture designs. Communicate architecture decisions, technical trade-offs, and implementation approaches to stakeholders. Participate in roadmap discussions to ensure alignment between business goals and technology strategy. Collaborate across teams to drive successful implementation of data platform capabilities.
Mentor engineers on distributed systems design, Spark development, and data engineering best practices. Provide guidance through architecture reviews, design discussions, and technical workshops.
Required Knowledge And Experiences
Ensure architecture and design activities follow established SDLC and governance standards. Participate in code reviews, design reviews, and architecture assessments. Promote reliability, maintainability, scalability, and performance considerations throughout solution development. Drive continuous improvement in architecture practices and engineering standards.
Strong hands-on expertise with Apache Spark (Spark SQL, Structured Streaming, DataFrames, Dataset APIs). Experience designing and building large-scale distributed data processing systems. Strong knowledge of Spark optimization techniques including:
- Partitioning Strategies
- Shuffle Optimization
- Join Optimization
- Memory Management
- Resource Utilization
- Performance Tuning
Proficiency in Scala, Java, or Python. Experience with technologies such as:
- Hadoop
- Hive
- Iceberg
- AWS EMR
- AWS Glue
- GCP Dataproc
- BigQuery
Experience designing and operating batch and streaming data pipelines. Understanding of cloud-native architecture and distributed systems principles. Experience deploying Spark solutions on AWS and/or GCP platforms.
Individual contributor architecture role focused on distributed data platforms and Spark-based solutions. Provides architecture leadership and technical guidance across one or more engineering teams. Responsible for solution architecture quality, technology selection, design governance, and technical direction within the domain. Partners with Engineering Managers and Technical Leads to deliver scalable and reliable platform capabilities. Serves as a key technical advisor for Spark architecture, distributed systems design, and cloud-native data platforms.
- Experience with AWS services such as EMR, Glue, S3, Lambda, EKS, ECS, Step Functions, and CloudWatch.
- Experience with GCP services such as Dataproc, BigQuery, Cloud Storage, Dataflow, Pub/Sub, and Composer.
- Experience managing terabyte-to-petabyte scale data processing environments.
- Experience with Spark Structured Streaming, real-time analytics, and event-driven data processing.
- Contributions to Spark optimization, platform engineering, or open-source data ecosystem projects are a plus.
TransUnion Overview:
At TransUnion, we encourage and are committed to creating a real, positive impact and shared sense of purpose within our Workforce for Good, which empowers our people to grow, innovate and contribute to a better future for our communities and customers. We strive to build an environment where our associates are in the driver’s seat of their professional development— while having access to help along the way. We recognize that success comes when our associates thrive both professionally and personally; that’s why we prioritize work/life flexibility and offer resources for our teams across the globe to collaborate and drive excellence.
Be a part of our Workforce for Good – you’ll work with great people, pioneering products and cutting-edge technology.
TransUnion Job Title
Architect, Applications Programming
Skills Required
- 10+ years of experience in software engineering, data engineering, or distributed systems development
- Strong hands-on expertise with Apache Spark (Spark SQL, Structured Streaming, DataFrames, Dataset APIs)
- Experience with Spark optimization techniques (partitioning, shuffle optimization, join optimization, memory management, resource utilization, performance tuning)
- Proficiency in Scala, Java, or Python
- Experience designing and building large-scale distributed data processing systems and operating batch and streaming data pipelines
- Experience with Hadoop, Hive, and Iceberg
- Experience deploying Spark solutions on AWS and/or GCP (experience with AWS EMR, AWS Glue, GCP Dataproc, BigQuery)
- Understanding of cloud-native architecture and distributed systems principles
- Experience with AWS services such as S3, Lambda, EKS, ECS, Step Functions, CloudWatch
- Experience with GCP services such as Cloud Storage, Dataflow, Pub/Sub, Composer
- Experience managing terabyte-to-petabyte scale data processing environments and Spark Structured Streaming / real-time analytics
- Contributions to Spark optimization, platform engineering, or open-source data ecosystem projects
What the Team is Saying
TransUnion Compensation & Benefits Highlights
-
Healthcare Strength — Healthcare is described as comprehensive with day-one medical, dental and vision coverage, plus HSA/FSA options and expanded mental-health support via Spring Health. Feedback suggests coverage depth and immediate eligibility are notable strengths.
-
Parental & Family Support — Family support includes paid parental leave with a gradual return, adoption and caregiver assistance, and backup care through a Care@Work membership. External recognition highlights inclusive offerings such as fertility support and bereavement provisions.
-
Leave & Time Off Breadth — Time-off programs feature flexible time off, paid holidays, two global wellness days, and paid volunteer time. Feedback suggests these leave elements contribute meaningfully to work-life balance.
TransUnion Insights
What We Do
TransUnion is a global information and insights company that makes trust possible by ensuring that each consumer is reliably and safely represented in the marketplace. We do this by having an accurate and comprehensive picture of each person. This picture is grounded in our legacy as a credit reporting agency which enables us to tap into both credit and public record data; our data fusion methodology that helps us link, match and tap into the awesome combined power of that data; and our knowledgeable and passionate team, who stewards the information with expertise, and in accordance with local legislation around the world. Because of our work, organizations can better understand consumers in order to make more informed decisions, and earn their trust through great, personalized experiences, and the proactive extension of the right opportunities, tools and offers. In turn, consumers can be confident that their data identities will result in the opportunities they deserve. We make trust possible, so businesses and consumers can transact with confidence and achieve great things. We call this Information for Good®—it’s our purpose, and what drives us every day.
Why Work With Us
Our culture is welcoming, energetic and innovative. There’s an overall synergy that flows throughout TransUnion, creating a sense of unity in knowing that we’re all working to achieve the same overall goal. We’re dedicated to providing opportunities for our people to get involved and stay connected with their colleagues across the globe.
Gallery
TransUnion Teams
TransUnion Offices
Hybrid Workspace
Employees engage in a combination of remote and on-site work.
