P03

Posted An Hour Ago
Be an Early Applicant
2 Locations
Hybrid
Senior level
Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
The Role
Design, develop, optimize, and operate scalable distributed data-processing applications and terabyte-scale production pipelines. Build recurring batch workloads, troubleshoot performance and reliability issues, implement fault-tolerant workflows, and develop cloud-based data engineering solutions using Spark, Scala, Python, and GCP or AWS. Collaborate on platform improvements, automation, monitoring, testing, CI/CD, and identity-resolution or AI/ML-related data solutions.
Summary Generated by Built In

TransUnion's Job Applicant Privacy Notice

Team Overview

The team is responsible for building, enhancing, and operating scalable data engineering solutions and distributed data processing platforms. The team processes large volumes of data, including terabyte-scale datasets, through recurring batch workloads and production pipelines, with a strong focus on performance, scalability, reliability, and cost efficiency.
The role will contribute to developing and optimizing data-intensive applications using Apache Spark, Scala, Python, and cloud technologies, with a preference for Google Cloud Platform (GCP). The team works on identity resolution and data-driven solutions, where linking identities and analyzing relationships across devices, digital identifiers, and other data signals are important. Experience in identity resolution, AdTech, MarTech, or digital identity platforms will be a strong advantage.
The team also explores AI/ML concepts and AI-assisted development approaches to improve engineering productivity and enable innovative solutions. This is a hybrid position and involves regular performance of job responsibilities virtually as well as in-person at an assigned TU office location for a minimum of two days a week.

Role Overview And Core Responsibilities

* Design, develop, test, and maintain scalable data processing applications using Apache Spark and Scala.
* Build and operate production-grade distributed data pipelines capable of processing terabytes of data per execution.
* Develop and maintain recurring batch processing workloads, including weekly and scheduled pipelines, ensuring timely completion, reliability, and data quality.
* Optimize large-scale Spark jobs for execution time, memory utilization, parallelism, resource consumption, and cloud infrastructure costs.
* Design robust data processing workflows with appropriate error handling, retries, checkpointing, recovery mechanisms, and idempotency.
* Monitor production jobs, troubleshoot failures and performance bottlenecks, and perform root-cause analysis to ensure operational stability.
* Develop cloud-based data engineering solutions, preferably on Google Cloud Platform (GCP), with AWS experience also considered.
* Work with technologies such as Google Dataproc, BigQuery, cloud object storage, Parquet, and equivalent cloud data services.
* Write clean, efficient, reusable, and maintainable code using Scala, Python, and shell scripting.
* Apply strong analytical and problem-solving skills to resolve complex data processing and distributed computing challenges.
* Contribute to platform enhancements, automation, workflow orchestration, and continuous improvement of data engineering systems.
* Leverage AI/ML concepts and AI-assisted development tools where appropriate to improve development productivity and engineering quality.
* Collaborate with engineers, architects, and cross-functional stakeholders to deliver scalable, reliable, and maintainable solutions.
* Follow software engineering best practices, including code reviews, automated testing, version control, CI/CD, monitoring, and production support.


Required Knowledge And Experiences

Core technical requirements

* Strong hands-on experience with Apache Spark and Scala for distributed data processing.
* Demonstrated experience designing, developing, and operating production-grade data pipelines processing terabytes of data per run.
* Experience managing large-scale recurring batch workloads, preferably weekly or more frequently, with responsibility for successful execution, performance, reliability, and data quality.
* Strong understanding of Spark performance optimization, including partitioning, shuffles, joins, caching, memory management, parallelism, and resource tuning.
* Experience troubleshooting failed or slow-running production jobs and implementing robust recovery, retry, and fault-tolerance mechanisms.
* Working knowledge of cloud platforms, preferably Google Cloud Platform (GCP); AWS experience is also acceptable.
* Familiarity with GCP Dataproc, BigQuery, cloud object storage, and distributed data formats such as Parquet, or equivalent AWS technologies.
* Proficiency in Python and shell scripting for automation, data processing, and operational workflows.
* Strong analytical thinking, problem-solving, debugging, and root-cause analysis skills.
* Good understanding of distributed computing, data structures, algorithms, and software engineering fundamentals.
* Familiarity with Git, CI/CD, automated testing, workflow orchestration, and production monitoring.
Preferred qualifications

* Experience operating terabyte-scale Spark workloads in production with defined execution schedules and delivery SLAs.
* Experience processing billions of records, handling incremental data loads, and optimizing large-scale data transformations.
* Experience with cloud cost optimization, workload sizing, capacity planning, and resource utilization.
* Exposure to machine learning pipelines, identity resolution, entity resolution, or graph-based data processing.
* Basic understanding of AI/ML concepts and familiarity with LLMs, generative AI, or AI-assisted software development tools.
* Ability to independently own complex engineering problems from design and implementation through production deployment and ongoing operations.

* Experience in identity resolution, identity graphs, entity resolution, or digital identity platforms, including linking and reconciling identifiers across multiple data sources.
*Experience in MarTech (Marketing Technology) or AdTech (Advertising Technology), particularly with digital identifiers such as mobile advertising IDs, cookies, IP addresses, device signals, or email identifiers.
*Familiarity with audience segmentation, household or individual identity mapping, cross-device identity, or identity graph construction.


TransUnion Overview:

At TransUnion, we encourage and are committed to creating a real, positive impact and shared sense of purpose within our Workforce for Good, which empowers our people to grow, innovate and contribute to a better future for our communities and customers. We strive to build an environment where our associates are in the driver’s seat of their professional development— while having access to help along the way. We recognize that success comes when our associates thrive both professionally and personally; that’s why we prioritize work/life flexibility and offer resources for our teams across the globe to collaborate and drive excellence.


Be a part of our Workforce for Good – you’ll work with great people, pioneering products and cutting-edge technology.


TransUnion Job Title


Sr Developer, Software Development

Skills Required

  • Strong hands-on experience with Apache Spark and Scala for distributed data processing.
  • Experience designing, developing, and operating production-grade data pipelines processing terabytes of data per run.
  • Experience managing large-scale recurring batch workloads with responsibility for execution, performance, reliability, and data quality.
  • Strong understanding of Spark performance optimization, including partitioning, shuffles, joins, caching, memory management, parallelism, and resource tuning.
  • Experience troubleshooting failed or slow-running production jobs and implementing recovery, retry, and fault-tolerance mechanisms.
  • Working knowledge of cloud platforms, preferably Google Cloud Platform; AWS experience is also acceptable.
  • Familiarity with GCP Dataproc, BigQuery, cloud object storage, and Parquet or equivalent AWS technologies.
  • Proficiency in Python and shell scripting for automation, data processing, and operational workflows.
  • Strong analytical thinking, problem-solving, debugging, and root-cause analysis skills.
  • Understanding of distributed computing, data structures, algorithms, and software engineering fundamentals.
  • Familiarity with Git, CI/CD, automated testing, workflow orchestration, and production monitoring.
  • Experience operating terabyte-scale Spark workloads in production with defined execution schedules and delivery SLAs.
  • Experience processing billions of records, handling incremental data loads, and optimizing large-scale data transformations.
  • Experience with cloud cost optimization, workload sizing, capacity planning, and resource utilization.
  • Exposure to machine learning pipelines, identity resolution, entity resolution, or graph-based data processing.
  • Basic understanding of AI/ML concepts and familiarity with LLMs, generative AI, or AI-assisted software development tools.
  • Ability to independently own complex engineering problems from design and implementation through production deployment and ongoing operations.
  • Experience with identity resolution, identity graphs, entity resolution, or digital identity platforms, including linking and reconciling identifiers across multiple data sources.
  • Experience in MarTech or AdTech, particularly with mobile advertising IDs, cookies, IP addresses, device signals, or email identifiers.
  • Familiarity with audience segmentation, household or individual identity mapping, cross-device identity, or identity graph construction.

What the Team is Saying

Patrick
Tiana
Jason
Lauren
TC
Jay
Aayushi
Paul
Alex Barnett
Sheetal Wathare
TransUnion
TransUnion

TransUnion Compensation & Benefits Highlights

How does TransUnion ensure its pay and bonus plans are competitive?

TransUnion supports competitive compensation through a broader Total Rewards approach that includes pay, benefits, retirement savings, stock ownership opportunities, performance-based rewards and programs designed to support associates’ physical, mental and financial well-being. 

  • Competitive total rewards: TransUnion describes its benefits as part of a comprehensive Total Rewards package developed with associates’ physical, mental and financial well-being in mind. The company also says it rewards and supports associates through competitive pay, benefits and perquisite programs that help associates and their families thrive. External compensation reviews describe salary as “very decent,” total compensation as fair for the work and benefits as a strength.
  • Performance, equity and long-term value: TransUnion’s rewards package includes performance bonuses, company equity and an associates stock purchase plan. U.S. associates with at least three months of service can participate in the Employee Stock Purchase Plan, which allows them to buy TransUnion stock at a 15% discount off the closing stock price on the final day of the offering period. The company also offers two annual ESPP offering periods and 401(k) savings options to help associates build long-term financial security.
  • Financial wellness and retirement support: TransUnion offers regular 401(k) and Roth 401(k) options, with immediate eligibility for the company match once an associate starts contributing. The company matches dollar-for-dollar on the first 3% contributed and $0.50 per dollar on the next 2%, for a total 4% match. It also provides a discretionary company retirement contribution, financial wellness resources, long-term care insurance and charitable contribution matching.
  • Benefits that increase total value: TransUnion’s compensation package is supported by health coverage, dental, vision, mental wellness resources, adoption assistance, paid parental leave, caregiver support, tuition reimbursement, Flexible Time Off for salaried associates, paid holidays, Global Wellness Days, Volunteer Time Off, commuter benefits and associates discounts. A senior vice president for financial planning and analysis said, “The benefits provided by TransUnion truly exceeded my expectations,” underscoring how the company’s total rewards package can support associates through major life events.
  • External signals:
  • Compensation sentiment: External reviews cite fair total compensation, good retirement contributions, flexible PTO, annual bonus opportunities and strong benefits as positives. (Comparably; Glassdoor)
  • Benefits and rewards: Associates rate TransUnion’s perks and benefits A-, placing it in the top 15% of companies with 10,000+ Associates. (Comparably)
  • Employee feedback: Reviews also mention “amazing benefits,” “good compensation,” “good hikes” and a flexible schedule as strengths, while some note compensation can vary compared with industry standards. (Comparably; Indeed)

Bottom line: TransUnion keeps total rewards competitive by combining market-aligned pay, performance incentives, stock purchase opportunities, retirement support, financial wellness resources and benefits that support associates across work, family, health and long-term financial planning.

TransUnion Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Chicago, IL
13,000 Employees
Year Founded: 1968

What We Do

TransUnion is a global information and insights company that makes trust possible by ensuring that each consumer is reliably and safely represented in the marketplace. We do this by having an accurate and comprehensive picture of each person. This picture is grounded in our legacy as a credit reporting agency which enables us to tap into both credit and public record data; our data fusion methodology that helps us link, match and tap into the awesome combined power of that data; and our knowledgeable and passionate team, who stewards the information with expertise, and in accordance with local legislation around the world. Because of our work, organizations can better understand consumers in order to make more informed decisions, and earn their trust through great, personalized experiences, and the proactive extension of the right opportunities, tools and offers. In turn, consumers can be confident that their data identities will result in the opportunities they deserve. We make trust possible, so businesses and consumers can transact with confidence and achieve great things. We call this Information for Good®—it’s our purpose, and what drives us every day.

Why Work With Us

Our culture is welcoming, energetic and innovative. There’s an overall synergy that flows throughout TransUnion, creating a sense of unity in knowing that we’re all working to achieve the same overall goal. We’re dedicated to providing opportunities for our people to get involved and stay connected with their colleagues across the globe.

Gallery

Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery

TransUnion Teams

Team
Invested in Tech Teams
About our Teams

TransUnion Offices

Hybrid Workspace

Employees engage in a combination of remote and on-site work.

Typical time on-site: Flexible
HQChicago, IL
Amsterdam, NL
Bengaluru, IN
Boca Raton, FL
Bogotá, Colombia
Burlington, ON
South Africa
Cerqueira César, Sao Paulo
Chennai, IN
Cherry Hill, NJ
Cork, County Cork
Crum Lynne, PA
Denver, CO
Greenwood Village, CO
Guaynabo, PR
Gurugram, IN
Hamburg, DE
Hyderabad, IN
Johannesburg, ZA
TransUnion UK Head Office
London, GB
Louisville, KY
Madrid, ES
Makati, PH
Mexico City, MX
Mumbai, IN
New York, NY
Pune, IN
Reston, VA
San Luis Obispo, CA
Santiago, CL
Sydney, NSW
Toronto, ON
Ulloa, La Aurora
Washington, US
White Plains, NY
Learn more

Similar Jobs

TransUnion Logo TransUnion

Associate Developer, SQL and UNIX operating system

Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Hybrid
Hyderabad, Telangana, IND
13000 Employees

TransUnion Logo TransUnion

Platform Engineer

Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Hybrid
2 Locations
13000 Employees

TransUnion Logo TransUnion

Consultant

Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Hybrid
4 Locations
13000 Employees

TransUnion Logo TransUnion

Senior Consultant

Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Hybrid
4 Locations
13000 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account