Data Platform Architect

Posted One Month Ago
Be an Early Applicant
Hiring Remotely in Chennai, Tamil Nadu, IND
In-Office or Remote
Senior level
Other
The Role
Lead design and stewardship of a scalable, cost-effective lakehouse data platform. Define standards, data governance, performance and FinOps practices, review designs and code, mentor engineers, and maintain architectural decision records to align platform evolution with business needs.
Summary Generated by Built In

About TaskUs: TaskUs is a provider of outsourced digital services and next-generation customer experience to fast-growing technology companies, helping its clients represent, protect and grow their brands. Leveraging a cloud-based infrastructure, TaskUs serves clients in the fastest-growing sectors, including social media, e-commerce, gaming, streaming media, food delivery, ride-sharing, HiTech, FinTech, and HealthTech. 


The People First culture at TaskUs has enabled the company to expand its workforce to approximately 60,000 employees globally. Presently, we have a presence in twenty-three locations across twelve countries, which include the Philippines, India, and the United States.


It started with one ridiculously good idea to create a different breed of Business Processing Outsourcing (BPO)! We at TaskUs understand that achieving growth for our partners requires a culture of constant motion, exploring new technologies, being ready to handle any challenge at a moment’s notice, and mastering consistency in an ever-changing world.


What We Offer: At TaskUs, we prioritize our employees' well-being by offering competitive industry salaries and comprehensive benefits packages. Our commitment to a People First culture is reflected in the various departments we have established, including Total Rewards, Wellness, HR, and Diversity. We take pride in our inclusive environment and positive impact on the community. Moreover, we actively encourage internal mobility and professional growth at all stages of an employee's career within TaskUs. Join our team today and experience firsthand our dedication to supporting People First.

Role Overview:

We are seeking an Enterprise Data Platform Architect to design and build a
next-generation, vendor-agnostic Data Platform from the ground up. You will own the
foundational platform architecture—decoupling compute from storage using Apache Iceberg,
designing containerized execution engines on Kubernetes, establishing multi-engine processing
(DuckDB, PyIceberg, dbt, serverless query engines), and embedding automated governance
from Day 1.

Work Mode : Remote

What You Will Own:
• Custom Platform Architecture: Lead design decisions for a decoupled Lakehouse
architecture on AWS and Kubernetes, establishing storage layout, partition evolution, and
open table formats (Apache Iceberg).
• Multi-Engine Compute Optimization: Match specific workloads to appropriate engines (e.g.,
DuckDB for in-memory batch processing, PyIceberg for atomic writes, serverless engines
for query serving) to maximize speed while minimizing cloud spend.
• Framework & Runner Engineering: Define config-driven pipeline patterns and containerized
execution runners that automatically handle idempotency, schema evolution, advisory
locking, and step-level logging.
• Embedded Governance & Cataloging: Architect automated, zero-trust data governance
from Day 1, integrating open-source catalogs (Apache Polaris, Glue) with column-level
security, PII discovery, and automated audit trails.
• Data Quality Frameworks: Design multi-layer, config-driven data quality gates that execute
inline in-memory and trigger automated pipeline halts before bad data propagates.
• Architectural Decision Records (ADRs): Maintain a version-controlled repository of
technical decisions, documenting trade-offs, compute boundaries, and long-term
implications.

Core Technical Qualifications:
• 8+ Years in Platform Architecture / Infrastructure Engineering: Proven track record of
designing and deploying production-grade, open-standard Data Platforms from scratch.
• Open Table Formats & Decoupled Storage: Deep expertise in Apache Iceberg (or Delta
Lake), including native PyIceberg APIs, ACID transactions on object storage (S3),
schema/partition evolution, and metadata management.
• High-Efficiency & Embedded Compute: Hands-on experience with in-process/vectorized
engines (DuckDB, PyArrow) and matching workload scales to the right compute engine
(single-node vs. distributed).
• Kubernetes & Container Orchestration: Experience in Kubernetes (EKS/K8s)—specifically
Pods, Deployments, ConfigMaps, Secrets, and custom containerized execution runners.
• Advanced Orchestration: Expert-level Apache Airflow skills, specifically designing dynamic,
dependency-aware DAGs and managing worker architecture in resource-constrained
environments.
• Transformation & Semantic Layers: Deep proficiency with dbt Core (SQL
compilation/modeling) paired with external execution engines and semantic YAML layer
definitions.
• Open-Source Catalogs & Governance: Experience implementing open catalog
specifications (Apache Polaris, Iceberg REST Catalog) and metadata platforms

(OpenMetadata, AWS Lake Formation).
• Cloud Infrastructure (AWS): Advanced knowledge of AWS core services (S3, EKS, IAM,
EC2, Serverless Query Serving) with strict adherence to storage-compute separation and
FinOps spend optimization.

Preferred / Bonus Qualifications (Pluses):
• Databricks & Snowflake Experience (Plus): Experience with Databricks (Lakehouse/Unity
Catalog) or Snowflake is a plus, particularly in the context of legacy migrations, hybrid
query serving, or cross-engine query federation.
• DataOps & CI/CD: Hands-on experience establishing automated testing, blue/green
deployment patterns for data models, and GitOps workflows.
• Executive Communication: Proven track record translating technical debt, compute
benchmarks, and architectural roadmaps for executive leadership and non-technical
stakeholders.

How We Partner To Protect You: TaskUs will neither solicit money from you during your application process nor require any form of payment in order to proceed with your application. Kindly ensure that you are always in communication with only authorized recruiters of TaskUs.

DEI: In TaskUs we believe that innovation and higher performance are brought by people from all walks of life. We welcome applicants of different backgrounds, demographics, and circumstances. Inclusive and equitable practices are our responsibility as a business. TaskUs is committed to providing equal access to opportunities. If you need reasonable accommodations in any part of the hiring process, please let us know.

We invite you to explore all TaskUs career opportunities and apply through the provided URL https://www.taskus.com/careers/.

Skills Required

  • 8+ years in Data Engineering or Data Architecture delivering production-grade Lakehouse environments for high-concurrency organizations
  • Extensive experience with Databricks and Unity Catalog
  • Experience with high-performance warehouses such as Amazon Redshift or Snowflake
  • Deep hands-on expertise with open-table formats (Apache Iceberg or Delta Lake) including partitioning and schema evolution
  • Mastery of dbt (Core) for complex SQL-based modeling
  • Proficiency in PySpark or Python for sophisticated data processing
  • Advanced experience designing resilient, dependency-aware Apache Airflow DAGs
  • Expert-level knowledge of cloud ecosystems (AWS S3, EC2, IAM) or equivalent Azure/GCP services with storage-compute separation
  • Deep understanding of Kubernetes concepts: Pods, Deployments, Services, ConfigMaps, Secrets
  • Experience with open-source catalog implementations like Apache Polaris
  • Knowledge of DataOps principles and automated data quality/testing frameworks
  • Familiarity with vectorized/embedded engines like DuckDB for specialized or cost-sensitive processing
  • Experience translating technical debt and architectural roadmaps for C-level executives
  • Background managing fixed-resource infrastructure (EC2/VM-based) versus elastic serverless models

TaskUs Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about TaskUs and has not been reviewed or approved by TaskUs.

  • Healthcare Strength — Core U.S. materials and filings show medical, dental, vision, life insurance, and short- and long‑term disability are standard offerings. Feedback suggests the company consistently emphasizes solid health coverage across roles, with country‑specific enhancements in some markets.
  • Retirement Support — Company filings confirm a 401(k) Savings Plan with an employer match (effectively up to a 4% match under the cited formula). Feedback suggests the program has clear categories and defined terms that candidates can verify at offer stage.
  • Wellbeing & Lifestyle Benefits — Careers materials highlight robust wellness programs, including a clinician‑led Wellness & Resiliency focus and other culture perks. Feedback suggests these wellbeing supports are a notable differentiator versus typical sector offerings.

TaskUs Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Santa Monica, CA
10,564 Employees
Year Founded: 2008

What We Do

TaskUs is a different breed of BPO. We are a collective of highly capable humans, who understand how to deploy technology and data to best serve your purpose. From Digital CX to Content Security, Data & AI Operations, Consulting, and anything in between, we consider ourselves responsible for protecting our partners’ interests and supporting their long term success through innovation and technology - powered by ridiculously smart people. TaskUs partners with the world’s most innovative and disruptive brands to protect what matters most and to thrive in an ever changing world.

Similar Jobs

Remote
India
1201 Employees

Zapier Logo Zapier

Director, India

Artificial Intelligence • Productivity • Software • Automation
Remote
India
800 Employees

The Aerospace Corporation Logo The Aerospace Corporation

Advanced Systems Engr

Aerospace • Artificial Intelligence • Cloud • Machine Learning • Software • Cybersecurity • Defense
Remote or Hybrid
India
4600 Employees

Optum Logo Optum

Machine Learning Engineer

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Remote
India
160000 Employees

Similar Companies Hiring

T-Mobile Thumbnail
Other • Utilities
Bellevue, WA
89016 Employees
Rosendin Thumbnail
Other • Manufacturing
San Jose, CA
6219 Employees
OmniCable Thumbnail
Other
Houston, Texas
815 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account