Lead Data Architect

Posted 8 Hours Ago
Be an Early Applicant
Herndon, VA, USA
In-Office
Senior level
Information Technology • Software • Consulting
The Role
Lead the architecture and implementation of enterprise Databricks data platforms supporting GenAI, LLM, and ML workloads. Build scalable batch and streaming pipelines, operationalize model training and deployment, implement RAG and embedding solutions, and establish governance, observability, and cost optimization. The role requires hands-on cloud, Spark, Python, MLOps, infrastructure-as-code, and data engineering expertise, while also mentoring teams and leading architecture and code reviews.
Summary Generated by Built In
Overview

Summary

The Lead Data Architect will design, build, and operate enterprise data platforms that power GenAI and AI/ML use cases. This is a highly technical, hands-on role responsible for data platform architecture, end-to-end data engineering, ML/LLM pipeline design, production model onboarding, and delivery of scalable Databricks- centric solutions across cloud environments.

Responsibilities

What You'll Be Doing:

  • Architect and implement enterprise data platforms (batch + streaming) optimized for ML, LLMs, and GenAI workloads.
  • Lead design and hands on implementation of Databricks workspaces, Unity Catalog, Delta Lake design patterns, cluster policies, and performance tuning.
  • Build and own end to end data pipelines (ingest, transform, feature engineering, serving) using PySpark, Databricks Jobs, Spark SQL, Delta Lake, and orchestration tools.
  • Design and operationalize model training, fine tuning (LLM), evaluation, deployment, and monitoring pipelines (MLOps/RAG/CAG) integrating Databricks MLflow, CI/CD, and infra-as-code.
  • Implement vectorless and vectorization/embedding pipelines, vector store integrations, and retrieval layers for RAG (FAISS, Pinecone, Weaviate, Milvus).
  • Define data schemas, governance, lineage, access controls, and data product APIs; implement Unity Catalog or equivalent for centralized governance.
  • Drive cost/performance optimization for storage, compute (spot/preemptible),and query patterns. 
  • Collaborate with engineers, data scientists, product owners, and security to translate business needs into production GenAI solutions. 
  • Mentor and lead engineering teams; conduct architecture reviews, code reviews, and run technical deep dives. 
  • Implement observability for data and ML pipelines (metrics, logging, data quality tests, alerting). 
  • Create reproducible experiment tracking, model registry, and rollout strategies (canary, shadow testing, rollback). 
  • Stay current on GenAI/LLM architectures and evaluate/introduce new tooling and frameworks.
Qualifications and Education

Required Qualifications:

  • BA or BS degree in CS, Computer Engineering, Information Technology or a related field.
  • 8+ years hands on experience in data engineering/platform architecture; 3+ years in an architect or lead role.
  • Candidate must hold an active AWS Certified Machine Learning – Specialty certification or equivalent AWS certification.
  • Proven, hands on Databricks experience (designing workspaces, Delta Lake, performance tuning, productionizing Spark jobs).
  • Deep Spark + PySpark expertise and experience with Databricks Runtime.
  • Strong experience building ML/LLM pipelines and operationalizing models (training, fine tuning, serving).
  • Practical experience with vector embeddings, semantic search, and RAG architectures.
  • Solid Python expertise and common ML libraries (PyTorch, TensorFlow, Hugging Face transformers) and MLflow.
  • Cloud platform experience (AWS strongly preferred).
  • Experience with containerization and orchestration while leveraging open source libraries for unstructured and structured data processing, serving/inference.
  • Strong SQL skills; experience with distributed query/warehouse systems and parquet/AVRO/Delta formats.
  • CI/CD and infra-as-code experience (Terraform, GitOps, Jenkins/GitHub Actions/GitLab CI).
  • Data governance, security, and IAM experience; experience implementing row/column level access controls and data lineage.
  • Demonstrated ability to design for scalability, reliability, and cost efficiency. 
Compensation

The proposed salary range for this role is $****** to $******* USD. The salary range provided is a good faith estimate representative of all experience levels. Karsun considers several factors when extending an offer, including but not limited to, the role, function and associated responsibilities, a candidate’s work experience, location, education/training, and key skills.

Skills Required

  • Bachelor's degree in Computer Science, Computer Engineering, Information Technology, or a related field
  • 8+ years of hands-on experience in data engineering or platform architecture
  • 3+ years of experience in an architect or lead role
  • Active AWS Certified Machine Learning Specialty certification or equivalent AWS certification
  • Hands-on Databricks experience, including workspace design, Delta Lake, performance tuning, and productionizing Spark jobs
  • Deep Apache Spark and PySpark expertise, including Databricks Runtime
  • Experience building ML and LLM pipelines and operationalizing model training, fine-tuning, and serving
  • Practical experience with vector embeddings, semantic search, and RAG architectures
  • Strong Python expertise and experience with PyTorch, TensorFlow, Hugging Face Transformers, and MLflow
  • Cloud platform experience, preferably AWS
  • Experience with containerization, orchestration, and open-source libraries for data processing and model serving
  • Strong SQL skills and experience with distributed query or warehouse systems and Parquet, Avro, or Delta formats
  • CI/CD and infrastructure-as-code experience using Terraform, GitOps, Jenkins, GitHub Actions, or GitLab CI
  • Data governance, security, IAM, row- and column-level access controls, and data lineage experience
  • Ability to design for scalability, reliability, and cost efficiency
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
316 Employees
Year Founded: 2009

What We Do

Karsun Solutions is an enterprise modernization and IT services firm helping government agencies modernize legacy systems and improve operations. It delivers modern software development, cloud, and data solutions, along with artificial-intelligence capabilities. Its work centers on secure digital architecture, operational efficiency, and strengthening agency capabilities to advance mission outcomes. The company also provides modernization expertise across acquisitions, aviation, fleet, and grants management.

Similar Jobs

Genworth Logo Genworth

Data Architect

Fintech • Insurance
In-Office or Remote
Richmond, VA, USA
5001 Employees
121K-234K Annually
In-Office
2 Locations
12000 Employees
135K-225K Annually

Inadev Logo Inadev

Data Architect

Information Technology
In-Office
Reston, VA, USA
120 Employees
205K-220K

Booz Allen Hamilton Logo Booz Allen Hamilton

Data Architect

Information Technology
In-Office
Chantilly, VA, USA
99K-225K Annually

Similar Companies Hiring

Revel Thumbnail
Aerospace • Hardware • Robotics • Software
Marina Del Rey, California
70 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
33 Employees
RapidScale Thumbnail
Cloud • Information Technology • Consulting
Raleigh, NC
600 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account