The Role
Lead end-to-end AI/ML, data, and ETL architecture for regulated insurance platforms. Design lakehouse and Medallion architectures, feature stores, MLOps and GenAI/RAG solutions, scalable batch and streaming pipelines, canonical and dimensional data models, governance controls, security, lineage, and compliance frameworks. Guide technology selection, performance and cost optimization, engineering standards, and architecture reviews with senior client stakeholders.
Summary Generated by Built In
DATAECONOMY is one of the fastest-growing Data & Analytics company with global presence. We are well-differentiated and are known for our Thought leadership, out-of-the-box products, cutting-edge solutions, accelerators, innovative use cases, and cost-effective service offerings.
We offer products and solutions in Cloud, Data Engineering, Data Governance, AI/ML, DevOps and Blockchain to large corporates across the globe. Strategic Partners with AWS, Collibra, cloudera, neo4j, DataRobot, Global IDs, tableau, MuleSoft and Talend.
AI/ML Data & ETL Data Architect
Charlotte, NC
Full-time
Key Responsibilities
- Architect feature stores, training/inference pipelines, and MLOps workflows for insurance use cases fraud detection, claims triage, underwriting risk scoring, loss reserving, and customer churn/retention.
- Design RAG and GenAI solution patterns for claims summarization, policy/document intelligence, and underwriter/agent copilots.
- Establish model lifecycle controls: versioning, lineage, drift monitoring, evaluation, and human-in-the-loop review.
- Define responsible-AI and governance guardrails appropriate to a regulated insurance environment (auditability, explainability, bias monitoring).
- Own the end-to-end target-state architecture for the insurance data platform policy administration, claims, billing, underwriting, actuarial, and reinsurance domains across raw, curated, and analytics-ready layers.
- Design lakehouse and AI/ML reference architectures (Bronze/Silver/Gold Medallion) that unify structured, semi-structured, and streaming insurance data.
- Define data domain boundaries, source-to-target mappings, and canonical insurance data models for shared enterprise consumption.
- Produce architecture diagrams, design decision records, and patterns that engineering teams can implement consistently.
- Make build-vs-buy, cloud service selection, and cost/performance trade-off decisions and defend them to client architecture review boards.
- Design scalable, production-grade ETL/ELT frameworks (PySpark, Spark SQL, Delta Live Tables / equivalent, orchestrated Workflows).
- Define ingestion patterns for batch, micro-batch, and streaming insurance feeds (policy, claims, payments, third-party/bureau data).
- Establish orchestration, monitoring, alerting, and automation standards for the engineering team.
- Design dimensional models (star/snowflake) and canonical/conformed models for analytical and actuarial workloads.
- Apply normalization/denormalization strategies balancing performance, usability, and regulatory traceability.
- Ensure data quality, integrity, and alignment with enterprise and insurance regulatory governance policies.
- Embed PII/PHI handling, masking, tokenization, and least-privilege access models into platform design.
- Align architecture with insurance regulatory and audit requirements (e.g., NAIC model standards, state DOI, HIPAA where health lines apply, SOC 2, GDPR/CCPA).
- Define metadata management, data lineage, and cataloging strategy (Unity Catalog or equivalent).
- Advanced hands-on data engineering: Spark, Delta Lake / lakehouse, Workflows, Unity Catalog (or cloud-native equivalents).
- AI/ML tooling: MLflow or equivalent, feature stores, model serving, and GenAI/RAG frameworks (LangChain/LangGraph or similar).
- Strong SQL and Python programming with performance tuning skills.
- Cloud platform depth (AWS / Azure / GCP), including managed data and ML services.
- Hands-on AI/ML pipeline and MLOps experience, including at least one production GenAI/RAG deployment.
- Strong command of Medallion architecture (Bronze/Silver/Gold) and modern data modeling for warehousing and analytics.
- Proficiency with PySpark, SQL, ETL/ELT frameworks, and Delta Lake (or equivalent) optimization.
- Experience with CI/CD, Git, and job orchestration tooling.
- Insurance, financial services, or other regulated-industry delivery experience.
- Demonstrated ability to present and defend architecture to senior client and review-board stakeholders.
- Data governance, metadata management, and Unity Catalog (or equivalent) advanced features.
- Streaming technologies (Auto-Loader / Structured Streaming / Kafka / Event Hubs / Kinesis).
- Data security, regulatory compliance, and fine-grained access models.
- Cost optimization and performance tuning in cloud environments.
- Responsible-AI / model governance frameworks (e.g., NIST AI RMF).
- Tools such as Airflow, Databricks Workflows, dbt, or similar.
Requirements
- Data architecture leadership lakehouse / Medallion (Bronze/Silver/Gold) target-state design
- Strong Python (PySpark) and SQL programming with performance tuning
- Databricks (or equivalent) Spark, Delta Lake, Workflows, Unity Catalog
- ETL/ELT framework design and data modeling (dimensional, star/snowflake, canonical)
- AI/ML pipelines + MLOps, plus at least one production GenAI/RAG deployment
- Cloud experience AWS, Azure, or GCP (managed data + ML services)
- CI/CD, Git, job orchestration
- 12+ years total; 3+ years as architect/lead; regulated-industry delivery
Benefits
Standard full-time benefits
Skills Required
- Hands-on AI/ML pipeline and MLOps experience, including at least one production GenAI/RAG deployment
- Strong command of Medallion architecture and modern data modeling for warehousing and analytics
- Proficiency with PySpark, SQL, ETL/ELT frameworks, and Delta Lake optimization
- Experience with CI/CD, Git, and job orchestration tooling
- Insurance, financial services, or other regulated-industry delivery experience
- Ability to present and defend architecture to senior client and review-board stakeholders
- Data governance, metadata management, and advanced Unity Catalog features
- Streaming technologies including Auto-Loader, Structured Streaming, Kafka, Event Hubs, or Kinesis
- Data security, regulatory compliance, and fine-grained access models
- Cost optimization and performance tuning in cloud environments
- Responsible-AI and model governance frameworks such as NIST AI RMF
- Experience with Airflow, Databricks Workflows, dbt, or similar tools
- 12+ years of total experience
- 3+ years of experience as an architect or lead
- Cloud experience with AWS, Azure, or GCP managed data and ML services
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
DATAECONOMY is a global, cloud-first data and AI consultancy delivering enterprise-grade solutions through an innovative intellectual-property suite. Its work spans data and BI platform modernization, self-service AI, data mesh and fabric, master data management, governance, cloud enablement, digital engineering, knowledge graphs, and machine lakes supporting cybersecurity and financial-crime use cases for enterprise clients.
