Lead Data Engineer

We are looking for an experienced Lead Data Engineer to lead the architecture, evolution, and delivery of our modern AWS-native data platform. This platform powers underwriting intelligence, GenAI-driven insights, and large-scale content processing, serving as the foundation for data products across the organization.

You will own the full data lifecycle—from ingestion and transformation to storage, retrieval, and AI consumption—working across Python-based processing pipelines, .NET services, Aurora PostgreSQL, OpenSearch, and LangGraph-powered agentic workflows.

This is a senior individual contributor role with significant technical leadership responsibilities. You will define engineering standards, drive architectural decisions, mentor engineers, and collaborate closely with product, platform, and AI teams to build reliable, scalable, and high-performing data systems.

• Design, build, and evolve scalable, fault-tolerant data pipelines across ingestion, transformation, storage, and serving layers on AWS.
• Own the delivery of cloud-native data solutions, including AWS Batch, Lambda, Python, .NET APIs (REST/GraphQL), Aurora PostgreSQL, and OpenSearch.
• Define data architecture, schema design, indexing strategies, and data contracts to support analytics and GenAI-powered applications.
• Establish and enforce data quality, testing, monitoring, and operational standards using CloudWatch, Splunk, Pentaho, and ThoughtSpot.
• Partner with AI/ML and product teams to enable LangGraph-based agentic RAG solutions, ensuring reliable, high-quality, and retrieval-optimized data.
• Develop and optimize data pipelines supporting LLM evaluation, prompt engineering, ground-truth datasets, and synthetic test data generation.
• Review and validate AI-assisted engineering outputs, ensuring production readiness and technical excellence.
• Drive technical design through architecture documents, RFCs, and engineering best practices, balancing scalability, maintainability, and business needs.
• Lead deployment strategies, incident investigations, and continuous improvements to platform reliability and operational resilience.
• Mentor engineers, establish data engineering standards, and provide technical leadership through design reviews and hands-on guidance.
• Collaborate across engineering teams to manage data dependencies, align technical direction, and communicate delivery plans, risks, and architectural decisions.

• Solid software and data engineering experience building and operating scalable, production-grade data platforms.
• Strong Python skills for data processing and transformation, including structured and unstructured data.
• Experience building data services and APIs using .NET (C#), REST, and GraphQL.
• Hands-on expertise with AWS services, including S3, Batch, Lambda, Aurora PostgreSQL, EKS, SNS, and CloudWatch.
• Strong SQL and PostgreSQL skills, including schema design, query optimization, and data modeling.
• Experience designing distributed, fault-tolerant systems with robust testing, observability, and operational excellence.
• Proficiency troubleshooting complex data pipelines using logs, metrics, and monitoring tools.
• Experience with CI/CD, modern DevOps practices, and Agile software delivery.
• Excellent communication skills, with the ability to document technical designs and influence architectural decisions.

Nice to have:

• Experience building GenAI or Retrieval-Augmented Generation (RAG) applications using LangGraph, LangChain, or similar frameworks.
• Familiarity with OpenSearch, vector search, or AI retrieval architectures.
• Experience with API gateways (e.g., Kong) or Model Context Protocol (MCP).
• Experience processing unstructured documents and content transformation pipelines.
• Exposure to BI and analytics platforms such as ThoughtSpot or Pentaho.

We offer:

• The opportunity to own and shape a cloud-native data platform powering AI, analytics, and intelligent underwriting solutions.
• The chance to work with modern technologies, including AWS, GenAI, agentic workflows, and large-scale distributed data systems.
• A high-impact technical leadership role with the autonomy to influence architecture, engineering standards, and platform evolution.
• A collaborative, international engineering environment that values innovation, knowledge sharing, and continuous learning.
• A hybrid work model with flexible working hours.
• A benefits package, including private health insurance, medical care, and a Multisport card.

#LI-AA1

#LI-Hybrid    

Our People, Our Culture

For more than 50 years, Verisk has helped property and casualty insurers make smarter decisions about risk through AI-powered risk modeling, advanced analytics, and technology solutions spanning the entire policy lifecycle.  We are a leading strategic data, analytics, and technology partner to the global insurance industry, guided by core values of learning, caring, and results while maintaining the highest ethical standards as stewards of the industry's most comprehensive datasets. Learn more about Verisk and what we are doing within the insurance industry. 

For the eighth consecutive year, Verisk is proudly recognized as a Great Place to Work® for outstanding workplace culture in the US, the fourth consecutive year in the UK, Spain, and India, and the second consecutive year in Poland. In addition, we’ve been recognized by The Wall Street Journal as one of the Best-Managed Companies and by Forbes as a World’s Best Employer, testaments to the value we place on workplace culture. 

Our Culture: Explore our inclusive, people-first culture that fosters innovation, collaboration, and belonging.

Awards & Recognition: See why Verisk is consistently recognized as a Great Place to Work™ around the world.

Our Businesses: Learn about the diverse industries we serve — from insurance and energy to financial services and beyond.

Life at Verisk: Discover what it’s like to work at Verisk through employee stories, team highlights, and culture moments.

Careers at Verisk: Join a global team of problem-solvers and innovators doing meaningful work that’s shaping the future of industries. Whether you're just starting out or looking to take your career to the next level, Verisk offers growth, purpose, and a people-first culture.

Let’s build something meaningful together!

Verisk Analytics is an equal opportunity employer.

All members of the Verisk Analytics family of companies are equal opportunity employers. We consider all qualified applicants for employment without regard to race, religion, color, national origin, citizenship, sex, gender identity and/or expression, sexual orientation, veteran's status, age or disability. Verisk’s minimum hiring age is 18 except in countries with a higher age limit subject to applicable law.

Verisk offers a competitive salary aligned with market standards and your experience, disclosed in accordance with the Pay Transparency Directive. In addition to base pay, our compensation package includes a variety of incentive opportunities for eligible roles, designed based on market research and our internal grading structure.

You’ll also receive a comprehensive benefits package featuring private medical care, life insurance, Employee Capital Plans (PPK), as well as paid vacation and sick leave in line with the Polish Labour Code.

Verisk also supports flexible working arrangements through a hybrid work model, access to professional development programs, and a work environment focused on well-being and career growth. Additional perks include access to sports and wellness programs, well-being initiatives, and team-building events.

https://www.verisk.com/company/careers/

Unsolicited resumes sent to Verisk, including unsolicited resumes sent to a Verisk business mailing address, fax machine, or email address, or directly to Verisk employees, will be considered Verisk property. Verisk will NOT pay a fee for any placement resulting from the receipt of an unsolicited resume.

Verisk Employee Privacy Notice