Senior Data Scientist- Machine Learning & Data Engineering

National General is a part of The Allstate Corporation, which means we have the same innovative drive that keeps us a step ahead of our customers’ evolving needs. We offer home, auto and accident and health insurance, as well as other specialty niche insurance products, through a large network of independent insurance agents, as well as directly to consumers. 

Job Description

We are seeking a Senior Data Scientist with strong Machine Learning & Data Engineering skills, focused on developing intelligent systems for parsing, mapping, forecasting, and deriving insight from structured and unstructured data. This hybrid role emphasizes machine learning, AI-assisted automation, and predictive analytics (70%), while also requiring foundational data engineering experience (30%) to support scalable, cloud-native data workflows.
You will work on a wide range of analytical problems — from automated document understanding to predictive modeling — and help design the infrastructure needed to support these solutions in production. Ideal candidates combine a deep understanding of data science methods with the practical know-how to operationalize them efficiently and securely.

Key Responsibilities:

Data Science & AI/ML (70%)

• Design and implement machine learning models for schema inference, semantic field mapping, and entity recognition from PDFs, Excel, and CSV data sources.

• Leverage Large Language Models (LLMs) and AI frameworks (e.g., GPT, LangChain, unstructured.io) for document understanding, schema matching, and zero/low-shot data extraction.

• Develop forecasting, extrapolation, and predictive analysis models to inform business and compliance decision-making.

• Apply statistical analysis and data mining techniques to identify trends, anomalies, and actionable insights.

• Collaborate with stakeholders to define analytical goals and translate them into modeling strategies.

• Communicate model results and explain to non-technical teams through dashboards, reports, or presentations

Data Engineering (30%)

• Build and maintain scalable ETL/ELT pipelines to support model training and inference workloads.

• Normalize and integrate data from disparate sources to support both analytical and operational needs.

• Optimize ingestion and storage in PostgreSQL using indexing, partitioning, and performance tuning.

• Support production deployment of models in cloud-native environments (Azure, Docker, Kubernetes).

• Ensure traceability and auditability of data pipelines and transformations through proper documentation and versioning.

Required Qualifications:

• 5+ years of experience in data science, applied machine learning, or a hybrid data science/engineering role. (Preferred)

• Strong Python skills, including libraries such as pandas, scikit-learn, PyMuPDF, pdfplumber, tabula-py, openpyxl.

• Hands-on experience with LLM-based tools for document parsing, information extraction, or schema inference.

• Proficient in SQL with an emphasis on PostgreSQL — capable of writing performant queries and managing data models.

• Experience developing and operationalizing predictive and classification models in a real-world setting.

• Working knowledge of cloud-based data systems (e.g., Azure Data Factory, AWS Glue, GCP Functions).

• Ability to work independently and communicate complex ideas clearly across technical and non-technical teams.

Preferred Qualifications:

• Familiarity with document AI APIs (e.g., Azure Form Recognizer, AWS Textract, Google Document AI).

• Experience building automated workflows for model training, evaluation, and monitoring.

• Understanding of data governance, compliance, and security best practices.

• Exposure to containerized deployments and orchestration using Docker and Kubernetes.

• Experience working in highly regulated environments (healthcare, insurance, finance) is a plus.

Education

Masters Degree (Preferred)

Supervisory Responsibilities

This job does not have supervisory duties.

Education & Experience (in lieu)

In lieu of the above education requirements, an equivalent combination of education and experience may be considered.

#LI-MF1

Compensation

Base compensation offered for this role is $95,700.00 - $152,000.00 annually and is based on experience and qualifications.

 

*** Total compensation for this role is comprised of several factors, including the base compensation outlined above, plus incentive pay (i.e., commission, bonus, etc.) as applicable for the role.

At National General, great things happen when our people work together. That’s why when you join our team, we make sure it isn’t just a job — it’s an opportunity. One that takes your skills and pushes them to the next level. One that encourages you to challenge the status quo. And one where you can impact the future for the greater good.  

You’ll do all this in a flexible environment that embraces connection and belonging. And with the recognition of several inclusivity and diversity awards, we’ve proven that Allstate empowers everyone to lead, drive change and give back where they work and live. 

Good Hands. Greater Together.

National General Holdings Corp., a member of the Allstate family of companies, is headquartered in New York City. National General traces its roots to 1939, has a financial strength rating of A– (excellent) from A.M. Best, and provides personal and commercial automobile, homeowners, umbrella, recreational vehicle, motorcycle, supplemental health, and other niche insurance products. We are a specialty personal lines insurance holding company. Through our subsidiaries, we provide a variety of insurance products, including personal and commercial automobile, homeowners, umbrella, recreational vehicle, supplemental health, lender-placed and other niche insurance products.

Companies & Partners

Direct General Auto & Life, Personal Express Insurance, Century-National Insurance, ABC Insurance Agencies, NatGen Preferred, NatGen Premier, Seattle Specialty, National General Lender Services, ARS, RAC Insurance Partners, Mountain Valley Indemnity, New Jersey Skylands, Adirondack Insurance Exchange, VelaPoint, Quotit, HealthCompare, AHCP, NHIC, Healthcare Solutions Team, North Star Marketing, Euro Accident.

Benefits

National General Holdings Corp. is an Equal Opportunity (EO) employer – Veterans/Disabled and other protected categories. All qualified applicants will receive consideration for employment regardless of any characteristic protected by law. Candidates must possess authorization to work in the United States, as it is not our practice to sponsor individuals for work visas. In the event you need assistance or accommodation in completing your online application, please contact NGIC main office by phone at (336) 435-2000.