-
We are looking for a Senior Data Engineer to play a key senior technical role on our Data Engineering team, helping optimize and scale our existing large-scale data platform. This is a hands-on engineering role with a strong technical leadership component. You will design and build data pipelines and platform improvements while setting a strong technical example, reviewing the work of other engineers, and serving as a go-to resource for complex architecture and data engineering challenges.
Reporting directly to the Director, Data Engineering, you will work alongside a small, focused team to build reliable, scalable data products and maintain a high-performing platform that supports evolving analytics and AI/ML needs. This role is well suited to an experienced data engineer seeking significant hands-on ownership, along with opportunities to mentor others and influence technical practices across the team.
Key Accountabilities/Deliverables:
Hands-On Engineering & Technical Ownership
Design, build, and optimize data pipelines, ingestion frameworks, and platform components that support analytics, reporting, and downstream AI/ML use cases.
Take direct ownership of high-complexity or high-risk engineering initiatives, from technical design through implementation and rollout.
Identify and resolve performance, scalability, and reliability bottlenecks across the existing data platform.
Contribute to the platform's AI/ML readiness by improving data models, governance, and access patterns (e.g., feature stores, curated datasets, streaming access).
Diagnose and resolve complex production issues, including on-call support for critical data pipeline failures.
Write clean, well-tested, well-documented code and infrastructure-as-code, and maintain strong engineering hygiene across your work.
Technical Leadership & Collaboration
Act as a senior technical resource for the team, providing guidance on architecture, design patterns, and best practices.
Conduct code and design reviews, helping raise the technical bar across the Data Engineering team.
Partner closely with the Director, Data Engineering on technical design decisions, tooling evaluations, and platform roadmap input.
Mentor other data engineers, sharing knowledge and helping develop their technical skills.
Partner cross-functionally with Product and Engineering stakeholders to understand data needs and translate them into technical solutions.
Help document standards, runbooks, and architectural decisions to support team scalability and knowledge sharing.
- Other duties as assigned.
Technical Knowledge and Understanding:
Deep expertise in data pipeline design, optimization, and distributed data processing architectures with experience identifying and resolving performance and scalability bottlenecks
Strong knowledge of cloud data platforms (AWS, Azure, or GCP) and modern data warehouse/Lakehouse architectures (Delta Lake, Iceberg)
Strong working proficiency with at least two programming languages (Python, Scala, SQL)
Solid understanding of data governance, security, and compliance practices in a regulated or enterprise environment
Ability to troubleshoot and reverse engineer existing codebases and data pipeline
Working knowledge of how data platforms enable AI/ML solutions
Experience:
Applicants must be authorized to work for any employer in the U.S. We are unable to sponsor or take over work authorization sponsorship now or in the future for this position.
Bachelor's degree in related field or demonstrated equivalent experience in a related field required.
7+ years of experience in data engineering, with experience working on large-scale, mature data platforms.
Demonstrated experience taking technical ownership of complex projects from design through delivery.
Experience contributing to data platforms that support downstream AI/ML use cases (e.g., feature stores, curated/governed datasets, streaming access for model training and inference).
Experience with IAC and CI/CD practices for data systems (i.e. Terraform, GitHub Actions)
Strong experience writing code modular code with notebooks
Platforms: Hands on experience with Snowflake or Databricks, Azure Synapse Analytics/MS Fabric
Experience mentoring or informally guiding other engineers, even without formal people management responsibility.
Experience optimizing the developer experience
Experience designing or implementing Agentic workflows for data engineering preferred
Experience working with Property & Casualty insurance carriers preferred
Experience with Data Vault 2.0 or Ensemble data modeling techniques preferred
#LI-Hybrid
-
At Core Specialty, you will receive a competitive salary and opportunities for professional development and advancement. We offer medical, dental, vision, and life insurances; short and long-term disability; a Company-match of 100% of a 6% contribution 401(k) plan; an Employee Assistance Plan; Health Savings Account, Flexible Spending Account, Health Reimbursement Account, and a wellness program
Skills Required
- Bachelor's degree in a related field or demonstrated equivalent experience
- 7+ years of experience in data engineering
- Experience working on large-scale, mature data platforms
- Experience taking technical ownership of complex projects from design through delivery
- Experience contributing to data platforms supporting downstream AI/ML use cases
- Experience with infrastructure-as-code and CI/CD practices for data systems
- Strong experience writing modular code with notebooks
- Hands-on experience with Snowflake or Databricks and Azure Synapse Analytics or Microsoft Fabric
- Experience mentoring or informally guiding other engineers
- Experience optimizing developer experience
- Proficiency in at least two of Python, Scala, and SQL
- Experience designing or implementing agentic workflows for data engineering
- Experience working with Property and Casualty insurance carriers
- Experience with Data Vault 2.0 or Ensemble data modeling techniques
- Authorization to work for any employer in the United States without sponsorship
What We Do
Core Specialty, through its subsidiary insurers, offers a diversified range of property, casualty, and marine insurance products for small to mid-sized businesses. We have the capital to take on risk, the underwriting talent in place, decisive leadership team, infrastructure, and a proven track record of making things happen – fast! When you’re ready to solve your toughest insurance needs, we’re ready to get it done for you. We free customers up to focus on their business by taking the load of complicated specialty insurance off their hands. We break down the walls of bureaucracy to provide optimal underwriting solutions for brokers. We’re ready, equipped, and motivated to get the job done, efficiently and professionally, by empowering experts with what they need to move quickly on behalf of customers and their brokers. Our specialty focus is the essential part of our identity. It is at our core
