Join Starr, a global leader in commercial insurance with over a century of expertise. We empower our employees to innovate, make impactful decisions, and build lasting client relationships worldwide. At Starr, you'll work in an entrepreneurial culture alongside accessible leaders, leveraging our financial strength and vast industry experience to deliver solutions for our clients, no matter how complex. Grow your career with a rapidly growing company that invests in its people and their ability to drive real progress.
Security Observability Engineer
Job Overview
We are seeking an experienced Security Observability Engineer to lead the migration, optimization, and secure operation of our log ingestion and observability pipelines. The role emphasizes secure data delivery, advanced SIEM coverage, Splunk expertise, data reduction strategies, and robust load balancing and high availability for log infrastructure.
Key Responsibilities
• End-to-End SIEM Pipeline Management & Optimization
• Lead the migration of log sources from Splunk ingestion to Cribl Stream/Edge
pipelines, ensuring load-balanced, fault-tolerant delivery and processing of security and IT logs.
• Architect and manage scalable, resilient pipelines—including design,
implementation, and operation of load balancing solutions (e.g., Cribl worker groups, external load balancers, or syslog load distribution) for high ingest volumes.
• Analyze, tune and securely onboard all log sources (firewalls, EDR, cloud,
authentication, proxies, etc.)—covering parsing, filtering, and data reduction techniques to minimize Splunk ingest/storage costs without sacrificing security coverage.
• Develop and maintain Cribl and Splunk configurations, including advanced
transformations, field normalization, masking, and enrichment for security analytics.
• Ensure optimal distribution of logging workload across Cribl worker nodes and
Splunk indexers to prevent bottlenecks, data loss, or single points of failure. • Security Event Visibility and SOC Enablement
• Collaborate with SOC, IR, and threat detection teams to ensure all security logs
reach the SIEM eRiciently and reliably, and logs are tuned for actionable detection. • Actively monitor, test, and remediate pipeline balancing and ingestion health to
maximize uptime and forensic visibility.
• Respond to and resolve SIEM and pipeline issues that impact security, detection, or
compliance visibility.
• Governance, Compliance & Documentation
• Apply and document security policies for log routing, load balancing, event
retention, and integrity—ensuring pipeline architecture meets audit, legal, and privacy requirements.
• Track and report ingest reduction, Splunk cost savings, pipeline health, and event
loss/drop rates across the load-balanced infrastructure.
• Maintain clear, up-to-date documentation of log flows, pipeline topology, load
balancing strategies, and operational procedures.
Required Skills & Qualifications
• Extensive hands-on experience with Splunk SIEM engineering (indexers, search
heads, clustering, forwarders, CIM, performance/load optimization).
• 2+ years with Cribl Stream/Edge, including deployment and tuning of distributed,
load-balanced pipelines.
• Deep understanding of machine data transport (syslog, HEC, TCP, UDP), log
balancing strategies (syslog balancers, DNS round-robin, Cribl worker groups, etc.), and high-availability logging environments.
• Proven expertise onboarding, parsing, and tuning security log sources (firewalls,
cloud, EDR/XDR, IAM, authentication, and networking) for best possible coverage and SOC/IR support.
• Advanced Splunk SPL, data model, event parsing, and alert tuning skills; practical
SIEM optimization experience.
• Scripting/automation ability (Python, shell/CLI, or similar) for pipeline management
and validation.
• Strong troubleshooting, monitoring, and operational dashboard skills for both
pipeline and SIEM health.
Preferred Qualifications
• Hands-on experience designing and operating clustered/HA Cribl and Splunk
deployments (worker groups, clustered indexers, resilient data forwarders, etc.). • Splunk ES or Cribl certifications.
Key Success Metrics
• No loss of security data or alert coverage during/after pipeline migration and
optimization.
• Documented, measurable reductions in Splunk ingest volume and
operational/storage cost.
• Consistently balanced log throughput and minimal risk of bottlenecks or
overloads—pipeline health and reliability metrics maintained above SLA. • Well-documented, adaptable pipeline and load balancing architecture.
The estimated salary range for this position is 90k-120k
Starr is an equal opportunity employer, which means we'll consider all suitably qualified applicants regardless of gender identity or expression, ethnic origin, nationality, religion or beliefs, age, sexual orientation, disability status or any other protected characteristic. We recruit and develop our people based on merit and we're committed to creating an inclusive environment for all employees. We offer first class training and development opportunities to all employees. Our aim is to grow our own talent and bring out the best in people.
Skills Required
- Extensive hands-on experience with Splunk SIEM engineering (indexers, search heads, clustering, forwarders, CIM, performance/load optimization).
- 2+ years with Cribl Stream/Edge, including deployment and tuning of distributed, load-balanced pipelines.
- Deep understanding of machine data transport (syslog, HEC, TCP, UDP) and log balancing strategies (syslog balancers, DNS round-robin, Cribl worker groups).
- Proven expertise onboarding, parsing, and tuning security log sources (firewalls, cloud, EDR/XDR, IAM, authentication, networking).
- Advanced Splunk SPL, data model, event parsing, and alert tuning skills; practical SIEM optimization experience.
- Scripting/automation ability (Python, shell/CLI) for pipeline management and validation.
- Strong troubleshooting, monitoring, and operational dashboard skills for pipeline and SIEM health.
- Ability to design and document secure log routing, retention, and load balancing to meet audit, legal, and privacy requirements.
- Hands-on experience designing and operating clustered/HA Cribl and Splunk deployments (worker groups, clustered indexers, resilient forwarders).
- Splunk ES or Cribl certifications.
Starr Insurance Compensation & Benefits Highlights
The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Starr Insurance and has not been reviewed or approved by Starr Insurance.
-
Healthcare Strength — Health coverage is frequently characterized as excellent, with comprehensive medical, dental, and vision options supported by wellness tools and an EAP. Feedback suggests additional voluntary protections (critical illness, accident, hospital indemnity) broaden overall healthcare security.
-
Retirement Support — Retirement offerings include both a 401(k) and a defined‑benefit pension, which is often highlighted as a standout feature. Profit sharing is also referenced as part of the broader financial benefits mix.
-
Strong & Reliable Incentives — Top performers in underwriting are described as receiving excellent pay with good bonuses and clear advancement opportunities.
Starr Insurance Insights
What We Do
Starr Insurance is a leading insurance and investment organization, providing commercial property and casualty insurance, including travel and accident coverage, to almost every imaginable business and industry in virtually every part of the world. Cornelius Vander Starr established his first insurance company in Shanghai, China in 1919. Today, we are one of the world’s fastest growing insurance organizations, capable of writing in 128 countries on 6 continents. Starr has the global knowledge, vision and relationships to help your company succeed, anywhere you do business.
