Kafka Administrator

Reposted 11 Days Ago
Be an Early Applicant
Floors, Oakview, MB, CAN
In-Office
Mid level
Insurance
The Role
Administer, operate, and support Confluent/Apache Kafka clusters (ZooKeeper or KRaft). Provision, upgrade, scale, patch, monitor, and troubleshoot brokers, replication, and consumer issues. Manage Kafka components, implement security controls, automate operations, perform RCA for incidents, and participate in on-call rotations while partnering with application teams on topic design and performance tuning.
Summary Generated by Built In

At Allstate, great things happen when our people work together to protect families and their belongings from life’s uncertainties. And for more than 90 years, our innovative drive has kept us a step ahead of our customers’ evolving needs. From advocating for seat belts, air bags and graduated driving laws, to being an industry leader in pricing sophistication, telematics, and, more recently, device and identity protection. 

Job Description

The Vended Application Consultant II successfully delivers product features utilizing vended software, particularly ERP applications, by applying knowledge of the application’s functionality and client business processes. The role contributes to the creation of technical designs for their area and configures vendor software under supervision for new or existing enterprise-wide and business-unit applications.

Key Responsibilities
  • Administer, operate, and support Confluent Kafka clusters (ZooKeeper and/or KRaft)
  • Perform Kafka cluster provisioning, upgrades, scaling, and patching
  • Monitor cluster health, performance, and capacity using JMX, metrics, and observability tools
  • Troubleshoot and debug Kafka issues including:
    • Broker failures
    • ISR shrink/expand issues
    • Leader elections
    • Replication lag
    • Consumer lag and performance problems
  • Manage Kafka components:
    • Kafka Brokers
    • Kafka Connect
    • Schema Registry
    • REST Proxy / ksqlDB (if applicable)
  • Analyze logs and metrics to perform root cause analysis (RCA) for production incidents
  • Work with application teams to support topic design, partitioning, retention, and performance tuning
  • Ensure high availability, reliability, and data durability of Kafka clusters
  • Implement and maintain security controls (ACLs, TLS, authentication, authorization)
  • Automate Kafka operations using scripts and configuration management tools
  • Participate in on-call support and incident response rotations
Required Skills & Qualifications
  • 3+ years of hands-on experience as a Kafka / Confluent Platform Administrator
  • Strong understanding of Kafka architecture and internals:
    • Brokers, partitions, replicas, ISR
    • Leader election and replication
    • ZooKeeper and/or KRaft mode
  • Excellent debugging and troubleshooting skills
  • Strong Linux/Unix experience (processes, networking, file systems, performance tuning)
  • Experience with Kafka monitoring and metrics (JMX, Prometheus, Datadog, Grafana, etc.)
  • Ability to read and analyze Kafka broker logs and JVM metrics
  • Experience working in production environments with high data volume and throughput
  • Good scripting skills (Bash, Shell; Python is a plus)
Nice-to-Have / Add-On Skills
  • Experience with AWS services (EC2, EBS, S3, IAM, VPC, MSK or self-managed Kafka on AWS)
  • Infrastructure as Code experience using Terraform
  • Configuration management using Ansible
  • Experience with CI/CD pipelines
  • Knowledge of Kafka security best practices
  • Experience with disaster recovery and multi-cluster Kafka setups

Primary Skills

Apache Kafka, Defect Resolution, DevOps, ERP Applications, Functional Designs, Integration Testing, Issue Management, Red Hat Ansible, Terraform

Shift Time

Recruiter Info

Yateesh B G

[email protected]

About Allstate

Joining our team isn’t just a job — it’s an opportunity. One that takes your skills and pushes them to the next level. One that encourages you to challenge the status quo. One where you can shape the future of protection while supporting causes that mean the most to you. Joining our team means being part of something bigger – a winning team making a meaningful impact.

The Allstate Corporation is one of the largest publicly held insurance providers in the United States. Ranked No. 84 in the 2023 Fortune 500 list of the largest United States corporations by total revenue, The Allstate Corporation owns and operates 18 companies in the United States, Canada, Northern Ireland, and India. Allstate India Private Limited, also known as Allstate India, is a subsidiary of The Allstate Corporation. The India talent center was set up in 2012 and operates under the corporation's Good Hands promise. As it innovates operations and technology, Allstate India has evolved beyond its technology functions to be the critical strategic business services arm of the corporation. With offices in Bengaluru and Pune, the company offers expertise to the parent organization’s business areas including technology and innovation, accounting and imaging services, policy administration, transformation solution design and support services, transformation of property liability service design, global operations and integration, and training and transition.

Learn more about Allstate India here.

Skills Required

  • 3+ years of hands-on experience as a Kafka / Confluent Platform Administrator
  • Strong understanding of Kafka architecture and internals (brokers, partitions, replicas, ISR)
  • Knowledge of leader election and replication, ZooKeeper and/or KRaft mode
  • Excellent debugging and troubleshooting skills for Kafka production issues
  • Strong Linux/Unix experience (processes, networking, file systems, performance tuning)
  • Experience with Kafka monitoring and metrics (JMX, Prometheus, Datadog, Grafana, etc.)
  • Ability to read and analyze Kafka broker logs and JVM metrics
  • Experience working in production environments with high data volume and throughput
  • Good scripting skills (Bash, Shell)
  • Python scripting (listed as a plus)
  • Implement and maintain security controls (ACLs, TLS, authentication, authorization)
  • Automate Kafka operations using scripts and configuration management tools
  • Experience with AWS services (EC2, EBS, S3, IAM, VPC, MSK)
  • Infrastructure as Code experience using Terraform
  • Configuration management experience using Ansible
  • Experience with CI/CD pipelines
  • Experience with disaster recovery and multi-cluster Kafka setups

Allstate Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Allstate and has not been reviewed or approved by Allstate.

  • Retirement Support Retirement offerings include a 401(k) with company match and a company-funded pension, providing notable long-term value. These features are described as standout components of the package.
  • Leave & Time Off Breadth Paid time off is described as generous, with vacation and parental leave available. This breadth of leave supports work-life needs for a range of situations.
  • Flexible Benefits Flexible work arrangements include options for in-office, remote, or hybrid work. This flexibility is positioned as a meaningful part of the total rewards experience.

Allstate Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Northbrook, IL
57,671 Employees
Year Founded: 1931

What We Do

At Allstate, we're advocates for peace of mind and a good life. And that comes through in everything we do. From building diverse and innovative teams that truly understand our customers' needs, to challenging each other to develop our careers in a meaningful way, and finally to the incredible results we're able to achieve together. See how we’re creating a better future through innovation, advocacy, and empowering people and communities

Similar Jobs

Coursera + Udemy  Logo Coursera + Udemy

Program Manager

Artificial Intelligence • Consumer Web • Edtech • Enterprise Web • HR Tech • Social Impact • Generative AI
Remote or Hybrid
Canada
1500 Employees
78K-98K Annually

Cash App Logo Cash App

Program Manager

Blockchain • Fintech • Mobile • Payments • Software • Financial Services
Remote or Hybrid
8 Locations
3500 Employees
123K-223K Annually

Cash App Logo Cash App

Regulatory Examination Manager

Blockchain • Fintech • Mobile • Payments • Software • Financial Services
Remote or Hybrid
8 Locations
3500 Employees
136K-245K Annually

Samsara Logo Samsara

Lead Machine Learning Engineer

Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Easy Apply
Remote or Hybrid
Canada
4000 Employees
196K-270K Annually

Similar Companies Hiring

Globe Life Thumbnail
Insurance • Financial Services
McKinney, TX
3000 Employees
MassMutual India Thumbnail
Big Data • Fintech • Information Technology • Insurance • Financial Services
Hyderabad, Telangana
Granted Thumbnail
Artificial Intelligence • Healthtech • Insurance • Mobile • Financial Services
New York, New York
23 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account