Sr Infrastructure / DevOps Engineer

A production engineer is a developer who thinks deeply about systems and how they behave in the wild. Whether it be networking, or the Linux kernel, or even a specific interest in scaling, algorithms, or distributed systems. You are a systems engineer who aims to code themselves out of a job by automating all the things and leverages great development practices like Test-Driven-Development or continuous integration (to start with).

Like all engineers at Benzinga, we expect you to be comfortable operating within our application and service environments. While your primary focus will be on defining, building, and maintaining our robust and scalable infrastructure, you will collaborate closely with development teams to ensure seamless integration and deployment. While there’s plenty of work for you to do upfront on our infrastructure, your true effect at Benzinga is the creation of a reliable, high-performing platform that supports the amazing products our users come to know and love.

• Radiate knowledge about the service's infrastructure and reliability to the rest of the development team.
• Identify parts of the system that do not scale, provide immediate palliative measures and drive long term resolution of these incidents.
• Plan the growth of Benzinga's infrastructure.

• Document every action so your learnings turn into repeatable actions and then into automation.
• Improve the deployment process to make it as boring as possible.
• Define, provision, and manage our production infrastructure using Kubernetes and Terraform.

• Proactively identify and reduce security risks
• Develop security training and guidance to internal development teams
• Ability to discover and patch SQLi, XSS, CSRF, SSRF, authentication and authorization flaws, and other web-based security vulnerabilities
• Knowledge of common authentication technologies including OAuth, SAML, CAs, OTP/TOTP
• Adhere to SOC2 compliance standards and assist with ongoing auditing and reporting.

• Design, build and maintain core infrastructure pieces that allow Benzinga to scale to support thousands of concurrent users.
• Be on an on-call rotation to respond to benzinga.com availability incidents and provide support for service engineers with customer incidents.
• Debug production issues across services and levels of the stack.

• Make monitoring and alerting alert on symptoms and not on outages.
• Manage day-to-day maintenance and evolution of Benzinga's Prometheus monitoring and alerting infrastructure
• Bundle Prometheus monitoring as an out-of-the-box monitoring solution for Benzinga products
• Build and maintain the benzinga.com public monitoring gateway
• Help migrate our current performance monitoring solution to Prometheus
• Improve coverage of Benzinga performance monitoring
• Create automated alerts to notify team members of regressions

• Experience with Kubernetes required
• Experience with some of these technologies a must: EKS, Terraform, GitLab CI/CD, OpenSearch/Elasticsearch, Postgres, MySQL, Kafka, BigQuery, Python, NodeJS, Go, Java, Prometheus, Grafana, Coralogix, Varnish, Fastly, Cloudfront, Nginx, Kong
• You can reason about software, algorithms, and performance from a high level.
• You have experience thinking about systems - edge cases, failure modes, behaviors, and specific implementations.
• You have worked with distributed systems and have a solid understanding of how modern web stacks are built, and why.
• You know your way around Linux and the Unix Shell.
• Strong communication skills
• Experience with managing large amounts of telemetry
• Experience developing time-series databases
• Self-motivated with strong organizational skills