At Arctic Wolf, we're redefining the cybersecurity landscape. With our employee Pack members spread across the globe, we’re committed to setting new industry standards. Our accomplishments speak for themselves — from recognition in the Forbes Cloud 100, CNBC Disruptor 50, Fortune Future 50, and Fortune Cyber 60, to winning the 2024 CRN Products of the Year award. We’re proud to be named a Leader in the IDC MarketScape for Worldwide Managed Detection and Response Services and to have earned a Customers' Choice distinction from Gartner Peer Insights. Join a company that’s not only leading, but also shaping the future of security operations and customer engagement.
Our mission is simple: End Cyber Risk.
About the Role
The AI Security Assistant is an interactive security assistant that leverages generative AI (GenAI) and large language models (LLMs) to provide added context and assistance within the Arctic Wolf Unified Portal. It allows for natural language interaction, enabling customers to ask questions and gain more context about their security environment.
The Arctic Wolf AI Security Assistant represents a significant advancement in security operations, offering a powerful tool that combines the strengths of generative AI with the specific needs of security professionals. As it continues to evolve through its beta phase, incorporating feedback from users and expanding its skill set, it is poised to become an indispensable asset for Arctic Wolf customers seeking to bolster their security capabilities.
We are building an AI-driven product that redefines how cybersecurity assessments and insights are delivered. As a Senior Quality Engineer, you’ll play a pivotal role in ensuring the reliability, accuracy, and performance of our LLM-powered platform. You’ll collaborate closely with product, data, and developers teams to establish testing strategies, leverage GenAI tools for smarter quality assurance, and uphold the highest product quality standards.
What You Will DoDesign, develop, execute, and automate comprehensive test plans, cases, and scripts for AI-driven and cloud-native products.
Collaborate with Product Managers and Developers to validate system functionality, usability, and performance.
Define and track quality metrics to ensure continuous improvement and timely delivery.
Own the end-to-end quality lifecycle — from test strategy and automation to defect triage and release sign-off.
Leverage GenAI tools (e.g., ChatGPT, Claude, Copilot) to accelerate test design, automate testing tasks, and analyze defects efficiently.
Use Langfuse for LLM tracing, output validation, and LLM-as-a-Judge scoring to assess accuracy, relevance, and consistency.
Conduct API, integration, and system testing for distributed, cloud-based environments.
Build and maintain automation frameworks using Python, Playwright, Selenium, and BDD methodologies.
Identify quality risks, define mitigation strategies, and maintain measurable quality checkpoints.
Champion a culture of accountability, precision, and quality-first mindset within the team.
Must have exposure to LLM evaluation techniques, including output scoring, benchmarking, and validation frameworks.
Understanding of prompt engineering, Retrieval-Augmented Generation (RAG), model orchestration, and hallucination detection.
Experience testing accuracy, relevance, and consistency of AI model outputs and generated responses.
Ability to define and validate performance metrics for AI-driven services.
Awareness of AI safety, bias detection, and explainability methods to ensure fair and interpretable outcomes.
Familiarity with AI governance frameworks such as NIST AI RMF and EU AI Act, ensuring responsible and compliant testing practices.
Strong belief in ethical AI, transparency, and maintaining end-user trust.
Own the end-to-end quality lifecycle — from test strategy and automation to defect triage and release sign-off.
Leverage GenAI tools (e.g., ChatGPT, Claude, Copilot) to accelerate test design, automate testing tasks, and analyze defects efficiently.
Conduct API, integration, and system testing for distributed, cloud-based environments.
Build and maintain automation frameworks using Python, Selenium, and BDD methodologies.
Identify quality risks, define mitigation strategies, and maintain measurable quality checkpoints.
Champion a culture of accountability, precision, and quality-first mindset within the team.
Bachelor’s degree in Computer Science, Engineering, or related discipline.
5–8 years of QA automation and testing experience across complex applications.
Proficiency in Python, Selenium, and BDD frameworks.
Strong experience in API testing, cloud-based distributed systems, and test management systems.
Familiarity with GenAI tools to enhance testing productivity and efficiency.
Understanding of LLM evaluation techniques, prompt testing, and hallucination detection.
Analytical mindset with strong debugging and problem-solving skills.
Excellent communication and collaboration abilities.
Self-starter who thrives in a fast-paced, innovative environment.
Why Arctic Wolf
At Arctic Wolf, success comes from delighting our customers. We work together to ensure that happens every day. We believe in diversity and inclusion, and we value the unique perspectives every employee brings. By protecting people’s and organizations’ sensitive data — and now enhancing how they interact with AI securely — we’re advancing a mission that serves the greater good.
We celebrate unique perspectives through our Pack Unity program, encourage alliances across teams, and believe in giving back through our Pledge 1% Movement — dedicating our time, equity, and product for community impact.
What We Offer
Equity for all employees
Flexible annual leave, paid holidays, and volunteer days
Training and career development programs
Comprehensive private benefits plan, including medical coverage for you and your family
Life insurance (3x compensation) and personal accident insurance
Fertility support and paid parental leave
On-Camera Policy
To support a fair, transparent, and engaging interview experience, candidates interviewing remotely are expected to be on camera during all video interviews.
Being on camera fosters authentic connection, improves communication, and allows for full engagement from both candidates and interviewers.
We understand that technical, bandwidth, or location-related challenges may occasionally prevent video use. If this applies, candidates are required to notify us in advance so we can explore appropriate accommodations.
Security Requirements
Conducts duties and responsibilities in accordance with AWN’s Information Security policies, standards, processes, and controls to protect the confidentiality, integrity, and availability of AWN business information (in accordance with our employee handbook and corporate policies).
Background checks are required for this position.
This position may require access to information protected under U.S. export control laws and regulations, including the Export Administration Regulations (“EAR”). Please note that, if applicable, an offer for employment will be conditioned on authorization to receive software or technology controlled under these laws and regulations
Top Skills
What We Do
The cybersecurity industry has an effectiveness problem. Every year new technologies, vendors, and solutions emerge, and yet despite this constant innovation we continue to see high profile breaches in the headlines. All organizations know they need better security, but the dizzying array of options leave resource-constrained IT and security leaders wondering how to proceed. At Arctic Wolf, our mission is to End Cyber Risk through effective security operations. To achieve this, we believe that organizations must do three key things:
