QA Engineer (AI Voice)

Posted Yesterday
Be an Early Applicant
Hiring Remotely in Poland
Remote
Mid level
Information Technology • Software • Business Intelligence
The Role
Execute end-to-end, API, integration, regression, and release testing for AI voice journeys across markets. Prepare test data, support UAT, extend automated suites, evaluate conversational AI behavior, document defects, and track them to resolution. The role requires software QA experience, voice or conversational AI testing, API testing, basic Python or JavaScript/TypeScript automation, and troubleshooting through logs and transcripts.
Summary Generated by Built In
Growth through diversity, equity, and inclusion. As an ethical business, we do what is right — including ensuring equal opportunities and fostering a safe, respectful workplace for each of us. We believe diversity fuels both personal and business growth. We're committed to building an inclusive community where all our people thrive regardless of their backgrounds, identities, or other personal characteristics.

What you'll be doing

  • Run E2E, API, integration, regression and release tests on voice journeys across markets and configurations.
  • Prepare test data and expected outcomes, and support business testers during UAT.
  • Run and extend the automated suites and evaluation sets designed by the Senior.
  • Log defects with clear reproduction steps and transcripts, and follow them to closure.
  • Flag risks and blockers early.

What we're looking for

    • At least 3 years of experience in software QA.
    • At least 1 year of experience in testing voice or conversational AI. 
    • Hands-on voice agent testing: speech recognition and speech output basics, intents, latency, reading transcripts.
    • Understanding why the same input can produce different AI outputs, and how to test for it.
    • API testing and basic automation in Python or JavaScript/TypeScript.
    • Jira or Azure DevOps and a test management tool.Troubleshooting from logs and transcripts.
    • Clear written and verbal communication.
    • Voice testing platforms (Hamming, Cekura, Coval, Botium) is a plus.
    • Experience with LLM evaluation tools (Promptfoo, DeepEval, LangSmith) is a plus.
    • Experience with Telephony basics (SIP, Twilio, Azure Communication Services) is a plus.
    • ISTQB certification is a plus.
    • Experience in Responsible-AI testing (PII, bias, safety) is a plus.
    • Experience validating customer journeys across multiple markets is a plus.
    • English at least B2.

We offer

    • Stable employment. On the market since 2008, 1500+ talents currently on board in 7 global sites.
    • “Office as an option” model. You can choose to work remotely or in the office.  
    • Workation. Enjoy working from inspiring locations in line with our workation policy. 
    • Great Place to Work® certified employer.
    • Flexibility regarding working hours and your preferred form of contract.  
    • Comprehensive online onboarding program with a “Buddy” from day 1.   
    • Cooperation with top-tier engineers and experts.  
    • Unlimited access to the Udemy learning platform from day 1.
    • Certificate training programs. Lingarians earn 500+ technology certificates yearly. 
    • Upskilling support. Capability development programs, Competency Centers, knowledge sharing sessions, community webinars, 110+ training opportunities yearly.
    • Grow as we grow as a company. 76% of our managers are internal promotions.  
    • A diverse, inclusive, and values-driven community.   
    • Autonomy to choose the way you work. We trust your ideas.  
    • Create our community together. Refer your friends to receive bonuses.  
    • Activities to support your well-being and health.
    • Plenty of opportunities to donate to charities and support the environment. 
    • Modern office equipment. Purchased for you or available to borrow, depending on your location.

Skills Required

  • At least 3 years of software QA experience
  • At least 1 year of voice or conversational AI testing experience
  • Hands-on testing of voice agents, including speech recognition, speech output, intents, latency, and transcripts
  • Understanding of variability in AI outputs and methods for testing it
  • API testing experience
  • Basic automation skills in Python or JavaScript/TypeScript
  • Experience with Jira or Azure DevOps and a test management tool
  • Ability to troubleshoot using logs and transcripts
  • Clear written and verbal communication
  • English proficiency of at least B2
  • Experience with voice testing platforms such as Hamming, Cekura, Coval, or Botium
  • Experience with LLM evaluation tools such as Promptfoo, DeepEval, or LangSmith
  • Knowledge of telephony basics, including SIP, Twilio, or Azure Communication Services
  • ISTQB certification
  • Experience in responsible AI testing, including PII, bias, and safety
  • Experience validating customer journeys across multiple markets
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Warsaw
1,340 Employees
Year Founded: 2008

What We Do

You’ve got the data — now what’s next? Many businesses are overwhelmed by data and struggle to turn it into real business impact. At Lingaro, we empower global brands and companies to achieve more with data. From strategy development to scalable solutions, we guide you every step of the way. We transform raw data into actionable insights with deep tech expertise, sharp business sense, and a user-centric mindset. Rooted in Europe and operating worldwide, we deliver with agility, a fresh perspective, and a proven track record. Join 75+ leading CPG companies across 30+ countries that have already transformed their business with Lingaro. Be data ready. Be business ready. Be future ready. Contact us at https://lingarogroup.com/contact_us to get started. Want to know more? Check our recognitions & awards below. 🏆 Positioned as a Leader in ISG Provider Lens™ 2025 – Generative AI Services in Strategy & Consulting and Development & Deployment 🏆 Identified as a Leader in ISG Provider Lens™ 2025 – Specialty Analytics Services in Supply Chain and Retail & CPG 🏆 Strong Performer in the Gartner® Peer Insights™ 2022 and 2024 "Voice of the Customer" reports 🏆 Major Contender in the Everest Group® 2024 Analytics and AI Services Specialists PEAK Matrix® Assessment 🏆 Listee in the Gartner® 2024 Guide to Service Providers for GenAI Initiatives 🏆 Great Place to Work® in Poland, the Philippines, and India in 2024

Similar Jobs

Lingaro Logo Lingaro

Quality Assurance Engineer

Information Technology • Software • Business Intelligence
Remote
Poland
1340 Employees

Capco Logo Capco

Product Manager

Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Remote or Hybrid
Poland
6000 Employees

RapDev Logo RapDev

Senior Cloud Engineer

Information Technology • Productivity • Professional Services • Software
Remote or Hybrid
27 Locations
130 Employees
75K-95K Annually

Capco Logo Capco

Project Manager

Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Remote or Hybrid
Poland
6000 Employees

Similar Companies Hiring

Revel Thumbnail
Aerospace • Hardware • Robotics • Software
Marina Del Rey, California
70 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
33 Employees
RapidScale Thumbnail
Cloud • Information Technology • Consulting
Raleigh, NC
600 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account