The Role
Quality Assurance Engineer responsible for developing QA strategies, creating manual and automated tests, and validating web, API, AI, and LLM-powered features. The role evaluates model accuracy, fairness, hallucinations, bias, and edge cases; investigates defects; tracks quality metrics; and drives continuous improvement. The engineer collaborates with product and development teams in an Agile/Scrum environment and presents quality reports to management.
Summary Generated by Built In
About Us
Gurully Technologies is a leading EdTech company with a seven-year legacy of excellence. Specializing in the development of in-house educational software, Gurully is at the forefront of transforming the preparation landscape for exams such as PTE, IELTS, DUOLINGO, and CELPIP. Our commitment to innovation and educational excellence has positioned us as pioneers in providing top-notch services to institutes and students across more than 40 countries.
Job Description:
As a Quality Assurance Engineer at Gurully, you will play a crucial role in ensuring the reliability, performance, and security of our AI-driven learning platforms used by students and institutes worldwide. The ideal candidate will have a minimum of 3 years of hands-on experience in QA, with a proven track record of building test strategies, automating tests, and validating AI-powered features. The role involves working closely with product and development teams to identify issues early, drive continuous improvement, and deliver a seamless experience to our users.
Responsibilities:
- Review and analyze requirements, specifications, and technical design documents, providing timely feedback.
- Plan, create, and manage the overall quality strategy, including detailed test plans and test cases.
- Design, develop, and execute automated tests, along with manual, UI, and regression testing.
- Test AI-powered features such as automated scoring and feedback for speaking and writing tasks, validating accuracy, consistency, and fairness against expected benchmarks.
- Evaluate AI/LLM outputs for hallucinations, bias, edge cases, and response quality, and build reusable test datasets for ongoing validation.
- Leverage AI-assisted tools to accelerate test case generation, test automation, and defect analysis.
- Test current products, identify and report defects, and verify that issues are resolved.
- Investigate the root causes of quality issues and suggest practical solutions to improve customer satisfaction.
- Define and track key quality KPIs such as defect density, open defect counts, and AI scoring accuracy.
- Identify bottlenecks in the QA process and drive continuous improvement initiatives.
- Collaborate with the product and development teams to ensure consistent, on-time project delivery.
- Stay updated with new testing tools, AI testing techniques, and industry best practices.
- Prepare and present quality reports and metrics to senior management.
Requirements:
- Bachelor's or Master's degree in Computer Science or a related field.
- Minimum of 3 years of proven experience as a Quality Assurance Engineer.
- Hands-on experience with white box and black box testing, UI testing, and automated testing.
- Proficient in automated testing tools and frameworks (e.g., Selenium, Cypress, Playwright).
- Experience testing AI/ML-based applications, including validating model outputs, scoring accuracy, and consistency.
- Understanding of LLM concepts such as prompts, hallucinations, bias, and non-deterministic outputs, and how to design test strategies around them.
- Familiarity with AI-powered testing tools (e.g., Testim, Mabl, Applitools, GitHub Copilot) for smarter and faster testing.
- Experience with API testing (e.g., Postman, REST Assured), including testing third-party AI APIs.
- Coding experience in languages such as Java, PHP, Python, or JavaScript, along with HTML5 and XML.
- Solid knowledge of SQL and scripting.
- Experience writing clear, concise, and comprehensive test plans and test cases.
- Familiarity with web security technologies and QA tools and processes.
- Experience working in an Agile/Scrum development environment.
- Excellent analytical, problem-solving, time-management, and communication skills.
- Experience with performance and/or security testing is a plus.
Skills Required
- Bachelor's or Master's degree in Computer Science or a related field
- At least 3 years of experience as a Quality Assurance Engineer
- Hands-on experience with white box, black box, UI, and automated testing
- Proficiency with Selenium, Cypress, Playwright, or similar automated testing frameworks
- Experience testing AI/ML applications, including model outputs, scoring accuracy, and consistency
- Understanding of LLM prompts, hallucinations, bias, and non-deterministic outputs
- Familiarity with AI-powered testing tools such as Testim, Mabl, Applitools, or GitHub Copilot
- Experience with API testing using tools such as Postman or REST Assured, including third-party AI APIs
- Coding experience with Java, PHP, Python, or JavaScript, plus HTML5 and XML
- Solid knowledge of SQL and scripting
- Experience writing comprehensive test plans and test cases
- Familiarity with web security technologies and QA tools and processes
- Experience working in an Agile/Scrum development environment
- Excellent analytical, problem-solving, time-management, and communication skills
- Experience with performance and/or security testing
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Gurully is an education-technology platform focused on English-language proficiency test preparation. It provides AI-powered practice and mock tests, instant scoring, detailed feedback, performance analytics, and preparation resources for PTE, IELTS, Duolingo English Test, and CELPIP. The platform serves students and institutes, helping learners improve exam performance and pursue study-abroad goals through accessible, technology-enabled language learning with personalized insights and structured practice workflows.
