The Role
Leads QA strategy and builds automation frameworks for AI-powered applications. Owns functional, non-functional, API, integration, acceptance, system, regression, and end-to-end testing across distributed systems. Tests LLM behaviors including hallucinations, factual accuracy, prompt consistency, and RAG flows. Mentors junior QA engineers, collaborates with developers and data teams, and integrates testing into CI/CD pipelines.
Summary Generated by Built In
Job Title: SDET (Software Development Engineer in Testing) – Senior Automation QA & Manual Testing
Experience: 8+ Years
Employment Type: Full-Time
Dept: FSD
Designation: Senior Specialist – FSD QA
Location: Hybrid
About Us
At InxiteOut Pvt. Ltd., we specialize in building next-generation AI-powered applications that solve real-world business problems. We are expanding our QA team to ensure our AI systems are robust, secure, and aligned with user expectations.
Key Responsibilities
· Strong knowledge of Python.
· Configure and test AI/ML product/system.
· Lead the quality assurance strategy and establish automation frameworks from scratch.
· Manage and mentor junior QA engineers and guide them in test automation and AI system validation.
· Own functional and non-functional testing across complex distributed systems.
· Own API, integration, acceptance, system, and end-to-end (E2E) testing.
· Lead test planning, test case design, and execution for web, API, and AI-based systems.
· Design and maintain regression test suites and ensure full coverage.
· Collaborate with Full Stack Developers, Data Scientists, and AI Engineers to define test strategies for LLM-based features.
· Own testing of AI-specific behaviors including:
o Hallucination and factual accuracy
o Consistency in prompt-response pairs
o Retrieval-Augmented Generation (RAG) flows
· Collaborate with developers, product owners, and other QA engineers in Agile teams.
Must-Have Skills
· Deep experience in Functional Testing:
o Advanced REST API Testing (Postman, REST Assured, etc.)
o Integration, Acceptance, E2E, Smoke, Sanity, System Testing
· Strong foundation in Non-Functional Testing:
o Performance and Load Testing
o Stress and Security Testing
· Proficiency in Testing Methods:
o Black-box, White-box, Grey-box Testing
· Regression Testing lifecycle ownership
· Expertise in automation frameworks (e.g., Selenium, Playwright, Cypress, TestNG)
· Strong scripting/coding skills in Python (preferable) / Java / JavaScript
· Familiarity with containerized environments (Docker, Kubernetes)
Good-to-Have (AI Testing Focus)
· Exposure to testing AI/ML products/systems.
· Experience of integrating tests into CI/CD pipelines (GitHub Actions, GitLab, Azure DevOps etc.).
· Hands-on experience with AI testing tools or custom frameworks.
· Familiarity with evaluating LLM (large language model) outputs.
· Experience of validating RAG-based architecture.
· Understanding of hallucination detection metrics and prompt tuning.
· Exposure to safety audits for AI applications.
· Collaboration experience with Full Stack, Data Science, Data Engineering & DevOps teams.
Qualifications
· Bachelor’s or master’s in computer science, Engineering, or related field.
· Certifications in testing or security are a plus (e.g., ISTQB, OWASP).
Skills Required
- 8+ years of experience in software development engineering in test, quality assurance, automation, and manual testing
- Strong knowledge of Python
- Experience configuring and testing AI/ML products or systems
- Experience leading QA strategy and establishing automation frameworks
- Experience managing or mentoring junior QA engineers
- Advanced REST API testing using tools such as Postman or REST Assured
- Experience with integration, acceptance, end-to-end, smoke, sanity, and system testing
- Performance, load, stress, and security testing experience
- Knowledge of black-box, white-box, and grey-box testing
- Regression testing lifecycle ownership
- Expertise with automation frameworks such as Selenium, Playwright, Cypress, or TestNG
- Strong scripting or coding skills in Python, Java, or JavaScript
- Familiarity with Docker and Kubernetes
- Bachelor's or master's degree in computer science, engineering, or a related field
- Exposure to testing AI/ML products or systems
- Experience integrating tests into CI/CD pipelines using GitHub Actions, GitLab, or Azure DevOps
- Hands-on experience with AI testing tools or custom frameworks
- Experience evaluating large language model outputs
- Experience validating RAG-based architectures
- Understanding of hallucination detection metrics and prompt tuning
- Exposure to safety audits for AI applications
- ISTQB, OWASP, or other testing or security certifications
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
InXiteOut is a strategic AI and data analytics partner for enterprises. It advises, architects, builds, and deploys production-grade AI solutions, data platforms, analytics systems, custom applications, and automation. The company helps organizations identify high-value use cases, turn complex data into actionable intelligence, and embed AI into daily workflows, enabling faster decisions, smarter operations, measurable business impact, and scalable growth across industries.
