Automation Tester for AI-Driven Applications

Job Purpose:

The tester will be responsible for validating the functionality, performance, and reliability of AI-driven systems and applications. This includes designing and executing test cases for automation flows, and intelligent features integrated into enterprise platforms.

Important: This role is not about using AI for testing. Instead, this is about testing applications where AI has been used. Testers are required to understand the probabilistic nature of AI driven systems and should be able to test applications for factual correctness, halucinations, guardrail breaches, and safety. Testers should have firm understanding of GenAI tools such as ChatGTP, Gemini, etc. with an understanding of the patterns on how they provide answers and where can they provide inconsistent and incorrect answers. Elementary courses in prompt engineering will be highly beneficial.

Mandatory skills:

Experience:

• At least 5 - 7 years in the field of test engineering.
  

AI Tools:

• Knowledge of GPT-4.0 or other LLMs, as well as expertise in prompt engineering, is highly beneficial.
  

• Familiarity with the Microsoft Responsible AI Toolbox, AI Foundry, and Guardrail.

Automation Tools:

• Significant expertise in automation tools such as Selenium, JMeter, LoadRunner, or Appium for automating web, mobile, and performance testing.
  

CI/CD Integration:

• A strong grasp on continuous integration and continuous deployment tools such as Jenkins, Bamboo, or GitLab CI/CD to automate build and deployment processes.
  

Quality Assurance:

• Thorough understanding of Quality Assurance principles, testing methodologies, and best practices. The ability to develop robust automation frameworks to ensure the highest standards of quality in our products.
  

Preferred Skills:

• Experience in the Fintech/BFSI domain
  
• Availability for remote or onsite work
  
• Proficiency in automation and framework
  
• Expertise in validating data correction and hallucination
  
• Expertise in AzureDevOps or Jira

Industry Standards: 

• Understanding of software development methodologies like Agile, Scrum, and DevOps practices.

Roles and Responsibilities:

• Role Focus: Testing and evaluating AI Driven Applications
  
• Skills Required: Proficiency in testing and evaluating technology
  
• Work Environment: Agile model, collaboration with Development and Integration teams
  
• Independence: Ability to work independently and bring innovative ideas
  
• Daily and weekly test reporting
  
• Defect tracking
  
• Attending daily stand-up and retrospective meetings
  
• Problem-solving and root cause analysis
  
• Automation scripts and test case accuracy
  
• Analytical Skills: Ability to analyze test results, identify trends, and derive meaningful insights
  
• Desired Traits: Detail-oriented, innovative, analytical.

Availability

• Available immediately