What You Will Do
- Develop and implement QA strategies for AI-powered applications, focusing on accuracy, bias, fairness, robustness, and performance.
- Design and execute automated and manual test cases to validate AI Agents/LLM models, APIs, and data pipelines and good understanding of data integrity, data models, etc
- Assess AI models using quality metrics such as precision/recall and hallucination detection.
- Test AI models for bias, fairness, explainability (XAI), drift, and adversarial robustness.
- Validate prompt engineering, fine-tuning techniques, and model-generated responses for accuracy and ethical AI considerations.
- Service/tool development.
- Conduct scalability, latency, and performance testing for AI-driven applications.
- Collaborate with data engineers to validate data pipelines, feature engineering processes, and model outputs.
- Design, develop, and maintain automation scripts using Selenium and Playwright for API and web testing
- Work closely with cross-functional teams to integrate automation best practices into the development lifecycle.
- Identify, document, and track bugs while conducting detailed regression testing to ensure product quality.
What You Will Bring
- Proven expertise in testing AI models, LLMs, and Generative AI applications, with hands-on experience in AI evaluation metrics and testing tools like Arize, MAIHEM, and LangTest and Playwright MCP for automated testing workflows.
- Strong proficiency in Python for writing test scripts and automating model validation, along with a deep understanding of AI bias detection, adversarial testing, model explainability (XAI), and AI robustness.
- Demonstrate strong SQL expertise for validating data integrity and backend processes, particularly in PostgreSQL and MySQL.
- Strong analytical and problem-solving skills with keen attention to detail, along with excellent communication and documentation abilities to convey complex testing processes and results.
Why You Will Love It Here
- Next-Level QA – Go beyond traditional testing to challenge AI agents, LLMs, and GenAI systems with intelligent, self-evolving test strategies
- Agentic AI Frontier – Be at the forefront of validating autonomous, ethical AI in high-impact applications trusted by global enterprises
- Full-Stack Test Engineering – Combine Python, SQL, and tools like LangTest, Arize, Selenium & Playwright to test everything from APIs to AI fairness
- Purpose-Driven Mission – Join a remote-first team that protects critical supply chains — ensuring vital products reach people when they need them most
Top Skills
What We Do
Every year, tens of thousands of events – ranging from natural disasters to factory fires to health epidemics – shut down manufacturing and wreak havoc on global supply chains. We believe a resilient supply chain is good for everyone: it keeps product flowing, the world moving, and most importantly, people in their jobs.
Resilinc was founded with the purpose of strengthening global supply chains, making them resilient, sustainable, transparent, and secure. We do this via our technology-driven solutions, which create an ecosystem where organizations can collaborate with their suppliers and customers with a spirit of transparency and trust to acquire unmatched visibility into their multi-tier supply networks, and partner across tiers seamlessly to recover supply chains during disruptions.
Since our launch in 2010, Resilinc has defined the supply chain mapping, monitoring, and resiliency space and is widely considered the gold standard for supply chain resiliency, worldwide. With over 1 million supplier sites mapped encompassing over 4 million parts and raw materials, we are the first line of defense for our customers, helping them navigate supply disruptions.
Our early-warning alert system monitors and predicts potential disruptions across suppliers, sites, and materials; our platform enables them to collaborate closely with their suppliers; our historical data-backed insights give them options on appropriate actions to take. Always innovating, our AI-powered predictive solutions can predict delivery delays, price movements, and supply constraints for raw materials and commodities before they happen. Resilinc helps our customers protect revenue and turn supply chain risks into opportunities to gain a competitive advantage
