49 Solutions is seeking an experienced AI Tester / Architect Consultant to join our client at IRCC for innovative projects within the Canadian Federal Government. You will play a critical role in ensuring the functionality, performance, and reliability of AI applications, combining technical expertise with a deep understanding of AI technologies to contribute effectively across multiple initiatives.
Requirements
What You’ll Do:
- Design, develop, and execute test strategies for AI applications, including AI agents, RAG, and LLMs. Create test plans and cases, conduct functional, performance, and integration testing, identify potential model biases and edge cases, and collaborate with developers and data scientists to improve AI reliability and accuracy.
- Leverage AI tools to test applications on a regular basis, using prompts and outputs to generate and maintain test cases that evolve as AI models are updated or modified.
- Evaluate and validate machine learning models, algorithms, and AI frameworks such as TensorFlow, Keras, and PyTorch to ensure optimal performance and accuracy.
- Implement test automation for AI systems using Python, Java, or R to streamline testing processes and ensure repeatable results.
- Collaborate with cross-functional teams to identify, analyze, and resolve performance or functionality issues, ensuring AI applications meet project requirements.
- Support DevOps processes and CI/CD pipelines in AI application deployment, enabling seamless integration and continuous delivery.
- Ensure compliance with best practices and security requirements for federal government projects, maintaining high standards of quality and reliability.
What We’re Looking For:
- Minimum 5 years of experience in software testing or test automation, specifically with AI applications.
- Strong proficiency with AI frameworks, testing tools, and AI technologies.
- Experience with AI agents, RAG, LLMs, and machine learning models.
- Strong programming skills in Python, Java, or R.
- Familiarity with DevOps processes and CI/CD pipelines.
- Excellent analytical, problem-solving, and communication skills.
- A degree in Computer Science, Engineering, or a related field.
- A valid Canadian Government Security Clearance is a non-negotiable
Benefits
Remote/Hybrid Work
Similar Jobs
What We Do
At 49 Solutions we consistently deliver top quality consulting services, supporting both business and technology. We work alongside our clients to assess their needs and ensure our approach, recommendations and move-forward plan will deliver the results they expect and the value they deserve.
Our clients have mandates; our expert consultants help them develop comprehensive strategic plans to achieve their goals. We understand the machinery of government and how to articulate plans to meet the needs of central agencies, senior managers and program administrators. We can facilitate direction setting, research and analyze the issues, outline the options and define resource requirements. 49 Solutions will bring together project management, communications, human resource, IT and finance professionals to help you do what you set out to do.
The core of our business foundation is trust. When you work with 49 Solutions, you can count on us to bring the right skills and capabilities to the table, at the right place and at the right time, in order to achieve a successful outcome. The majority of our clients are repeat customers who time and time again reach out to 49 Solutions as their first call when a new project, deliverable, or issue lands on their desk. We are well versed in both business and technology and often become the bridge between the two, creating cohesive projects that achieve desired outcomes.
