The Role
Develop and maintain automated test frameworks for GenAI applications, AI agents, MCP-based solutions, APIs, integrations, and distributed software. Create test plans and cases, execute manual and automated testing, analyze results, manage defects, and support release validation. Work in Linux environments using Python, Shell, Docker, CI/CD, and related tools. Collaborate with developers, architects, DevOps, and product teams to improve test coverage, reliability, quality metrics, and end-to-end software performance.
Summary Generated by Built In
Job Summary & Responsibilities Preferred Qualifications
If you want to directly shape the future, this is your opportunity. Advantest is one of the world’s leading providers of automatic test equipment for the semiconductor industry. We work closely with leading companies in emerging markets. Almost any modern AI, cloud, or embedded system is based on semiconductor devices that have been tested with our products.
We are looking for a Test Automation Engineer AI Solutions with strong experience in Linux, Python-based test automation, software testing, and testing of AI-based solutions. The role focuses on developing and maintaining automated test environments, defining and executing test strategies, and ensuring the quality and reliability of GenAI applications, MCP-based solutions, interfaces, integrations, and large-scale software applications.
Key Responsibilities
• Test automation: Design, develop, and maintain automated test frameworks and reusable test components for functional, interface, integration, regression, and system-level testing. Develop automated tests for GenAI applications, AI agents, and MCP-based solutions.
• Test planning and execution: Translate product requirements into test scenarios and test cases. Maintain test plans and strategies, execute manual and automated tests, analyze results, and track defects through resolution.
• AI, interface, and integration testing: Test GenAI-based functionality, MCP servers and clients, tool integrations, APIs, and interfaces between distributed software components. Validate functional behavior, error handling, reliability, and end-to-end workflows.
• Linux and test infrastructure: Work with Linux-based development and test environments, application configurations, deployment environments, and test infrastructure. Develop automation using Python and Shell scripting and support Docker-based workflows.
• Test results and development integration: Consolidate test results, generate test reports and quality metrics, and improve test coverage and release quality. Work closely with developers, architects, DevOps engineers, and product teams and support release validation.
'Required Qualifications
• Bachelor's degree in Computer Science, Software Engineering, Information Technology, or a related field, with 3+ years of experience in software testing and test automation.
• Strong experience with Linux, Python, Shell scripting, and automated test frameworks, including manual and automated testing methods.
• Good knowledge of interface, API, integration, end-to-end, and system testing, including testing across distributed software components.
• Experience with Docker, CI/CD and build tools such as Jenkins and Bazel, and source control systems such as Git and Bitbucket.
• Strong understanding of test strategies, test design, defect management, debugging, and troubleshooting, with experience working in Agile/Scrum development environments.
Preferred Skills
• Experience testing GenAI applications, AI agents, LLM-based applications, and MCP-based solutions, including MCP servers, clients, tools, and their integration with external systems.
• Knowledge of Advantest V93000 (93K) and SmarTest 8 (SMT8).
• Experience with AI-assisted or AI-native software development workflows.
• Experience testing complex enterprise or distributed applications, including API, interface, integration, and end-to-end testing.
• Exposure to performance and scalability testing, cloud-based test environments, and test reporting or quality dashboards.
• Experience with security testing of LLM applications (e.g. OWASP Top 10 for LLM Applications: prompt injection, insecure tool use, data leakage).
Personal Skills
• Strong analytical, debugging, and problem-solving skills.
• Clear communication and effective collaboration with software development teams.
• Ability to work independently, take ownership of tasks, and maintain a strong focus on software quality.
Skills Required
- Bachelor’s degree in Computer Science, Software Engineering, Information Technology, or a related field
- 3+ years of experience in software testing and test automation
- Strong experience with Linux, Python, Shell scripting, automated test frameworks, and manual and automated testing
- Knowledge of interface, API, integration, end-to-end, and system testing across distributed software components
- Experience with Docker, CI/CD, Jenkins, Bazel, Git, and Bitbucket
- Understanding of test strategies, test design, defect management, debugging, and troubleshooting
- Experience working in Agile/Scrum development environments
- Experience testing GenAI applications, AI agents, LLM-based applications, and MCP-based solutions
- Knowledge of Advantest V93000 and SmarTest 8
- Experience with AI-assisted or AI-native software development workflows
- Experience testing complex enterprise or distributed applications
- Exposure to performance and scalability testing, cloud-based test environments, and test reporting or quality dashboards
- Experience with security testing of LLM applications, including prompt injection, insecure tool use, and data leakage
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
For over a half-century, Advantest has been designing innovative electronic measuring equipment and semiconductor test systems essential to the development and manufacture of advanced computer and telecommunications products. On April 1, 2012, Advantest completed its integration of Verigy Ltd. Additional Information about Advantest can be found at www.advantest.com.








