SDET Contractor

Posted 4 Days Ago
Be an Early Applicant
Waterloo, ON, CAN
In-Office
51-77 Annually
Mid level
Gaming
The Role
Develop and execute quality engineering strategies for Agentic AI systems, APIs, microservices, and distributed services. Build scalable automated testing frameworks covering functional, integration, contract, regression, and end-to-end testing. Evaluate non-deterministic AI behavior, guardrails, tool use, groundedness, and failure handling. Integrate tests into CI/CD pipelines, investigate distributed-system defects, establish quality metrics, and support root-cause analysis, defect triage, and release readiness.
Summary Generated by Built In

Why Sony Interactive Entertainment?

Sony Interactive Entertainment isn’t just the Best Place to Play — it’s also the Best Place to Work. Sony Interactive Entertainment (SIE) is the company behind the PlayStation brand. As a subsidiary of Sony Group Corporation, we’re part of a proud legacy of innovation and excellence. SIE is a dynamic technology company, delivering cutting-edge hardware and network services to more than 100 million people and an entertainment leader, home to some of the most beloved and recognizable intellectual properties (IP) in the world. Our role at SIE is to create and nurture the experiences under the PlayStation brand, a name synonymous with entertainment excellence and creativity.

SDET Contractor

We are looking for an experienced Software Development Engineer in Test (SDET) to help ensure the quality, reliability, and resilience of Agentic AI systems and backend services. This role will design and implement automated testing strategies across AI/LLM-driven applications, APIs, microservices, and distributed systems.

The ideal candidate brings strong Quality Engineering and test automation fundamentals, with experience testing Generative AI, LLM-based, or Agentic AI systems. They should be comfortable testing complex systems where behavior may be non-deterministic and traditional pass/fail testing alone may not be sufficient.


Responsibilities
  • Develop and execute comprehensive QE strategies and test plans for Agentic AI systems, backend services, APIs, and service integrations.
  • Design and maintain scalable automated test frameworks across API, functional, integration, contract, regression, and end-to-end testing.
  • Develop testing approaches for non-deterministic AI behavior and establish meaningful evaluation criteria.
  • Identify quality risks early and apply risk-based, shift-left testing practices throughout the development lifecycle.
  • Integrate automated testing into CI/CD pipelines and establish effective quality gates.
  • Partner with Software Engineers, Product Managers, and other QE teams to improve testability, observability, and overall product quality.
  • Analyze defects and production issues to identify root causes, testing gaps, and opportunities for continuous QE improvement.
  • Establish quality metrics and provide visibility into test coverage, reliability, defects, and release readiness.

What You'll Do
  • Test Agentic AI workflows, including multi-step execution, tool selection and invocation, task completion, and failure handling.
  • Validate AI-generated responses using appropriate measures for accuracy, relevance, groundedness, consistency, and adherence to expected behavior.
  • Test AI guardrails, unexpected inputs, and failure scenarios, including incorrect tool usage and prompt-based attacks.
  • Validate interactions between AI agents, APIs, tools, data sources, and downstream services.
  • Develop reusable automation, test data, and evaluation scenarios for repeatable AI and services testing.
  • Validate API contracts, schemas, authentication, business rules, error handling, and service integrations.
  • Investigate failures across distributed systems and distinguish application, integration, environmental, automation, and AI-related issues.
  • Participate in defect triage, root-cause analysis, regression testing, and release-readiness activities.

Qualifications
  • Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field, or equivalent practical experience.
  • 4+ years of experience in Software Development Engineering in Test, Quality Engineering, Software Engineering, or a related discipline.
  • Strong experience testing REST APIs, backend services, microservices, or distributed systems.
  • Hands-on experience developing automated tests using Python, Java, JavaScript, TypeScript, or a similar programming language.
  • Experience with functional, integration, regression, contract, and end-to-end testing.
  • Experience integrating automated tests into CI/CD pipelines.
  • Strong analytical, debugging, and root-cause-analysis skills.
  • Experience working in an Agile development environment and collaborating across engineering teams.

Required Skills
  • Strong understanding of software testing methodologies and Quality Engineering practices.
  • Strong API and services testing experience, including REST, JSON, schemas, authentication, and error handling.
  • Experience designing and maintaining automated test frameworks.
  • Experience with Git and CI/CD development workflows.
  • Ability to develop risk-based test strategies and identify appropriate testing layers.
  • Experience testing distributed systems and service-to-service integrations.
  • Hands-on experience testing Generative AI, LLM-powered applications, or Agentic AI systems.
  • Understanding of testing non-deterministic AI behavior and evaluating outputs beyond exact-match assertions.

Preferred Skills (Good to have)
  • Experience testing Agentic AI workflows, including planning, multi-step execution, tool calling, memory, or state management.
  • Experience with AI evaluation techniques such as evaluation datasets, semantic evaluation, LLM-as-a-Judge, groundedness, relevance, or hallucination detection.
  • Experience with AI/LLM evaluation or observability tools such as LangSmith, DeepEval, Ragas, Arize Phoenix, Promptfoo, or similar technologies.
  • Experience testing RAG systems or AI agents interacting with APIs, databases, or external tools.
  • Experience with contract testing tools such as Pact and API specifications such as OpenAPI/Swagger.
  • Experience with event-driven architectures, Kafka, Docker, Kubernetes, or cloud platforms such as AWS.
  • Experience with performance, scalability, reliability, or resiliency testing.
  • Experience establishing QE metrics, quality gates, or cross-team testing standards.

Sony is an Equal Opportunity Employer. All persons will receive consideration for employment without regard to race, color, religion, gender, pregnancy, national origin, ancestry, citizenship, age, legally protected physical or mental disability, covered veteran status, status in the U.S. uniformed services, sexual orientation, marital status, genetic information or membership in any other legally protected category.

We strive to create an inclusive environment, empower employees and embrace diversity. We encourage everyone to respond.

We sincerely appreciate the time and effort you spent in contacting us and we thank you for your interest in PlayStation.

At SIE, we consider several factors when setting each role’s base pay range, including the competitive benchmarking data for the market and geographic location.
Please note that the base pay range may vary in line with our hybrid working policy and individual base pay will be determined based on job-related factors which may include knowledge, skills, experience, and location. 

The estimated Ontario base pay range for this role is listed below.
$51.61$77.42 CAD

Please note, Sony Interactive Entertainment conducts background checks at the offer stage for all new employees (which may include criminal background checks for some roles) and will need to process personal information to support these checks.

Please refer to our Candidate Privacy Notice for more information about what personal information we collect, how we use it, who we share it with, and your data protection rights.

Equal Opportunity Statement:

Sony is an Equal Opportunity Employer. All persons will receive consideration for employment without regard to gender (including gender identity, gender expression and gender reassignment), race (including colour, nationality, ethnic or national origin), religion or belief, marital or civil partnership status, disability, age, sexual orientation, pregnancy, maternity or parental status, trade union membership or membership in any other legally protected category.

We strive to create an inclusive environment, empower employees and embrace diversity. We encourage everyone to respond. 

Sony Interactive Entertainment is a Fair Chance employer and qualified applicants with arrest and conviction records will be considered for employment.


Skills Required

  • Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field, or equivalent practical experience
  • 4+ years of experience in SDET, Quality Engineering, Software Engineering, or a related discipline
  • Experience testing REST APIs, backend services, microservices, or distributed systems
  • Hands-on experience developing automated tests using Python, Java, JavaScript, TypeScript, or a similar programming language
  • Experience with functional, integration, regression, contract, and end-to-end testing
  • Experience integrating automated tests into CI/CD pipelines
  • Strong analytical, debugging, and root-cause-analysis skills
  • Experience working in an Agile development environment and collaborating across engineering teams
  • Strong understanding of software testing methodologies and Quality Engineering practices
  • Strong API and services testing experience, including REST, JSON, schemas, authentication, and error handling
  • Experience designing and maintaining automated test frameworks
  • Experience with Git and CI/CD development workflows
  • Ability to develop risk-based test strategies and identify appropriate testing layers
  • Experience testing distributed systems and service-to-service integrations
  • Hands-on experience testing Generative AI, LLM-powered applications, or Agentic AI systems
  • Understanding of testing non-deterministic AI behavior and evaluating outputs beyond exact-match assertions
  • Experience testing Agentic AI workflows, including planning, multi-step execution, tool calling, memory, or state management
  • Experience with AI evaluation techniques such as evaluation datasets, semantic evaluation, LLM-as-a-Judge, groundedness, relevance, or hallucination detection
  • Experience with AI/LLM evaluation or observability tools such as LangSmith, DeepEval, Ragas, Arize Phoenix, Promptfoo, or similar technologies
  • Experience testing RAG systems or AI agents interacting with APIs, databases, or external tools
  • Experience with contract testing tools such as Pact and API specifications such as OpenAPI/Swagger
  • Experience with event-driven architectures, Kafka, Docker, Kubernetes, or AWS
  • Experience with performance, scalability, reliability, or resiliency testing
  • Experience establishing QE metrics, quality gates, or cross-team testing standards

PlayStation Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about PlayStation and has not been reviewed or approved by PlayStation.

  • Healthcare Strength The package includes medical, dental, vision, prescription coverage, mental wellbeing programs, disability, and life insurance. Feedback suggests coverage breadth is a consistent strength that helps anchor overall comp satisfaction.
  • Retirement Support U.S. offerings include a 401(k) with a company match along with student‑loan repayment and a college‑savings plan match. These financial supports are positioned as strong additions to total compensation.
  • Parental & Family Support Offerings include paid parental and family‑care leave, fertility and family‑forming support, backup care, and child developmental resources. Feedback suggests these programs are valued and contribute to a well‑rounded package.

PlayStation Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: San Mateo, CA
8,768 Employees
Year Founded: 1994

What We Do

PlayStation has been at the forefront of interactive and digital entertainment since the debut of our first console in 1994. Our products delight millions across the world through incredible games, cutting edge experiences and access to many types of media. This commitment to amazing our fans is at the core of who we are and one we share with Sony Corporation, internationally known as a leader in music, movies and consumer electronics. We can only achieve this goal by welcoming talented people and empowering them to do their best work. From game developers to data scientists, software engineers to cybersecurity experts, marketing to accounting and finance professionals, we’re always looking for talented people who share a passion for creating and our commitment to delivering amazement.

Similar Jobs

Ericsson Logo Ericsson

Radio Algorithm Co-op

Cloud • Information Technology • Internet of Things • Machine Learning • Software • Cybersecurity • Infrastructure as a Service (IaaS)
In-Office
Ottawa, ON, CAN
88000 Employees
26-37 Hourly

Square Logo Square

Software Engineer

eCommerce • Fintech • Hardware • Payments • Software • Financial Services
Remote or Hybrid
8 Locations
12000 Employees
185K-327K Annually

Block Logo Block

Software Engineer

Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
In-Office or Remote
8 Locations
12000 Employees
185K-327K Annually

Opendoor Logo Opendoor

Marketing Manager

eCommerce • Fintech • Real Estate • Software • PropTech
Hybrid
3 Locations
1600 Employees

Similar Companies Hiring

DraftKings Thumbnail
Digital Media • Gaming • Information Technology • Software • Sports • Esports • Big Data Analytics
Boston, MA
6400 Employees
bet365 Thumbnail
Digital Media • Gaming • Software • Esports • Automation
Denver, Colorado
10000 Employees
ARB Interactive Thumbnail
Gaming • Mobile • Software
Miami, Florida
190 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account