Senior QA Engineer

Posted 8 Days Ago
Be an Early Applicant
Calgary, AB
In-Office
5-5
Senior level
Information Technology • Design
The Role
Lead quality assurance initiatives, design testing strategies, execute manual testing, mentor junior team members, and ensure software quality across projects.
Summary Generated by Built In

 

Senior Automation QA Engineer 

Location: Calgary, Canada (Remote-Friendly) 

Robots & Pencils is seeking a Senior Automation QA Engineer to lead automated quality assurance efforts for an AI-powered chat platform that helps students get instant, intuitive answers across a wide range of topics. 

Our client currently operates an existing chatbot that is experiencing critical issues. This project will rebuild and modernize the chat experience, leveraging large language model (LLM) capabilities to deliver natural, conversational interactions that meet modern user expectations. You’ll play a key role in ensuring this next-generation platform achieves exceptional reliability, performance, and user satisfaction. 

As a senior member of the engineering team, you’ll take ownership of the end-to-end automation strategy and execution, from framework design and test coverage planning to implementation, integration, and continuous improvement. You’ll collaborate closely with engineers, designers, and product managers to deliver intelligent, high-quality, and seamless user experiences. 

 

Key Responsibilities 

Automation Strategy & Framework Development 

  • Define and implement comprehensive automation strategies, frameworks, and coding standards. 
  • Design, develop, and maintain automated test suites for web, API, and backend components. 
  • Translate complex user flows and LLM-based conversational logic into automated validation scripts. 
  • Integrate automation into CI/CD pipelines to support continuous testing and fast feedback loops. 

Testing & Reporting 

  • Execute automated test runs across multiple environments and platforms. 
  • Monitor, debug, and maintain automated test results, ensuring stability and reliability. 
  • Identify, document, and track software defects through resolution in collaboration with development teams. 
  • Deliver clear automation metrics and quality reports to stakeholders, including test coverage and release readiness. 

Engineering Collaboration 

  • Partner with developers and product managers to ensure quality is embedded at every stage of the SDLC. 
  • Participate in sprint planning, backlog grooming, and code reviews to align automation efforts with release goals. 
  • Collaborate on test data setup, mocking strategies, and CI/CD optimization for faster deployments. 
  • Advocate for testability, maintainability, and performance in architectural discussions. 

Process & Tooling Excellence 

  • Evaluate and adopt modern automation tools (e.g., Playwright, Cypress, Selenium, or equivalent). 
  • Implement API testing using tools such as Postman, REST Assured, or similar frameworks. 
  • Support performance testing (e.g., JMeter, k6) and monitor system behavior under load. 
  • Ensure adherence to accessibility, security, and performance testing standards. 

Team Leadership & Knowledge Sharing 

  • Mentor QA engineers in automation best practices, code quality, and test design principles. 
  • Collaborate with distributed teams to maintain consistency in automation coverage and practices. 
  • Champion a culture of continuous improvement, emphasizing reliability, observability, and proactive testing. 

 

Required Skills & Qualifications 

  • 5+ years of experience in software quality assurance, with a strong focus on test automation. 
  • Proficiency in JavaScript/TypeScript or Python for building and maintaining test automation frameworks. 
  • Experience with UI and API automation tools such as Playwright, Cypress, Selenium, or REST Assured. 
  • Solid understanding of CI/CD pipelines and integration of automated tests (GitHub Actions, GitLab CI, Jenkins, etc.). 
  • Strong knowledge of QA methodologies, SDLC, and Agile processes. 
  • Familiarity with web application architecture and REST/GraphQL API testing. 
  • Upper-Intermediate English proficiency (written and spoken) for effective collaboration. 

 

Nice to Have 

  • Experience testing AI-powered or LLM-based chat applications. 
  • Familiarity with performance and load testing tools (k6, JMeter). 
  • Experience working with cloud environments (AWS, GCP, Azure) and containerized systems (Docker, Kubernetes). 
  • Knowledge of data validation, logging, and observability tools (Datadog, Kibana, Sentry). 
  • Contributions to automation frameworks, open-source tools, or QA process documentation. 

 

Personal Competencies 

  • Accountability – Owns the automation quality process from framework to release validation. 
  • Technical Craftsmanship – Writes efficient, maintainable, and scalable automation code. 
  • Collaboration – Builds trust and alignment with engineers and product teams. 
  • Analytical Thinking – Solves complex test and infrastructure challenges through data-driven insights. 
  • Adaptability – Excels in fast-paced, evolving projects that leverage cutting-edge AI technologies. 
  • Quality Mindset – Balances speed, precision, and long-term maintainability. 

 

Top Skills

Appium
AWS
Azure
Cypress
GCP
Git
GraphQL
JIRA
Jmeter
K6
Playwright
Postman
Rest
Testrail
Zephyr
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Chagrin Falls, OH
149 Employees
Year Founded: 2009

What We Do

Robots & Pencils develops digital strategies and products that deliver exponential impact to our clients. We design and build solutions that unlock data and insights, infuse intelligent automation, and accelerate product innovation across the organization.

Everything we do starts by blending the sciences with the humanities, the Robots with the Pencils. Our top-tier talent fuses creativity + technology to help brands transform their businesses, deliver delightful customer and employee experiences, and maintain a competitive edge amidst a constantly changing industry landscape.

Similar Jobs

Robots & Pencils Logo Robots & Pencils

Quality Assurance Engineer

Information Technology • Design
In-Office or Remote
Calgary, AB, CAN

CrowdStrike Logo CrowdStrike

Engineer III - Backend, Ingestion (Remote, CAN)

Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Remote or Hybrid
6 Locations
115K-165K Annually

CrowdStrike Logo CrowdStrike

Senior Software Engineer

Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Remote or Hybrid
6 Locations
175K-250K Annually

Wells Fargo Logo Wells Fargo

Consultant

Fintech • Financial Services
Hybrid
3 Locations
118K-201K Annually

Similar Companies Hiring

Axle Health Thumbnail
Logistics • Information Technology • Healthtech • Artificial Intelligence
Santa Monica, CA
17 Employees
Scrunch AI Thumbnail
Software • SEO • Marketing Tech • Information Technology • Artificial Intelligence
Salt Lake City, Utah
Standard Template Labs Thumbnail
Software • Information Technology • Artificial Intelligence
New York, NY
10 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account