As a Senior Software Development Engineer in Test, you will be responsible for ensuring the quality and reliability of our AI-powered chatbot systems and email drafting tools. You will design and execute both manual and automated tests, identify bugs, and collaborate closely with developers, product owners, and data scientists to deliver exceptional user experiences.
The candidate MUST demonstrate real depth in at least ONE of the following areas:
A) Deep AI/Chatbot testing
B) Automation ownership (frameworks from scratch, design/architecture, not just implementation).
Location: Only LATAM 100% remote. Working hours are based on the US Central or Eastern Time Zone. With at least 6 hours of overlap.
*Opening not available for the US or Europe.
About Us:
Abstra is a fast-growing, Nearshore Tech Talent services company, providing top Latin American tech talent to U.S. companies and beyond. Founded by U.S.-bred engineers with over 15 years of experience, Abstra specializes in sourcing skilled professionals across a wide range of technologies to meet our clients’ needs, driving innovation and efficiency.
Job Description:
Key Responsibilities:
- Design, develop, and execute manual and automated end-to-end test cases for conversational AI systems and generative email tools.
- Validate natural language processing (NLP) outputs for accuracy, tone, and contextual relevance.
- Build and maintain automated test frameworks for regression, integration, and performance testing.
- Collaborate with cross-functional agile teams to define test strategies, acceptance criteria, and quality benchmarks.
- Identify, document, and track bugs and inconsistencies in Azure DevOps Services and collaborate with the development team to prioritize and resolve issues.
- Perform exploratory testing to uncover edge cases and usability issues.
- Monitor and analyse test results and provide actionable feedback to the development team for rapid iteration and improvement.
- Create and maintain test documentation, including test cases, scripts, and reports.
- Work in an agile development environment, collaborating with the team to deliver quality software iteratively
- Contribute to the continuous improvement of QA processes and best practices.
- Responsible for protecting, securing, and properly handling all confidential data held by Ascensus to ensure against unauthorized access, improper transmission, and/or unapproved disclosure of information that could result in harm to Ascensus or our clients.
- The I-Client philosophy and the Core Values of People Matter, Quality First, and Integrity Always® should be visible in your actions on a day-to-day basis, showing your support of our organizational culture.
Requirements:
- At least 8 years of experience as a Software Engineer, a Software Engineer in Test, a Test Automation Engineer, or equivalent roles with a testing mindset.
- Experience testing AI/ML-based applications, especially chatbots or generative tools (mandatory).
- Experience with LLM-based systems (e.g., GPT, Claude, Gemini).
- Knowledge of prompt engineering and evaluation techniques.
- Passion for AI and a keen eye for user experience and conversational quality.
- Familiarity with A/B testing, telemetry, and user feedback analysis.
- Strong experience developing test frameworks with tools such as Playwright, Selenium, Cucumber, or Leapwork (mandatory).
- Familiarity with writing automation scripts with at least one object-oriented programming language, such as C# or Java.
- Strong experience with a scripting language such as Python, PowerShell, Perl, JavaScript.
- Strong experience working with Database testing techniques using Postgres, Sybase, SQL Server, or Oracle with a strong understanding of database schemas.
- Experience working with performance Engineers or Performance tools such as BlazeMeter, JMeter, and/or Taurus.
- Experience with testing REST API frameworks.
- Excellent verbal and written communication skills.
- Experience working in a fast-paced Agile or SCRUM development methodology.
What We Offer:
- Opportunities for professional growth and development.
- A collaborative and inclusive work environment.
- The chance to work on impactful projects with a talented team.
- Excellent compensation in USD.
- Hardware and software setup (mandatory)
Skills Required
- At least 8 years of experience in Software Engineering or equivalent roles
- Experience testing AI/ML applications, especially chatbots or generative tools
- Experience with LLM-based systems (e.g., GPT, Claude, Gemini)
- Knowledge of prompt engineering and evaluation techniques
- Strong experience developing test frameworks with tools such as Playwright, Selenium, Cucumber, or Leapwork
- Strong experience with a scripting language like Python, PowerShell, Perl, JavaScript
- Experience with database testing techniques (Postgres, Sybase, SQL Server, or Oracle)
- Experience with performance testing tools (BlazeMeter, JMeter, Taurus)
- Experience testing REST API frameworks
- Excellent verbal and written communication skills
What We Do
At Abstra, we bridge the gap between North American companies and Latin America’s top talent. With over 15 years of experience in the U.S. market and US-bred founders leading the way, we focus on building long-term partnerships to achieve your organization’s unique goals, fueling your growth with flexible tech solutions and highly qualified professionals.
.png){kind=logo}
