The Role
Build and maintain Java backend, API, database, UI, integration, and Playwright end-to-end automation. Establish CI quality gates, mocking and fixture strategies, test data management, and reliable regression coverage. Develop LLM evaluation practices using golden datasets, observability, trace capture, guardrails, structured outputs, and measurable release thresholds. Improve test maintainability, reduce flakiness, optimize runtime, and monitor model reliability and drift.
Summary Generated by Built In
About Qualysoft
· 25 years of experience in software engineering, established in Vienna, Austria
· Active in Romania since 2007, with office in central Bucharest (Bd. Iancu de Hunedoara 54B)
· Delivering End to End IT Consulting Services - From Team Augmentation and Dedicated Teams to Custom Software Development
· We deliver scalable enterprise systems, intelligent automation frameworks, and digital transformation platforms
· Cross-industry experience by sustaining global players in BSFI (Banking, financial services and insurance), Telecom,Retail & E-commerce, Energy and Utilities, Automotive, Manufacturing, Logitics, High Tech
· Global Presence: Switzerland, Germany, Austria, Sweden, Hungary, Slovakia, Serbia, Romania, and Indonesia
· International team of 500+ software engineers
· Strategic partnerships: Microsoft Cloud Certified Partner, Tricentis Solutions Partner in Test Automation and Test Management, Creatio Exclusive Partner, Doxee Implementation Partner
· Powered by cutting-edge technologies: AI, Data & Analytics, Cloud, DevOps, IoT, and Test Automation.
· Project beneficiaries ranging from large-scale enterprises to startups
· Stable growth and revenue increase year over year, a resilient organisation in volatile IT market conditions
· Quality-first mindset, culture of innovation, and long-term client partnerships
· Global and local reach – trusted by key industry players in Europe and the US
Responsibilities:
Build API and integration test coverage for Java/Spring routers and database workflows.
• Design and implement Playwright E2E tests for critical user journeys.
• Create stable mocking and fixture strategies (MSW, test fixtures, seeded test data).
• Set up CI quality gates for linting, unit/integration/E2E tests, coverage, and reporting.
• Expand coverage to untested UI and workflow areas and prevent regressions.
• Build an LLM evaluation discipline with golden datasets and measurable quality gates.
• Track model output reliability, detect drift, and gate releases on quality/stability thresholds.
• Improve test maintainability, remove flakiness, and optimize runtime.
• LLM QA Tooling
• Use LangChain4j for JVM-native LLM application support, including agents, RAG, and guardrails.
• Use Langfuse Java SDK and OpenTelemetry for trace capture, prompt/version tracking, and experiment comparison.
• Use the OpenAI Java SDK for structured outputs, function calling, webhook verification, and response validation.
• Use Promptfoo in CI when you need language-agnostic regression and red-team checks around the Java stack.
• Use Java schema validation and guardrail patterns with Jackson, Swagger annotations, and LangChain4j guardrails.
• Use OpenTelemetry-based traces and Langfuse datasets as the reproducible source of truth for release gates.
• Design and implement Playwright E2E tests for critical user journeys.
• Create stable mocking and fixture strategies (MSW, test fixtures, seeded test data).
• Set up CI quality gates for linting, unit/integration/E2E tests, coverage, and reporting.
• Expand coverage to untested UI and workflow areas and prevent regressions.
• Build an LLM evaluation discipline with golden datasets and measurable quality gates.
• Track model output reliability, detect drift, and gate releases on quality/stability thresholds.
• Improve test maintainability, remove flakiness, and optimize runtime.
• LLM QA Tooling
• Use LangChain4j for JVM-native LLM application support, including agents, RAG, and guardrails.
• Use Langfuse Java SDK and OpenTelemetry for trace capture, prompt/version tracking, and experiment comparison.
• Use the OpenAI Java SDK for structured outputs, function calling, webhook verification, and response validation.
• Use Promptfoo in CI when you need language-agnostic regression and red-team checks around the Java stack.
• Use Java schema validation and guardrail patterns with Jackson, Swagger annotations, and LangChain4j guardrails.
• Use OpenTelemetry-based traces and Langfuse datasets as the reproducible source of truth for release gates.
Qualifications:
3+ years in QA automation for production systems.
• Strong Java backend testing skills.
• Strong frontend testing.
• Experience building and maintaining Playwright/Cypress/Selenium E2E suites.
• API testing experience (REST, schema validation, auth/error scenarios).
• Experience with SQL/PostgreSQL testing and test data management.
• CI/CD pipeline experience (GitHub Actions, Azure Pipelines, or Jenkins).
• Practical understanding of testing non-deterministic LLM outputs.
• Strong Java backend testing skills.
• Strong frontend testing.
• Experience building and maintaining Playwright/Cypress/Selenium E2E suites.
• API testing experience (REST, schema validation, auth/error scenarios).
• Experience with SQL/PostgreSQL testing and test data management.
• CI/CD pipeline experience (GitHub Actions, Azure Pipelines, or Jenkins).
• Practical understanding of testing non-deterministic LLM outputs.
Nice to have:
Familiarity with Java-compatible LLM evaluation and regression tooling such as Promptfoo and Langfuse, plus tracebased observability and output validation via API/OpenTelemetry integrations.
• Performance and benchmark testing experience.
• WebSocket testing experience.
• Security and dependency scanning awareness.
• Experience in legal-tech, fintech, or other regulated domains.
• Performance and benchmark testing experience.
• WebSocket testing experience.
• Security and dependency scanning awareness.
• Experience in legal-tech, fintech, or other regulated domains.
What We Offer
· Premium medical package
· Lunch Tickets & Pluxee Card
· Bookster subscription
· 13th salary/yearly bonuses
· Enterprise job security with a startup mentality (diverse & engaging environment, international exposure, flat hierarchy) under the stability of a secure multinational
· A supportive culture (we value ownership, autonomy, and healthy work-life balance) with great colleagues, team events and activities
· Flexible working program and openness to remote work
· Collaborative mindset – employees shape their own benefits, tools, team events and internal practices
· Diverse opportunities in Software Development with international exposure
· Flexibility to choose projects aligned with your career path and technical goals
· Access to leading learning platforms, courses, and certifications (Pluralsight, Udemy, Microsoft, Google Cloud)
· Career growth & learning – mentorship programs, certifications, professional development opportunities, and above-market salary
*We are an equal opportunity employer and value diversity. All employment decisions are made without regard to age, gender, disability, race, ethnicity, religion, sexual orientation, or any other protected characteristic. We encourage applicants from all backgrounds to apply.*
Skills Required
- 3+ years of QA automation experience for production systems
- Strong Java backend testing skills
- Strong frontend testing skills
- Experience building and maintaining Playwright, Cypress, or Selenium end-to-end test suites
- API testing experience, including REST, schema validation, authentication, and error scenarios
- Experience with SQL or PostgreSQL testing and test data management
- CI/CD pipeline experience with GitHub Actions, Azure Pipelines, or Jenkins
- Practical understanding of testing nondeterministic LLM outputs
- Familiarity with Promptfoo, Langfuse, trace-based observability, and API or OpenTelemetry output validation
- Performance and benchmark testing experience
- WebSocket testing experience
- Security and dependency scanning awareness
- Experience in legal-tech, fintech, or other regulated domains
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
With 25 years of experience serving more than 250 international clients, we are a one-stop shop for IT services and digitizing business needs. Powered by more than 500 specialists in Central and Eastern Europe, we have the expertise to close the gap between technology and your business growth
