This position will:
- Ensure the quality, reliability, and compliance of Gen AI/Agentic AI solutions, leveraging AWS for data engineering and Azure OpenAI for LLM-based applications
- Design, develop, and execute comprehensive test plans for AI/ML models, data pipelines, and system integrations
- Collaborate with architects, developers, and data scientists to validate requirements and solution outputs
- Maintain high standards for test documentation, defect tracking, and reporting
- Advocate for best practices in AI/ML testing, including fairness, bias, and explainability
- Continuously improve testing processes and tools to enhance solution quality
- Coordinate with ServiceNow and Workday teams to validate end-to-end workflows and ensure seamless integration of AI components
Responsibilities:
Participate in joint testing efforts, focusing on the interfaces and data exchanges between Gen AI solutions, ServiceNow, and Workday< >Develop and maintain automated and manual test cases for Gen AI/Agentic AI use casesValidate outputs from Azure OpenAI LLMs for accuracy, relevance, fairness, and bias, using both quantitative and qualitative methodsTest AWS-based data pipelines for data integrity, transformation accuracy, and performance under various scenariosPerform end-to-end testing of APIs and microservices that integrate AWS data sources with Azure OpenAI LLMs and HR systemsConduct regression, performance, security, and compliance testing of AI solutions to ensure robustness and reliabilityDocument test results, track defects, and collaborate with developers to resolve issues efficientlyEnsure comprehensive test coverage for HR system integrations (e.g., Workday) and business-critical workflowsParticipate in design and code reviews to provide a quality perspective and identify potential risks earlyWork with ServiceNow and Workday teams to validate data exchanges and workflow triggers, ensuring AI outputs are correctly consumed and acted upon by downstream systemsSupport user acceptance testing (UAT) and integration testing phases involving ServiceNow and Workday and other HR systems
Qualifications:
Education:
Bachelor’s degree in Computer Science, Engineering, or a related field
Experience:
2+ years of experience in software, AI/ML, or cloud testing roles
Required Qualifications:
Familiarity with AWS data engineering environments and test automation frameworks (e.g., PyTest, Selenium)Experience with Prompt finetuning and hyperparameter tuningExperience validating LLM outputs and AI/ML model performanceUnderstanding of data quality, privacy, and compliance requirements in cloud environmentsStrong analytical skills and attention to detailProficiency in writing clear test documentation and defect reportsExperience working in cross-team environments and validating integrations with external platforms
Preferred Qualifications:
Certifications in AWS or Azure cloud platformsExperience testing Gen AI solutions, LLMs, or HR systemsKnowledge of Azure OpenAI and LLM evaluation techniquesExposure to MLOps and automated model validation pipelinesStrong communication and collaboration skillsExperience with agile development and QA methodologiesFamiliarity with ServiceNow and Workday integration testing is a plus
About Evernorth Health Services
Evernorth Health Services, a division of The Cigna Group, creates pharmacy, care and benefit solutions to improve health and increase vitality. We relentlessly innovate to make the prediction, prevention and treatment of illness and disease more accessible to millions of people. Join us in driving growth and improving lives.
Top Skills
What We Do
At Cigna, we're more than a health insurance company. We are your partner in total health and wellness. And we’re here for you 24/7 – caring for your body and mind.
As a global health service company, Cigna's mission is to improve the health, well-being, and peace of mind of those we serve by making health care simple, affordable, and predictable.
Our values are the core of our culture. Our values guide how all 74,000 of us around the world work together, serve our customers, patients, clients, communities, and deliver on our mission.
