Free · takes a minute · your resume fills your profile
CoreHRx never asks candidates for money. If anyone does, .
Sign in to see your fit: the skills, experience and location this job asks for, checked against your profile.
The Role
We are seeking a Senior SDET to take ownership of quality across our customer-facing products, including web dashboards, APIs, notifications, and GenAI solutions such as chatbots, voice IVR, and AI agents. In this role, you will define testing methodologies for AI agents, develop automated evaluations, and establish quality criteria for releases.
You will be responsible for the test strategy in your area and continually build and enhance UI, API, and end-to-end automation and regression coverage. This is a hands-on engineering position working closely with GenAI engineering, CX engineering, and product teams.
Key Responsibilities
● Own the test strategy and release quality criteria for your product area.
● Build and maintain UI automation (using Selenium or Playwright), API automation, and end-to-end regression test suites.
● Design evaluation strategies for AI agents, utilizing LLM-as-judge, rubrics, and other assessment techniques.
● Create evaluation datasets and automated evaluation suites in Python, integrated with CI/CD pipelines.
● Test AI agents for accuracy, consistency, tool usage, safety, and data handling.
● Debug agent failures using LangSmith or similar AI observability tools.
● Analyze quality across releases to identify regressions.
● Review product requirements to pinpoint gaps in requirements and test coverage.
● Review test code, define testing standards, and mentor other SDETs.
● Communicate findings and recommendations with engineering, product, and leadership teams.
Requirements
● 6–9 years of software testing experience, including hands-on SDET roles.
● Proficiency in both Python and Java.
● Experience owning test strategies and enhancing automation and release quality.
● Experience with UI automation tools such as Selenium or Playwright.
● Experience developing API automation and end-to-end frameworks within CI/CD.
● Ability to analyze product requirements and spot testing gaps.
● Previous experience testing chatbots, IVRs, and GenAI agents.
● Experience designing evaluations, rubrics, and LLM-as-judge assessments.
● Working knowledge of LangChain, LangGraph, LangSmith, or similar tools.
● Experience creating evaluation frameworks or tools for AI agents.
● Strong analytical and communication skills.
Tech You Will Work With
Python, Java, Selenium, Playwright, REST API testing, Postman, TestNG, PyTest, LangChain, LangGraph, LangSmith, LLM evaluation, LLM-as-judge, RAG testing, AI agent testing, end-to-end testing, regression testing, CI/CD, GitHub Actions, Jenkins, AWS.
From live job posts on CoreHRx, updated 7 Nov 2026. Skill figures count posts that list their skills; salary figures only posts that show a salary.
The post does not state the pay. As an estimate, comparable posts on CoreHRx advertise ₹11L – ₹25L a year (typical for IT Jobs in India, 903 posts).
The post asks for 6–9 Yrs experience.
It is a full time job.
Interview preparation · 60 questions
This guide is for Indian job seekers preparing for Java backend roles, from freshers facing campus and first-job interviews to experienced developers moving into senior positions at product companies and IT services firms. It covers what interviewers ask most in 2025 and 2026 on Java 17 and 21 and Spring Boot 3: OOP, collections, equals and hashCode, streams, exceptions, concurrency and JVM memory, then dependency injection, auto-configuration, REST controllers, validation, JPA and Hibernate, security, microservices and testing. Questions are grouped as easy, medium and hard. Read each answer, compile the code yourself and practise explaining it aloud.
Questions you may be asked
Create a free profile and we email you new IT jobs, at most once a week.
