CoreHRx Jobs

Apply in one click, save jobs and take free mock tests.

Register free Log in
Hiring? Post jobs with CoreHRx
My account
user

View & update profile

Senior Quality & AI Evaluation Engineer (AIRE)

BMW TechWorks

Full time
Location
India
Work mode
Not stated
Posted 3 days ago

Free · takes a minute · your resume fills your profile

CoreHRx never asks candidates for money. If anyone does, .

How well do your skills match this job?

Sign in to see your fit: the skills, experience and location this job asks for, checked against your profile.

Job description

Role

Senior Quality & AI Evaluation Engineer (Senior)

Visit our website bmwtechworks.in to know more.

Follow us on LinkedInInstagramFacebookX for the exciting updates.

About the UNIT/ Unit Overview

AIRE (AI-native Requirements Engineering) is a new BMW Group tool for managing vehicle requirements, the descriptions of what a car and its software must do. It keeps the full history of every requirement, shows how requirements are linked to each other and to tests, and shows what is affected when something changes. It helps teams meet automotive standards such as Automotive SPICE and ISO 26262. AIRE is built for AI agents first and a user interface later, and the team uses AI agents to help build it. In this role you make sure AIRE and its AI features work correctly, through automated tests and checks.

Location

Pune or Bangalore

Experience

Number of openings

1

What awaits you/ Job Profile

• Define and maintain the test and evaluation strategy, including test levels, quality gates and evidence.

• Automate BDD scenarios from the acceptance criteria, contract tests and property-based tests.

• Run load tests against the quality goals and report the results.

• Build and run evaluations for agents and AI functions together with the Lead Agentic Engineering & AI Platform Engineer.

• Produce test evidence for tool qualification together with the Functional Safety & Process Compliance Specialist.

• Act as the human approver for changes to test oracles, evaluation datasets and quality-gate thresholds.

• Set up automated quality gates (tests, coverage, static analysis, performance and security tests) from the very beginning, and keep them mandatory for every change.

What should you bring along

• Degree in Computer Science or a related field, or equivalent relevant experience.

• Minimum of five years of test or quality engineering experience.

• Demonstrated experience automating tests for data-intensive systems.

Must have technical skill

• Strong test engineering experience in Python (for example pytest), including test automation in CI/CD.

• Experience with BDD (for example Gherkin, behave or pytest-bdd) and contract testing.

• Experience with property-based testing (for example Hypothesis) for complex data logic.

• Experience with load and performance testing (for example k6 or Locust) against defined service-level objectives.

• Experience evaluating LLM-based systems and agents with golden datasets, regression evaluations and acceptance thresholds.

• Proficient AI-native engineering practice, including using agents to generate tests while keeping test oracles and evaluation criteria under human control.

Good to have technical skills

• Knowledge of ISO 26262 Part 8 tool qualification and of test evidence for Automotive SPICE.

• Experience with mutation testing and test-quality metrics.

• Experience with observability-based testing (for example OpenTelemetry).

• ISTQB certification.

Key skills

Required

CI/CDOpenTelemetryPerformance QA & TestingPyTestPythonTest AutomationObservability

Job market insights

3,675active CI/CD jobs in India
182active OpenTelemetry jobs in India
255active Performance QA & Testing jobs in India
9 more open jobs at BMW TechWorks

Jobs asking for CI/CD in India also ask for

  • Python50%
  • AWS44%
  • Microsoft Azure38%
  • Kubernetes36%
  • Docker32%

From live job posts on CoreHRx, updated 19 Nov 2026. Skill figures count posts that list their skills; salary figures only posts that show a salary.

Questions about this job

What kind of job is this?

It is a full time job.

This page helps Indian job seekers prepare for Python interviews for backend, data engineering and data analyst roles, whether you are a fresher facing your first technical round or an experienced developer aiming for a senior position. It has 60 questions in three levels, easy, medium and hard, each with a concise sample answer and, where it helps, runnable Python 3.12 code. Topics include data structures, OOP, generators, decorators, the GIL and concurrency, type hints, testing and performance. Answer each question aloud in your own words first, then compare with the sample answer and run the code to check your understanding.

Questions you may be asked

  1. What is the difference between mutable and immutable types in Python?
  2. What is the difference between a list and a tuple?
  3. What is the difference between is and ==?
  4. What are *args and **kwargs?

Get IT jobs by email

Create a free profile and we email you new IT jobs, at most once a week.

Similar jobs

Free · takes a minute · your resume fills your profile