CoreHRx Jobs

Apply in one click, save jobs and take free mock tests.

Register free Log in
Hiring? Post jobs with CoreHRx
My account
user

View & update profile

Data Scientist - Evaluations, Chanakya

Sarvam Verified employer

Full time On-site
Location
Bangalore
Work mode
On-site
Salary
₹15L – ₹25L a year estimatetypical for IT Jobs in Bangalore, 254 posts
Posted 6 months ago Last checked 3 hr ago

Free · takes a minute · your resume fills your profile

CoreHRx never asks candidates for money. If anyone does, .

Role at a glance

You will
  • This is an intellectually demanding role.
  • The Data Scientist anchors the evaluations function for this vertical — designing, building, and maintaining evaluation frameworks that…
  • You are not running standard benchmarks.
Must-have skills
NumPyPandasPrompt EngineeringPython
Work mode
On-site

How well do your skills match this job?

Sign in to see your fit: the skills, experience and location this job asks for, checked against your profile.

Job description

Location: Bengaluru
Department: Engineering

About Sarvam

Sarvam is building the bedrock of Sovereign AI for India. The company is developing India's full-stack sovereign AI platform, building across research, models, infrastructure and applications with a singular focus on making AI genuinely work for India. Sarvam works with leading enterprises and public institutions and is backed by Lightspeed, Peak XV, and Khosla Ventures. Sarvam partners with India's leading brands, including Tata Capital, SBI Life, CRED, IDFC, and LIC.

About the Role

This is an intellectually demanding role. The Data Scientist anchors the evaluations function for this vertical — designing, building, and maintaining evaluation frameworks that measure model and system quality in operational context. You are not running standard benchmarks. You are building domain-specific eval harnesses for high-stakes use cases where a wrong answer carries real consequences.

You will work closely with the MLOps Engineer, the PM, and the deployment team. The evaluations you design are the mechanism by which the team determines whether what we've built is good enough to deploy — and whether it stays good after deployment.

What You'll Do

• Design and build evaluation frameworks for Sarvam's AI outputs across domain-specific requirements: document comprehension, command summarisation, geospatial reasoning, enterprise workflow automation, and others as they emerge

• Define quality metrics in collaboration with domain experts and clients; translate operational requirements into measurable, defensible signals

• Run structured evaluation cycles pre- and post-deployment; build dashboards that surface model quality in production

• Identify failure modes, edge cases, and distribution shifts — with the bias of someone looking for what's wrong, not confirming what's right

• Collaborate with the MLOps Engineer to operationalise eval pipelines — automated, triggered by deployment events, versioned, and reproducible

• Build and manage domain-specific datasets for fine-tuning, evaluation, and benchmarking — including human annotation workflows where needed

• Publish internal findings and quality reports that feed the product and engineering roadmap

What We're Looking For

• 3–6 years in data science, ML research, or applied AI; at least 2 years working with LLMs in production contexts

• Strong statistics and probability fundamentals — you understand what makes an evaluation valid and what makes it misleading

• Experience designing evaluation frameworks from scratch: custom metrics, inter-rater reliability, red-teaming methodologies

• Python proficiency; comfort with pandas, NumPy, HuggingFace datasets, RAGAS, EleutherAI Eval Harness, LangSmith, or equivalent

• Experience with prompt engineering, model fine-tuning, or RLHF in applied settings

• Ability to work with unstructured domain data: PDFs, doctrine documents, transcripts, and field reports

Bonus Points

• Prior work in high-stakes domains (healthcare, legal, defence, finance) where output quality carries real-world consequences

• Red-teaming or adversarial evaluation experience

Note: We are looking for people who can own the outcomes described here, not people who

match every line of this specification. If this problem excites you and you believe you can do this work, we want to hear from you.

Why Sarvam?

Sarvam is a fast-moving, high talent-density team building full-stack AI for India, working on problems that push the frontiers of AI with real population-scale impact.

• Work alongside researchers, engineers, builders, and business leaders who move fast and hold each other to a very high bar

• High ownership and high impact, from day one

• Everything we do is AI-first, from the way we build and ship to the way we think about problems

• You can work on problems that could change how an entire country learns, works, and communicates

If you want to work on problems at the frontier of AI in India, Sarvam is the place to be.

Key skills

Required

NumPyPandasPrompt EngineeringPython

Soft skills

Ownership

Job market insights

48active NumPy jobs in Bangalore
65active Pandas jobs in Bangalore
148active Prompt Engineering jobs in Bangalore
₹15L – ₹25L advertised yearly salary for IT Jobs in Bangalore (middle half of 254 posts that show a salary; median ₹19L)
46 more open jobs at Sarvam

Most IT Jobs posts in Bangalore ask for 3–6 years of experience (41% of 3805 that say).

Hiring most for IT Jobs in Bangalore

  • Bosch Group166 open jobs
  • Trigent Software Private Limited138 open jobs
  • Brillio116 open jobs

Jobs asking for NumPy in Bangalore also ask for

  • Python98%
  • Pandas85%
  • Scikit-learn60%
  • AWS52%
  • Machine Learning50%

From live job posts on CoreHRx, updated 9 Nov 2026. Skill figures count posts that list their skills; salary figures only posts that show a salary.

Questions about this job

What is the salary for this Data Scientist - Evaluations, Chanakya job?

The post does not state the pay. As an estimate, comparable posts on CoreHRx advertise ₹15L – ₹25L a year (typical for IT Jobs in Bangalore, 254 posts).

Where is this job based?

In Bangalore (on-site).

What kind of job is this?

It is a full time job.

This page helps Indian job seekers prepare for Python interviews for backend, data engineering and data analyst roles, whether you are a fresher facing your first technical round or an experienced developer aiming for a senior position. It has 60 questions in three levels, easy, medium and hard, each with a concise sample answer and, where it helps, runnable Python 3.12 code. Topics include data structures, OOP, generators, decorators, the GIL and concurrency, type hints, testing and performance. Answer each question aloud in your own words first, then compare with the sample answer and run the code to check your understanding.

Questions you may be asked

  1. What is the difference between mutable and immutable types in Python?
  2. What is the difference between a list and a tuple?
  3. What is the difference between is and ==?
  4. What are *args and **kwargs?

Get IT jobs in Bangalore by email

Create a free profile and we email you new IT jobs in Bangalore, at most once a week.

Similar jobs

Free · takes a minute · your resume fills your profile