Lead AI/ML Engineer

Location: Bengaluru, India

Position: Lead AI/ML Engineer
Experience: 8+ yrs
Location: Bangalore
Employment: Full time

Job Summary:
We’re looking for a Staff Engineer specializing in Agentic AI and Generative AI to join the team. Does this sound like a good role for you? In this role, you will be responsible for designing, developing, troubleshooting, and operating cloud-hosted AI applications and agentic platforms. You will collaborate with engineering, product, architecture, E2E and DevOps teams while contributing to architecture and design decisions that influence product direction. The ideal candidate will have strong software engineering experience with Python, REST APIs, distributed systems, cloud platforms, and modern AI technologies. We are looking for someone who can apply their experience with Large Language Models, Retrieval-Augmented Generation, agent orchestration, multi-agent patterns, evaluation frameworks, guardrails, and cloud-native deployment to build reliable, secure, scalable, and production-ready AI solutions.

Below is the list of key qualifications:

• Bachelor’s or master’s degree in Computer Science, Engineering, Artificial Intelligence, Data Science, or a related field.

• 8+ years of software development experience building enterprise-grade, customer-facing products and services, including at least 2 years of hands-on experience developing solutions using AI Agents, Skills, or Large Language Model technologies.

• Strong hands-on programming experience with Python , including object-oriented programming, asynchronous programming, concurrency, error handling, testing, packaging, and performance optimisation.

• Experience designing and implementing REST APIs and integrating internal and external services using structured JSON payloads, schemas, authentication, retries, rate limits, and resilient error-handling patterns.

• Experience building production-grade AI agents using frameworks such as Semantic Kernel, LangGraph, or LangChain.

• Experience with agent orchestration and multi-agent architecture patterns , including planning, tool selection, task delegation, workflow coordination, and communication between agents.

• Strong understanding of agent state, memory, context management, session management, checkpointing, and recovery for long-running or multi-step agent workflows.

• Experience designing and implementing agent evaluation frameworks, including quality, task completion, groundedness, safety, latency, reliability, and cost evaluations.

• Experience implementing AI guardrails, including input and output validation, prompt injection protection, access control, content filtering, structured responses, tool-use restrictions, and safe failure handling.

• Strong understanding of Large Language Models and transformer architectures, including attention mechanisms, tokenisation, context windows, model inference, and common model limitations.

• Experience with prompt engineering techniques such as system prompt design, few-shot prompting, structured prompting, tool or function calling, prompt templates, and prompt versioning.

• Experience designing and implementing Retrieval-Augmented Generation pipelines , including document ingestion, parsing, chunking, metadata enrichment, embedding generation, indexing, retrieval, re-ranking, citation, and response generation.

• Experience working with embeddings and vector databases or vector search platforms, including index design, metadata filtering, hybrid search, semantic search, and retrieval quality optimisation.

• Experience with context management and token optimisation techniques, including context selection, summarisation, compression, caching, prompt budgeting, and management of conversation history.

• Experience designing human-in-the-loop workflows for review, approval, exception handling, feedback collection, and escalation of sensitive or low-confidence AI decisions.

• Experience with model customization techniques such as fine-tuning, instruction tuning, parameter-efficient fine-tuning, or model adaptation, along with the ability to determine when fine-tuning is appropriate compared with prompting or RAG.

• Experience with AWS or GCP and their core compute, storage, networking, identity, security, logging, and monitoring services.

• Hands-on experience with Amazon Bedrock or equivalent managed Generative AI platforms, including model integration, inference configuration, security, observability, and cost management.

• Experience containerizing and deploying applications using Docker, with a sound understanding of container security, configuration, networking, resource management, and deployment fundamentals.

• Experience implementing CI/CD workflows using GitHub Actions or equivalent platforms for automated build, test, security validation, packaging, and deployment.

• Experience with AI and model observability, including monitoring of latency, availability, token consumption, cost, retrieval quality, model responses, evaluation scores, safety events, and production drift.

• Strong knowledge of software architecture, design patterns, data structures, algorithms, distributed systems, and cloud-native application design.

• Experience with test-driven development and automated testing using frameworks such as pytest, including unit, integration, API, workflow, evaluation, and end-to-end testing.

• Strong understanding of security, privacy, responsible AI, data governance, and secure software development practices for enterprise AI applications.

• Excellent problem-solving, analytical, debugging, and troubleshooting skills, with the ability to diagnose issues across application, AI model, agent, data, retrieval, and cloud infrastructure layers.

• Strong communication and collaboration skills, with the ability to explain complex AI and software architecture decisions to technical and non-technical stakeholders.

• High energy, a positive attitude, and a strong sense of ownership for delivering reliable and high-quality products.

Below is the list of preferred experience:

• Experience developing enterprise applications using Java or Go.

• Experience with multiple agentic AI frameworks, particularly Semantic Kernel, LangGraph, and LangChain.

• Experience implementing Model Context Protocol-based integrations or similar standards for connecting AI agents with tools, data sources, and enterprise systems.

• Experience with vector databases or vector-enabled data platforms such as Amazon OpenSearch Service, PostgreSQL with pgvector, Pinecone, Weaviate, Milvus, or equivalent technologies.

• Experience with orchestration and deployment platforms such as Kubernetes, Amazon ECS, Amazon EKS, Google Kubernetes Engine, or serverless compute services.

• Experience with model and experiment tracking, prompt management, evaluation platforms, tracing, and LLM observability tools.

• Experience designing event-driven or asynchronous systems using messaging and streaming technologies such as Kafka, RabbitMQ, Amazon SQS, Amazon SNS, or Google Pub/Sub.

• Experience working with relational and NoSQL databases, including PostgreSQL, MySQL, MongoDB, DynamoDB, or equivalent technologies.

• Familiarity with responsible AI practices covering fairness, transparency, explainability, privacy, human oversight, and risk management.

• Experience improving AI application performance through semantic caching, request batching, streaming, model routing, fallback strategies, and inference cost optimization.

• Experience with infrastructure-as-code technologies such as Terraform, AWS CloudFormation, AWS CDK, or equivalent tools.

• Experience working in a SaaS product environment with multi-tenant architecture, enterprise identity, authorisation, auditability, and data-isolation requirements.

Key Responsibilities

• Ensure the timely delivery of secure, reliable, and high-quality AI software that meets business requirements and maintains predictable performance in production environments.

• Design and develop cloud-native AI applications, agentic workflows, and reusable AI platform capabilities using Python, REST APIs, Large Language Models, and cloud services.

• Architect agent-based solutions using Semantic Kernel, LangGraph, LangChain, or related frameworks, selecting appropriate orchestration, state, memory, context, and multi-agent patterns.

• Design and implement Retrieval-Augmented Generation pipelines covering ingestion, document processing, chunking, embeddings, vector indexing, retrieval, re-ranking, grounding, citation, and response generation.

• Develop human-in-the-loop workflows that support review, approval, intervention, feedback, exception handling, and escalation for business-critical AI processes.

• Establish evaluation strategies and measurable quality criteria for agents, prompts, models, tools, and RAG pipelines, and integrate automated evaluations into the engineering lifecycle.

• Implement guardrails and security controls to reduce risks associated with prompt injection, sensitive data exposure, hallucinations, unauthorized tool usage, unsafe responses, and excessive agent autonomy.

• Optimize context usage, token consumption, latency, throughput, retrieval quality, model selection, and infrastructure cost while maintaining solution quality and reliability.

• Build and maintain REST APIs, asynchronous services, tool integrations, structured outputs, JSON schemas, and connectors required by AI agents and enterprise applications.

• Implement production monitoring and observability for AI applications, including application metrics, distributed traces, model interactions, agent execution paths, token usage, cost, evaluation outcomes, and safety events.

• Develop and maintain CI/CD pipelines using GitHub Actions or equivalent platforms, incorporating automated testing, AI evaluations, dependency scanning, container scanning, policy checks, and deployment validation.

• Deploy and operate AI services on AWS or GCP using appropriate containerized, managed, or serverless deployment patterns.

• Provide technical leadership and mentorship to engineers, guiding architecture, implementation, testing, debugging, secure development, and operational best practices.

• Conduct thorough design and code reviews, ensuring that implementations meet software quality, security, testability, observability, maintainability, and responsible AI standards.

• Collaborate closely with engineering teams, product managers, architects, security teams, data specialists, and stakeholders to define technical solutions, delivery plans, success criteria, and operational requirements.

• Evaluate AI models, frameworks, tools, and cloud services through structured prototypes and technical assessments, and recommend technologies based on quality, security, scalability, operability, and cost.

• Troubleshoot complex production issues across agents, prompts, models, tool integrations, RAG pipelines, vector indexes, APIs, containers, and cloud infrastructure.

• Communicate technical decisions, progress, risks, trade-offs, and recommended solutions clearly to technical and non-technical stakeholders.

• Stay current with developments in Generative AI, agentic frameworks, Large Language Models, cloud AI services, evaluation methodologies, and responsible AI, applying relevant advances to improve product capabilities and engineering practices.