Senior Engineer (India Office)

We are seeking a Senior Data Engineer with AI/GenAI expertise to join the Data Platform (Foundry) team. This role will focus on building and optimizing enterprise-scale data pipelines while also enabling metadata-driven, AI-driven use cases, including copilots, LLM integrations, and intelligent data workflows using Databricks, Azure, ADF, Azure SQL, Astronomer/Airflow or Control M

Detailed Description:

• Design, build, and maintain data pipelines using Databricks, Spark, ADF, Advanced SQL/Python, and Azure services
• Develop and optimize bronze → silver → gold transformations
• Schedule and orchestrate via ADF, Airflow, Control M. Airflow experience would be a huge plus
• Enable AI/GenAI use cases across the platform, including:
  • Copilot integrations
  • LLM-based workflows
  • Intelligent data enrichment pipelines
• Work with Databricks GenAI capabilities .
• Integrate external AI platforms (e.g., Anthropic, Azure OpenAI) into enterprise data workflows
• Ensure data reliability, performance, and operational stability
• Troubleshoot and resolve production issues with strong SLAs
• Support ingestion using tools such as:
  • Fivetran / ADF / Auto Loader
• Implement data quality, monitoring, and reconciliation frameworks
• Ability to read/write code in SQL, Python, other
• Collaborate across:
  • Data Engineering teams
  • Data Platform engineering
  • Business stakeholders
• Contribute to platform modernization and AI adoption strategy
• Follow governance best practices:
  • Code quality
  • Deployment controls
  • Audit and compliance
• Required Skills & Experience
• 8–10+ years in Data Engineering
• Strong experience with:
  • Databricks (Delta Lake, Spark)
  • Azure Data Platform - ADF, Synapse
  • SQL - Azure SQL, SQL Server, Teradata, DB2, Oracle
• Hands-on experience with AI/GenAI technologies, including:
  • Building or integrating copilot-style experiences
  • Working with LLMs (e.g., Anthropic Claude, OpenAI models)
• Strong programming skills:
  • Python, SQL
• Orchestration using — Airflow or Astronomer, Control M. Preferably Airflow
• Experience building scalable ETL/ELT pipelines
• Familiarity with production support and incident management

#LI-KS1