Free · takes a minute · your resume fills your profile
CoreHRx never asks candidates for money. If anyone does, .
Sign in to see your fit: the skills, experience and location this job asks for, checked against your profile.
Location: Chennai; Pune, India
We are looking for a highly skilled Senior Data Engineer to contribute to the engineering, modernization, and continuous evolution of data processing platform on Databricks on AWS. While supporting the transition from the legacy Cloudera Hadoop platform to Databricks on AWS, this role will continue to play a key part in enhancing performance, simplifying pipelines, and delivering new capabilities on the Databricks platform over the long term.
The ideal candidate is a strong hands‑on Spark engineer with solid design experience, capable of contributing to architectural decisions while leading complex implementation and optimization efforts.
Responsibilities:
Refactor and modernize existing Spark pipelines to Databricks native architectures
Eliminate legacy Hadoop dependencies and adopt cloud native AWS patterns
Enhance and extend existing processing logic using optimized Spark (ScalaSpark) on Databricks
Build and optimize solutions using Databricks features, including Delta Lake, Databricks Workflows for orchestration and Auto scaling and job clusters
Contribute to low and mid level architecture and design
Translate high level architecture into detailed technical designs
Define data models, pipeline patterns, and reusable components
Ensure solutions are scalable, maintainable, and production ready
Analyze, improve Spark job performance and simplify complex or over engineered pipelines into standardized, efficient patterns
Follow and contribute to Databricks and Spark engineering standards
Write clean, modular, and testable code
Contribute to shared frameworks, reusable libraries, and quality standards
Work closely with senior architects, platform teams, and DevOps engineers
Provide technical inputs, troubleshooting support, and implementation guidance
Participate in design discussions and technical decision making
Develop unit, integration, and data validation tests
Support production releases and post deployment validation
Qualifications:
Core Technical Skills:
8+ years in data engineering or distributed systems
Strong expertise in Apache Spark (ScalaSpark/ JavaSpark / PySpark), Databricks on AWS, and Delta Lake
Experience on SQL
Experience with AWS services and large‑scale distributed data processing
Modernization & Optimization Experience
Experience modernizing or refactoring legacy data platforms into cloud‑based architectures
Strong background in Spark performance tuning and large‑scale batch optimization
Design Capability
Ability to translate architecture into implementable designs
Understanding of data modeling and pipeline orchestration patterns
Behavioral Competencies
Strong problem‑solving mindset for complex distributed systems
Comfortable working in time‑bound, high‑impact environments
Proactive, accountable, and collaborative
Clear communication skills across global teams
Education:
Bachelor’s degree/University degree and equivalent experience
Job Family Group: Technology ------------------------------------------------------
Job Family: Applications Development ------------------------------------------------------
Time Type: Full time ------------------------------------------------------
Most Relevant Skills Please see the requirements listed above. ------------------------------------------------------
Other Relevant Skills For complementary skills, please see above and/or contact the recruiter. ------------------------------------------------------
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.
If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi .
View Citi’s EEO Policy Statement and the Know Your Rights poster.
Job data: Headcount Labs (headcountlabs.com), ODbL 1.0. The job description belongs to the employer. Location data © OpenStreetMap contributors and Overture Maps.
Most IT Jobs posts in Pune ask for 3–6 years of experience (44% of 1200 that say).
From live job posts on CoreHRx, updated 15 Nov 2026. Skill figures count posts that list their skills; salary figures only posts that show a salary.
The post does not state the pay. As an estimate, comparable posts on CoreHRx advertise ₹15L – ₹28L a year (typical for IT Jobs in Pune, 48 posts).
The post asks for 8 Yrs experience.
In Pune, Chennai.
It is a full time job.
Interview preparation · 60 questions
This page is for Indian job seekers preparing for cloud engineer, DevOps, backend and solutions architect roles that list AWS as a skill, whether you are a fresher with a first certification or an experienced engineer moving into a senior role. It covers 60 questions in three levels: core services and concepts, then design and configuration choices for one to three years of experience, and finally scenario, security, cost and troubleshooting questions asked in senior rounds. Read each answer, explain it aloud in your own words, and try the CLI commands in a practice account with a billing alert set.
Questions you may be asked
Create a free profile and we email you new IT jobs in Pune, at most once a week.
