Sign in to see your fit: the skills, experience and location this job asks for, checked against your profile.
Job description
Location: Bangalore, Karnataka, India Department: AI & Data Engineering
Lead Data Engineer
Primary Skills
GCP, Big Query, Python, Airflow, SQL, DBT
Job requirements
About the Role
We are seeking a Senior Data Engineer with deep expertise in Google Cloud Platform (GCP) and BigQuery to lead cloud modernization initiatives, develop scalable data pipelines, and enable real-time data processing for enterprise-level systems. This is a high-impact role focused on driving the transformation of legacy infrastructure into a robust, cloud-native data ecosystem.
Key Responsibilities
1. Data Migration & Cloud Modernization
Analyze legacy on-premises and hybrid cloud data warehouse environments (e.g., SQL Server).
Lead the migration of large-scale datasets to Google BigQuery.
Design and implement data migration strategies ensuring data quality, integrity, and performance.
2. Data Integration & Streaming
Integrate data from various structured and unstructured sources, including APIs, relational databases, and IoT devices.
Build real-time streaming pipelines for large-scale ingestion and processing of IoT and telemetry data.
3. ETL / Data Pipeline Development
Modernize and refactor legacy SSIS packages into cloud-native ETL pipelines.
Develop scalable, reliable workflows using Apache Airflow, Python, Spark, and GCP-native tools.
Ensure high-performance data transformation and loading into BigQuery for analytical use cases.
4. Programming & Query Optimization
Write and optimize complex SQL queries, stored procedures, and scheduled jobs within BigQuery.
Develop modular, reusable transformation scripts using Python, Java, Spark, and SQL.
Continuously monitor and optimize query performance and cost efficiency in the cloud data environment.
Required Skills & Experience
5+ years in Data Engineering with a strong focus on cloud and big data technologies.
Minimum 2+ years of hands-on experience with GCP, specifically BigQuery.
Proven experience migrating on-premise data systems to the cloud.
Strong development experience with Apache Airflow, Python, and Apache Spark.
Expertise in streaming data ingestion, particularly in IoT or sensor data environments.
Strong SQL development skills; experience with BigQuery performance tuning.
Solid understanding of cloud architecture, data modeling, and data warehouse design.
Familiarity with Git and CI/CD practices for managing data pipelines.
Preferred Qualifications
GCP Professional Data Engineer certification.
Experience with modern data stack tools like dbt, Kafka, or Terraform.
Exposure to ML pipelines, analytics engineering, or DataOps/DevOps methodologies.
Why Join Us?
Work with cutting-edge technologies in a fast-paced, collaborative environment.
Most IT Jobs posts in Bangalore ask for 3–6 years of experience (41% of 3732 that say).
Hiring most for IT Jobs in Bangalore
Bosch Group165 open jobs
Trigent Software Private Limited157 open jobs
Brillio115 open jobs
Jobs asking for Airflow in Bangalore also ask for
Python77%
SQL61%
AWS56%
CI/CD48%
Data Engineering46%
From live job posts on CoreHRx, updated 7 Nov 2026. Skill figures count posts that list their skills; salary figures only posts that show a salary.
Questions about this job
What is the salary for this Lead Data Engineer job?
The post does not state the pay. As an estimate, comparable posts on CoreHRx advertise ₹15L – ₹25L a year (typical for IT Jobs in Bangalore, 286 posts).
This page helps Indian job seekers prepare for Python interviews for backend, data engineering and data analyst roles, whether you are a fresher facing your first technical round or an experienced developer aiming for a senior position. It has 60 questions in three levels, easy, medium and hard, each with a concise sample answer and, where it helps, runnable Python 3.12 code. Topics include data structures, OOP, generators, decorators, the GIL and concurrency, type hints, testing and performance. Answer each question aloud in your own words first, then compare with the sample answer and run the code to check your understanding.
Questions you may be asked
What is the difference between mutable and immutable types in Python?
What is the difference between a list and a tuple?
