₹11L – ₹24L a year estimatetypical for IT Jobs in India, 940 posts
Posted 19 days ago
CoreHRx never asks candidates for money. If anyone does, .
Role at a glance
You will
This is a hands-on engineering role focused on designing, building, and optimising high-throughput data pipelines using Azure Databricks…
You will be responsible for delivering scalable ETL/ELT pipelines, supporting real-time and near-real-time integrations, and developing…
Working closely with data, engineering and business stakeholders, you will also help operationalise workloads through CI/CD, scheduling…
Must-have skills
AWSCI/CDConfluenceData WarehousingDatabricksELT
How well do your skills match this job?
Sign in to see your fit: the skills, experience and location this job asks for, checked against your profile.
Job description
Company Description Showtime Consulting is a leading provider of Shielded Cloud and Digital Solutions across Australia and New Zealand. We specialise in delivering secure, enterprise-scale technology solutions across cloud, infrastructure, cybersecurity, business applications, and digital transformation programs.
We partner with government and enterprise organisations to build high-performing technology teams that deliver scalable, efficient, and future-ready business solutions.
The Role We are seeking an experienced Data Engineer to join a large-scale enterprise data environment in Sydney.
This is a hands-on engineering role focused on designing, building, and optimising high-throughput data pipelines using Azure Databricks, Spark (Scala), SQL and Python.
You will be responsible for delivering scalable ETL/ELT pipelines, supporting real-time and near-real-time integrations, and developing modern data solutions using Medallion Architecture and Delta Lake best practices.
Working closely with data, engineering and business stakeholders, you will also help operationalise workloads through CI/CD, scheduling, secrets management and observability.
Key Responsibilities
Design, build and optimise high-throughput data pipelines using Azure Databricks.
Develop scalable ETL/ELT pipelines using Spark (Scala), SQL and Python.
Implement data solutions following Medallion Architecture across Bronze, Silver and Gold layers.
Apply Delta Lake principles and best practices across data workloads.
Design and develop real-time and near-real-time data integrations.
Develop data processing solutions using Spark Streaming and Apache/Confluent Kafka.
Build and manage Databricks jobs, workflows and scheduling processes.
Support CI/CD pipelines using Git and tools such as Azure DevOps.
Implement appropriate secrets management and observability practices.
Work across cloud platforms including Azure and AWS.
Support large-scale data processing using technologies including Spark, Hive, Impala and Sqoop.
Collaborate with technical, business and delivery stakeholders to deliver effective data solutions.
What You'll Need
Strong hands-on experience with Azure Databricks.
Strong commercial experience with Spark using Scala.
Strong SQL skills.
Strong Python skills.
Experience designing and building scalable ETL/ELT data pipelines.
Strong understanding of modern data warehousing concepts.
Experience with Medallion Architecture and Delta Lake.
Proven experience designing real-time and near-real-time integrations.
Experience with cloud platforms, particularly Azure and/or AWS.
Experience with big data technologies such as Hive, Impala, Sqoop and Spark.
Experience with real-time data processing frameworks such as Spark Streaming and Apache/Confluent Kafka.
Understanding of CI/CD practices and associated tools.
Excellent communication and stakeholder management skills.
Preferred Skills
Experience working with both Azure and AWS environments.
Experience with Master Data Management solutions.
Exposure to large-scale enterprise data platforms.
Experience with tools including GitHub, Jenkins, Jira and Confluence.
Strong understanding of data warehousing concepts.
Experience operationalising Databricks workloads.
Understanding of automation, monitoring and observability within data environments.
This page is for Indian job seekers preparing for cloud engineer, DevOps, backend and solutions architect roles that list AWS as a skill, whether you are a fresher with a first certification or an experienced engineer moving into a senior role. It covers 60 questions in three levels: core services and concepts, then design and configuration choices for one to three years of experience, and finally scenario, security, cost and troubleshooting questions asked in senior rounds. Read each answer, explain it aloud in your own words, and try the CLI commands in a practice account with a billing alert set.
Questions you may be asked
What is the difference between an AWS Region and an Availability Zone?
Explain the AWS shared responsibility model.
What is Amazon EC2, and how do you read an instance type name like m7g.large?
What is Amazon S3, and what are its main storage classes?
