Mid-Level Big Data Engineer_6 yrs_Hyderabad (Hybrid)

M&S Consulting | Mid-Level Big Data Engineer

Mid-Level Big Data Engineer
Experience: 6+ Years | Full-Time | Hyderabad (Hybrid)
About M&S Consulting
Founded in 2002, M&S Consulting is an enterprise digital transformation and IT advisory firm trusted by commercial
businesses, public sector agencies, and higher education institutions. M&S specializes in bridging complex technology gaps
through deep expertise across Enterprise Applications, Cloud Infrastructure, Data Analytics, and AI. With a culture rooted in
agility and innovation, M&S empowers team members to solve high-impact technical challenges.
Role Overview
We are looking for an experienced Big Data Engineer to design, develop, test, and support cloud-based big data applications on
AWS. The role involves building batch-processing solutions and API-based services, developing REST APIs and SOAP web services,
and improving the performance and reliability of data-processing workloads.
Key Responsibilities
Big Data Applications: Analyze, design, develop, implement, and test big data applications covering batch processing and API-based
workloads on AWS.
API & Web Services: Develop and maintain REST APIs and SOAP web services using Scala, Java, and related data-processing
technologies.
AWS Solutions: Build and support AWS-based data-processing applications and microservices using EMR, EC2, ECS, S3, Airflow, Step
Functions, and AWS API Gateway.
Performance Tuning: Monitor, troubleshoot, and tune Spark, Hadoop, EMR, Java, and Python workloads to improve performance and
reliability.
Linux & Scripting: Use Linux, shell scripting, and Python to support development, automation, troubleshooting, and operational activities.
Collaboration: Work with team members to deliver maintainable, reliable, and scalable data solutions.
Experience & Qualifications
Experience: 6+ years in analysis, coding, implementation, and testing of big data applications on AWS, including batch and API-based
solutions.
Programming: Hands-on experience with Scala, PySpark, and Java; working knowledge of Python.
APIs & Services: Experience developing REST APIs and SOAP web services.
AWS: Practical experience with EMR, EC2, ECS, S3, Airflow, Step Functions, and AWS API Gateway.
Big Data: Experience with Spark/Hadoop/EMR performance tuning; working knowledge of Hadoop, HDFS, MapReduce, and Cassandra.
Systems: Linux experience, including shell scripting.
M&S Consulting | Mid-Level Big Data Engineer
Core Skills Required
3 Must-Have Skills
Scala PySpark Java Python
Apache Spark Hadoop / HDFS MapReduce
Cassandra AWS EMR / EC2 / ECS S3
Airflow Step Functions API Gateway
REST APIs SOAP Linux Shell scripting
Performance tuning Microservices
H Good-to-Have Skills
DevOps CI/CD pipelines Maven Gradle
Jenkins Artifactory AWS CodeCommit
CloudFormation Git GitHub Bitbucket
GitLab IntelliJ IDEA PyCharm
Tools & Platforms
3 Platforms & Tools
AWS Amazon EMR Amazon EC2 Amazon ECS Amazon S3 Airflow Step Functions
AWS API Gateway Spark Hadoop Cassandra Git GitHub Bitbucket GitLab Maven
Gradle Jenkins Artifactory CodeCommit CloudFormation IntelliJ IDEA PyCharm
Preferred Qualifications
DevOps & Version Control: Familiarity with Git, GitHub, Bitbucket, GitLab, IntelliJ IDEA, or PyCharm.
CI/CD: Experience creating CI/CD pipelines using Maven, Gradle, Jenkins, Artifactory, AWS CodeCommit, or CloudFormation.
Role Details
Experience 6+ years
Location Hyderabad, India
Work Model Hybrid
Employment Type Full-Time