Principal Data Engineer-R01570250

Location: Hyderabad, Andhra Pradesh, India
Department: AI & Data Engineering

Senior Tech Specialist, Applications & Platforms

Job requirements

Experience Range: 7 to 10 years of experience in advanced AWS data engineering, including significant expertise in Master Data Management (MDM) with Stibo and designing large-scale data solutions Key Responsibilities:

• Design and architect scalable data pipelines and solutions using AWS native services to support complex business requirements
• Develop, implement, and optimize ETL processes leveraging AWS technologies such as EMR, Redshift, Athena, and DMS
• Configure and manage real-time data streaming and ingestion frameworks utilizing Kinesis and Amazon API Gateway
• Monitor, troubleshoot, and enhance data infrastructure performance using CloudWatch and Open Search
• Automate deployment and infrastructure provisioning with CloudFormation and manage schema migrations using SCT
• Lead the implementation and integration of Master Data Management (MDM) solutions using Stibo within enterprise data platforms
• Collaborate with cross-functional teams to integrate data solutions with enterprise applications and analytics platforms
• Ensure data security, quality, and compliance by implementing best practices and leveraging AWS security features
• Lead technical reviews, mentor junior engineers, and drive adoption of emerging data technologies within the organization

Required Skills:

• Advanced proficiency in AWS services including SNS, SQS, Amazon API Gateway, Athena, CloudFormation, CloudWatch, DMS, DynamoDB, EMR, Kinesis, Open Search, Redshift, SCT
• Expertise in designing and managing data pipelines and ETL workflows on AWS
• Strong experience with Spark using Scala for large-scale data processing
• Hands-on experience with Oozie workflow scheduling and orchestration
• Deep knowledge of real-time data streaming with AWS Kinesis
• Proven ability to configure and optimize Amazon Redshift for analytics workloads
• Experience with DynamoDB for NoSQL database solutions
• Ability to automate infrastructure and deployments using AWS CloudFormation
• Proficient in monitoring and logging with AWS CloudWatch
• Skill in managing data migration and schema transformation using AWS SCT and DMS
• Demonstrated expertise in implementing and integrating MDM solutions using Stibo

Preferred Skills:

• Experience integrating AWS data solutions with third-party analytics and visualization tools
• Proficiency in optimizing performance for large-scale distributed data systems
• Knowledge of emerging AWS data services and staying current with industry best practices
• Experience with Open Search for search and analytics use cases
• Background in mentoring and leading engineering teams in cloud data projects
• Familiarity with advanced data governance and data quality frameworks in MDM environments

Desired Qualifications:

• Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a closely related discipline
• AWS Certified Data Analytics – Specialty or AWS Certified Solutions Architect – Professional
• Certification in Apache Spark or relevant big data technologies (preferred)