Sr. Data Engineer - Pyspark AWS

4.00 to 9.00 Years Hyderabad, Pune, Mumbai City, Nagpur 04 May, 2022

Job Location	Hyderabad, Pune, Mumbai City, Nagpur
Education	Not Mentioned
Salary	Rs 12 - 24 Lakh/Yr
Industry	IT - Software
Functional Area	General / Other Software
EmploymentType	Full-time

Design and develop scalable data ingestion frameworks to transform a variety of datasets.
Define and Design Source target mappings.
Build reliable data pipelines to clean, aggregate, and transform large volumes of data from multiple sources.
Identifying, designing, and implementing internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes
Working with stakeholders including data, design, product, and executive teams, and assisting them with data-related technical issues

Hands-on experience from AWS cloud technologies like S3, AWS glue, Glue Catalog, Athena, AWS Lambda, AWS DMS, Redshift, RDS, Pyspark, and Kinesis.
Experience within Java or Python, SQL.
Practical experience with Python, pyspark, and SQL including the ability to write, analyze and debug processes supporting data transformation, data structures, metadata, dependency, and workload management.
Familiarity with Batch and Streaming pipelines.
Familiarity with the AWS ecosystem specifically Ec2, ECS, VPC, and Security Groups.
Communication skills, especially explaining technical concepts to non-technical business leaders.
Able to schedule and automate pipeline. Good to have DevOps knowledge.

Keyskills :
pythonawssqlathenaaws lambdaaws glue

APPLY NOW

Sr. Data Engineer - Pyspark AWS Related Jobs