Spark Developer

Job Description

About the Business & Position OverviewGodrej Infotech Ltd is a software services and consulting company headquartered in Mumbai, India. Formerly an IT division of Godrej & Boyce Mfg. Co. Ltd., which started operations in 1971, Godrej Infotech Ltd, commenced its operations as a separate corporate entity on April 1, 1999.Job ProfileKey ResponsibilitiesWe are looking for a Spark developer who knows how to fully exploit the potential of Spark cluster. You will clean, transform, and analyse vast amounts of raw data from various systems using Spark to provide ready-to-use data to our feature developers and business analysts. This involves both ad-hoc requests as well as data pipelines that are embedded in our production environment.Job Description- Create Scala/Spark jobs for data transformation and aggregation - Write, debug and performance tune Spark queries and related objects - Produce unit tests for Spark transformations and helper methods - Write Scaladoc-style documentation with all code - Design data processing pipelinesQualification DetailsEssential Qualification: - Bachelors degree in Computer Science or a related discipline- 3 - 5 years of experience in a medium complex data warehouse environment, hands-on Data warehousing project experience and experience in full life cycle Business Intelligence development, implementation, tuning and supportPreferred Qualification: -Experience DetailsEssential Experience: - Minimum Bachelor-s degree with 5+ years of experience in designing, building, and maintaining AWS data components- Expertise in Spark programming. Comfortable with working in various Spark API models- 3+ years of experience in data component configuration, related roles and access setup- Expertise in Spark query tuning and performance optimization- Programming: Spark, Python, JavaScript , SQL - Unix Utilities: Putty- AWS Skills including but not limited to S3, DMS, Redshift, EC2, VPC, Lambda, Delta Lake, CloudWatch etc.- Bigdata: Databricks, Spark, Glue and Athena- RDBMS & SQL Server experience required- Experience working in the SCRUM Environment.Preferred Experience: -Special SkillEssential : -Preferred : -, About the Business & Position OverviewGodrej Infotech Ltd is a software services and consulting company headquartered in Mumbai, India. Formerly an IT division of Godrej & Boyce Mfg. Co. Ltd., which started operations in 1971, Godrej Infotech Ltd, commenced its operations as a separate corporate entity on April 1, 1999.Job ProfileKey ResponsibilitiesWe are looking for a Spark developer who knows how to fully exploit the potential of Spark cluster. You will clean, transform, and analyse vast amounts of raw data from various systems using Spark to provide ready-to-use data to our feature developers and business analysts. This involves both ad-hoc requests as well as data pipelines that are embedded in our production environment.Job Description- Create Scala/Spark jobs for data transformation and aggregation - Write, debug and performance tune Spark queries and related objects - Produce unit tests for Spark transformations and helper methods - Write Scaladoc-style documentation with all code - Design data processing pipelinesQualification DetailsEssential Qualification: - Bachelors degree in Computer Science or a related discipline- 3 - 5 years of experience in a medium complex data warehouse environment, hands-on Data warehousing project experience and experience in full life cycle Business Intelligence development, implementation, tuning and supportPreferred Qualification: -Experience DetailsEssential Experience: - Minimum Bachelor-s degree with 5+ years of experience in designing, building, and maintaining AWS data components- Expertise in Spark programming. Comfortable with working in various Spark API models- 3+ years of experience in data component configuration, related roles and access setup- Expertise in Spark query tuning and performance optimization- Programming: Spark, Python, JavaScript , SQL - Unix Utilities: Putty- AWS Skills including but not limited to S3, DMS, Redshift, EC2, VPC, Lambda, Delta Lake, CloudWatch etc.- Bigdata: Databricks, Spark, Glue and Athena- RDBMS & SQL Server experience required- Experience working in the SCRUM Environment.Preferred Experience: -Special SkillEssential : -Preferred : -,

Keyskills :
sql serverlife cyclequery tuningdata processingcomputer sciencedata warehousingsoftware servicesdata transformationbusiness intelligencecompatibility testingproject administrationsqlapiaws