Senior Associate Pyspark Data Engineer Data Analytics Advisory

Job Description

Role Overview:You will be part of the data and analytics engineering team at PwC, where you will leverage advanced technologies and techniques to design and develop robust data solutions for clients. Your crucial role will involve transforming raw data into actionable insights, enabling informed decision-making and driving business growth. Additionally, you will focus on automating data internally using automation tools or software to streamline data-related processes within the organization.Key Responsibilities:- Develop, maintain, and optimize PySpark applications for large-scale data processing- Build and manage ETL/ELT pipelines using Spark Hive and distributed file systems- Process structured, semi-structured, and unstructured data from multiple sources- Optimize Spark jobs for performance, scalability, and cost efficiency- Collaborate with data engineers, analysts, and data scientists to meet business requirements- Implement data quality checks, validations, and error handling- Monitor and troubleshoot Spark jobs in production environments- Follow best practices for coding, testing, and documentationQualifications Required:- Mandatory skill set: Pyspark- Preferred skill set: Any cloud- Years of experience required: 5 - 8- Education qualification: B.Tech/MBA/MCAAdditional Details:At PwC, you will be part of a vibrant community that leads with trust and creates distinctive outcomes for clients and communities. The purpose-led and values-driven work, powered by technology in an innovative environment, will enable you to make a tangible impact in the real world. PwC believes in providing equal employment opportunities without any discrimination and strives to create an inclusive environment where individuals can bring their true selves and contribute to personal and firm growth. Role Overview:You will be part of the data and analytics engineering team at PwC, where you will leverage advanced technologies and techniques to design and develop robust data solutions for clients. Your crucial role will involve transforming raw data into actionable insights, enabling informed decision-making and driving business growth. Additionally, you will focus on automating data internally using automation tools or software to streamline data-related processes within the organization.Key Responsibilities:- Develop, maintain, and optimize PySpark applications for large-scale data processing- Build and manage ETL/ELT pipelines using Spark Hive and distributed file systems- Process structured, semi-structured, and unstructured data from multiple sources- Optimize Spark jobs for performance, scalability, and cost efficiency- Collaborate with data engineers, analysts, and data scientists to meet business requirements- Implement data quality checks, validations, and error handling- Monitor and troubleshoot Spark jobs in production environments- Follow best practices for coding, testing, and documentationQualifications Required:- Mandatory skill set: Pyspark- Preferred skill set: Any cloud- Years of experience required: 5 - 8- Education qualification: B.Tech/MBA/MCAAdditional Details:At PwC, you will be part of a vibrant community that leads with trust and creates distinctive outcomes for clients and communities. The purpose-led and values-driven work, powered by technology in an innovative environment, will enable you to make a tangible impact in the real world. PwC believes in providing equal employment opportunities without any discrimination and strives to create an inclusive environment where individuals can bring their true selves and contribute to personal and firm growth.

Keyskills :
Agile MethodologyBusiness AnalysisData AnalyticsData ArchitectureBusiness TransformationContinuous Process ImprovementCommunicationCreativityPysparkMicrosoft Azure AI foundryAlteryx Automation PlatformDataDriven InsightsAutomation Programming