Lead Cloud Data Programmer

Job Description

You will be responsible for designing, developing, and maintaining data pipelines on AWS or Azure to ingest, process, and transform data from various sources. You will optimize data storage and processing using cloud-native services and technologies such as AWS S3, AWS Glue, Azure Data Lake Storage, Azure Data Factory, etc. Additionally, you will implement and manage data warehouse solutions using Snowflake, including schema design, query optimization, and performance tuning. Collaboration with cross-functional teams to understand data requirements and translate them into scalable and efficient technical solutions will be part of your role. Ensuring data quality and integrity by implementing data validation, cleansing, and transformation processes will also be crucial. Developing and maintaining ETL processes for data integration and migration between different data sources and platforms will be another key responsibility. Implementing and enforcing data governance and security practices, including access control, encryption, and compliance with regulations, will also be part of your duties. Collaborating with data scientists and analysts to support their data needs and enable advanced analytics and machine learning initiatives will also be part of your responsibilities. Monitoring and troubleshooting data pipelines and systems to identify and resolve performance issues or data inconsistencies will be essential. Staying updated with the latest advancements in cloud technologies, data engineering best practices, and emerging trends in the industry will be important for this role.- Bachelors or Masters degree in Computer Science, Information Systems, or a related field- Minimum of 8 years of experience as a Data Engineer, with a focus on AWS/Azure, and Snowflake- Database experience in Azure SQL, Synapses, Redshift & PostgreSQL- Strong proficiency in data modeling, ETL development, and data integration- Expertise in cloud platforms such as AWS or Azure, including hands-on experience with data storage and processing services- In-depth knowledge of Snowflake, including schema design, SQL optimization, and performance tuning- Experience with scripting languages such as Python or Pyspark- Familiarity with data governance principles and security best practices- Strong problem-solving skills and ability to work independently in a fast-paced environment- Excellent communication and interpersonal skills to collaborate effectively with cross-functional teams and stakeholdersEPAM Systems, Inc. has been a global digital transformation services provider since 1993. With a focus on leading the industry in digital and physical product development and digital platform engineering services, EPAMs globally deployed hybrid teams help make the future real for clients and communities around the world. Ranked among the top 15 companies in Information Technology Services on the Fortune 1000, EPAM is dedicated to powering better enterprise, education, and health platforms that connect people, optimize experiences, and improve peoples lives. EPAM offers a variety of benefits to its employees, including insurance coverage, paid leaves, financial assistance for medical crises, and certification trainings for technical and soft skills. You will be responsible for designing, developing, and maintaining data pipelines on AWS or Azure to ingest, process, and transform data from various sources. You will optimize data storage and processing using cloud-native services and technologies such as AWS S3, AWS Glue, Azure Data Lake Storage, Azure Data Factory, etc. Additionally, you will implement and manage data warehouse solutions using Snowflake, including schema design, query optimization, and performance tuning. Collaboration with cross-functional teams to understand data requirements and translate them into scalable and efficient technical solutions will be part of your role. Ensuring data quality and integrity by implementing data validation, cleansing, and transformation processes will also be crucial. Developing and maintaining ETL processes for data integration and migration between different data sources and platforms will be another key responsibility. Implementing and enforcing data governance and security practices, including access control, encryption, and compliance with regulations, will also be part of your duties. Collaborating with data scientists and analysts to support their data needs and enable advanced analytics and machine learning initiatives will also be part of your responsibilities. Monitoring and troubleshooting data pipelines and systems to identify and resolve performance issues or data inconsistencies will be essential. Staying updated with the latest advancements in cloud technologies, data engineering best practices, and emerging trends in the industry will be important for this role.- Bachelors or Masters degree in Computer Science, Information Systems, or a related field- Minimum of 8 years

Keyskills :
AWSAzureSnowflakeSQLPythonETLData IntegrationData GovernancePysparkData Modelling