Data Engineer

Job Description

Some of the things that you can look forward to doing once you are here are:

• Collaborate with security and other technology teams to assist in the definition of security and compliance standards as we develop and deploy new data platforms.
• Provide best practice security guidance to data platform and analytics teams.
• Work with Cloud Engineering teams to design and maintain an appropriate set of shared Infrastructure components, with an emphasis on ensuring the solution is secure and complaint with all security standards.
• Developing code/configuration to enable enforcement of security policies in a cloud environment.
• Collaborate and provide guidance on designing solutions that deliver cross-platform compatibility in a secure manner.
• Active involvement in deployment and automation of solutions using reusable clean code and one-click deployment methodologies.
• Provide expertise in troubleshooting and resolving security related incidents, with a focus on ensuring long term resolutions are made to the code base.
• Work on strengthening the AWS VPC security and implementing the latest features by rolling upgrades ensuring higher availability.
• Writing libraries and plugins for the open-source products and tools used by the team using Python, Scala, Java, etc.
• Be a champion of the overall strategy for data governance, security, privacy, quality and retention that will satisfy business policies and requirements.
• Gate keep development process and quality by ensuring that best security practices are followed.
• Coach, train and mentor to improve the maturity and value of the cloud security practices within the team.

Relevant experience and exposure

• Education: Bachelor-Master s degree
• Proficient in English language.
• Should have been working experience in implementing cloud security for data products.
• Should have worked on AWS Cloud Environment in depth and AWS security services.
• A must-have experience in implementation of solutions using Cloud Data warehouse / ETL / AWS Data Pipeline / EMR capability engineering.
• Knowledge of all phases of software development including requirements analysis, design, coding, testing, debugging, implementation, and support.
• Hands-on experience in Scala, Pyspark & Python programming languages.
• Deep understanding of Apache Spark RDDs and building data frames. Create Scala/Spark jobs for data transformation and aggregation.
• Experience in building distributed environments using any of Kafka, Spark, Hive, Presto, Hadoop etc.
• Experience working with various file formats like Parquet, Avro, ORC for large volumes of data. Hands on experience on AWS cloud services and designing architecture for sizing / scalability and performance.
• Knowledge of Data Security, Data Masking and Data Encryption techniques.
• Demonstrated experience with AWS Security Services (AWS Shield, AWS Trusted Advisor, Macie, IAM, AWS WAF, CloudTrails).
• Experienced in creating multiple VPC s and public, private subnets as per requirement and distributed them as groups into various availability zones of the VPC.
• Used security groups, network ACL s, internet gateways and route tables to ensure a secure zone for organization in AWS public cloud.
• Have good knowledge on DMS (Data Migrations) and can perform tasks on migrations.
• Understanding of holistic enterprise architecture and interplay of analytic components with enterprise foundational components (data architecture/governance, security etc.).
• Experience in and understanding of data and information management, especially in Big Data trends * Experience integrating 3rd party KMS, HSM with cloud data services for building secure data solutions.
• Hands-on experience of working on setting up security practices for containers and Kubernetes.
• Experience with private and public cloud architectures, pros/cons, and migration considerations.
• Should have implemented centralized logging and monitoring of security events in AWS.
• Hands-on experience of Linux and Windows Operating System.
• Experience with creating secure CI/CD pipelines, including both integration of security tools/scans, as well as plugins that detect sensitive information in code.
• Experience with integration of security into automated processes.

This position is for Pune location with Tavisca, a cxLoyalty Technology Platform (Division of JP Morgan Chase & Co.),

Keyskills :
sqljavadata warehousinginformaticapythonbig datadata maskingapache sparkdata securitydata servicescloud securitydata governancesecurity servicesworking experience