Senior Data Engineer

Job Description

Role Overview:You will become a substantial part of the Data Platform team, providing engineering services across the Media Measurement department. Your responsibilities will include designing, developing, testing, and deploying big data applications and pipelines. It is crucial to ensure that your work and the teams work adhere to best practices for validity, efficiency, and maintainability. You will collaborate in a distributed cross-functional agile team of Data Scientists and Cloud Engineers to achieve shared goals. Taking ownership of technical implementations and architecture, along with your team, and aligning with communities of practices will be essential. Additionally, you will communicate with data producers and consumers to align on data exchange formats and interfaces, while also providing suggestions to the business in the Media Measurement context on utilizing existing and new data sources to enhance the product portfolio. Recommending ways to improve data reliability and quality continuously will also be part of your role.Key Responsibilities:- Design, develop, test, and deploy big data applications and pipelines- Ensure adherence to best practices for validity, efficiency, and maintainability- Collaborate in a distributed cross-functional agile team- Take ownership of technical implementations and architecture- Communicate with data producers and consumers to align on data exchange formats- Provide suggestions to enhance the existing product portfolio- Recommend improvements for data reliability and qualityQualifications:- 6 years of relevant experience in AWS Data Engineering- Solid hands-on experience with Python, Java, or Scala- Knowledge of developing data-intensive applications using Spark- Proficiency in writing SQL queries for relational databases- Ability to orchestrate data pipelines with Airflow or similar tools- Experience with AWS, Terraform, Data Lake & Lake Formation, Apache Iceberg- Experience in EMR- Familiarity with CI/CD tools like Gitlab & Github- Following best practices in software engineering such as pair programming, testing, clean code, and agile frameworksAdditional Company Details:Enjoy a flexible and rewarding work environment with peer-to-peer recognition platforms. You can recharge and revitalize with wellness plans designed for you and your family, and plan your future with financial wellness tools. Stay relevant and upskill yourself with career development opportunities.(Note: The additional company information and benefits have been included in the Additional Company Details section for your reference.) Role Overview:You will become a substantial part of the Data Platform team, providing engineering services across the Media Measurement department. Your responsibilities will include designing, developing, testing, and deploying big data applications and pipelines. It is crucial to ensure that your work and the teams work adhere to best practices for validity, efficiency, and maintainability. You will collaborate in a distributed cross-functional agile team of Data Scientists and Cloud Engineers to achieve shared goals. Taking ownership of technical implementations and architecture, along with your team, and aligning with communities of practices will be essential. Additionally, you will communicate with data producers and consumers to align on data exchange formats and interfaces, while also providing suggestions to the business in the Media Measurement context on utilizing existing and new data sources to enhance the product portfolio. Recommending ways to improve data reliability and quality continuously will also be part of your role.Key Responsibilities:- Design, develop, test, and deploy big data applications and pipelines- Ensure adherence to best practices for validity, efficiency, and maintainability- Collaborate in a distributed cross-functional agile team- Take ownership of technical implementations and architecture- Communicate with data producers and consumers to align on data exchange formats- Provide suggestions to enhance the existing product portfolio- Recommend improvements for data reliability and qualityQualifications:- 6 years of relevant experience in AWS Data Engineering- Solid hands-on experience with Python, Java, or Scala- Knowledge of developing data-intensive applications using Spark- Proficiency in writing SQL queries for relational databases- Ability to orchestrate data pipelines with Airflow or similar tools- Experience with AWS, Terraform, Data Lake & Lake Formation, Apache Iceberg- Experience in EMR- Familiarity with CI/CD tools like Gitlab & Github- Following best practices in software engineering such as pair programming, testing, clean code, and agile frameworksAdditional Company Details:Enjoy a flexible and rewarding work environment with peer-to-peer recognition platforms. You can recharge and revitalize with wellness plans designed for you and your family, and plan you

Keyskills :
PythonJavaScalaSparkSQLAirflowEMRGitlabGithubAWS Data EngineeringTerraformApache Iceberg