Big Data Engineer

Job Description

Experience in Big Data projectsSkills : Must be highly skilled in building scalable solutions in new data technologies including, Hadoop/Spark, Presto/Five, Columnar data design, Cloud based architectures, and Python/Scala ETL. Experience would ideally have been gained across a number of previous engagements and a variety of industries. Experience in implementing, or building systems to support, Machine Learning algorithms, Advanced Analytics in R, and data visualization.Key Responsibilities: To develop and maintain a data analysis platform for Lebara Provide typical DBA support for the platform Develop and maintain Python/Scala based ETL Data Ingestion Responsible for access, availability and performance of a Redshift database solution and Hadoop based Data Lake. (or similar solutions) Perform proactive capacity management and scaling Responsible for Backup and Recovery Responsible for security role implementation Work directly with AWS (or similar partner) as necessary for support and issue resolution Create load routines for data ingestion and aggregation Ensure quality design of data load routines to ensure accuracy, efficiency and extensibility Provide input into Data Schema, Reporting, and Analysis This role will have a high impact on the organisation building a new data platform enabling live, complex and predictive analytics to be delivered driving the business directly This role will impact the long term strategic decision making of the organisation directly additionally the platform this role will develop will directly drive the development and growth of new products. This role will require previous experience in big data technologies and will leverage that knowledge to lead and persuade other technical experts and business owners around strategic decisions related to the new data platform direction. The role will communicate with fellow technical experts; business owners and team members of the Data & Insights team Partner with other platform technologists within Lebara to ensure cross-platform integration is built appropriately Actively understand the needs of all data developers, analysts and technology partners to ensure that solutions meet requirements This role will work within a small team and may be required to lead junior engineers on a daily basis Become an expert at Lebara data and act a spokesperson for correct use and interpretation of the data Provide coaching and mentoring for the team, on database and SQL functionality Bring your industry knowledge of the best in data technologies to Lebara to ensure the correct tools and techniques are in us to meet requirements This role will be of middle management level prioritising and directing junior engineering resource and prirotising and planning backlogs of work related to the new data platform This role will require experience of managing cloud storage solutions effectively and the associated costs Key Skills: Experience in Big Data projectsSkills : Must be highly skilled in building scalable solutions in new data technologies including, Hadoop/Spark, Presto/Five, Columnar data design, Cloud based architectures, and Python/Scala ETL. Experience would ideally have been gained across a number of previous engagements and a variety of industries. Experience in implementing, or building systems to support, Machine Learning algorithms, Advanced Analytics in R, and data visualization.Key Responsibilities: To develop and maintain a data analysis platform for Lebara Provide typical DBA support for the platform Develop and maintain Python/Scala based ETL Data Ingestion Responsible for access, availability and performance of a Redshift database solution and Hadoop based Data Lake. (or similar solutions) Perform proactive capacity management and scaling Responsible for Backup and Recovery Responsible for security role implementation Work directly with AWS (or similar partner) as necessary for support and issue resolution Create load routines for data ingestion and aggregation Ensure quality design of data load routines to ensure accuracy, efficiency and extensibility Provide input into Data Schema, Reporting, and Analysis This role will have a high impact on the organisation building a new data platform enabling live, complex and predictive analytics to be delivered driving the business directly This role will impact the long term strategic decision making of the organisation directly additionally the platform this role will develop will directly drive the development and growth of new products. This role will require previous experience in big data technologies and will leverage that knowledge to lead and persuade other technical experts and business owners around strategic decisions related to the new data platform direction. The role will communicate with fellow technical experts; business owners and team members of the Data & Insights team Partner with other platform technologists within Lebara to ensure cross-platform integration is built appropriately Actively understand the needs of all data developers, analysts and technology partners to ensure that solutions meet requirements This role will work within a small team and may be required to lead junior engineers on a daily basis Become an expert at Lebara data and act a spokesperson for correct use and interpretation of the data Provide coaching and mentoring for the team, on database and SQL functionality Bring your industry knowledge of the best in data technologies to Lebara to ensure the correct tools and techniques are in us to meet requirements This role will be of middle management level prioritising and directing junior engineering resource and prirotising and planning backlogs of work related to the new data platform This role will require experience of managing cloud storage solutions effectively and the associated costs,

Keyskills :
javahivebig datahadoopapache webserverdata analysiscloud storagemachine learningmiddle managementstorage solutionsadvanced analyticscapacity management