hireejobs
Register for Job
Hyderabad Jobs
Banglore Jobs
Chennai Jobs
Delhi Jobs
Ahmedabad Jobs
Mumbai Jobs
Pune Jobs
Vijayawada Jobs
Gurgaon Jobs
Noida Jobs
Oil & Gas Jobs
Banking Jobs
Construction Jobs
Top Management Jobs
IT - Software Jobs
Medical Healthcare Jobs
Purchase / Logistics Jobs
Sales
Ajax Jobs
Designing Jobs
ASP .NET Jobs
Java Jobs
MySQL Jobs
Sap hr Jobs
Software Testing Jobs
Html Jobs
IT Jobs
Logistics Jobs
Customer Service Jobs
Airport Jobs
Banking Jobs
Driver Jobs
Part Time Jobs
Civil Engineering Jobs
Accountant Jobs
Safety Officer Jobs
Nursing Jobs
Civil Engineering Jobs
Hospitality Jobs
Part Time Jobs
Security Jobs
Finance Jobs
Marketing Jobs
Shipping Jobs
Real Estate Jobs
Telecom Jobs

Lead Data engineer

8.00 to 12.00 Years   Chennai   16 Jan, 2025
Cynosure Corporate Solutions
Job LocationChennai
EducationNot Mentioned
SalaryNot Disclosed
IndustryIT Services & Consulting
Functional AreaIT Operations / EDP / MIS
EmploymentTypeFull-time

Job Description

    Key responsibilities:
    • Working with clients to understand their data.
    • Basedontheunderstanding you will be building the data structures and pipelines.
    • You will be working on the application from end to end collaborating with UI and other development teams.
    • You will be responsible for building the data pipelines to migrate and load the data into the HDFSeither on-prem or in the cloud.
    • Developing Data ingestion/process/integration pipelines effectively.
    • Creating Hive data structures,metadata and loading the data into data lakes / Big Data warehouse environments.
    • Optimized(Performance tuning) many data pipelines effectively to minimize cost.
    • Codeversioning control and git repository is up to date.
    • You will be responsible for building and maintaining CI/CD of the data pipelines.
    • You will be managing the unit testing of all data pipelines
    Skills & Experience:
    • Bachelors degree in computer science or related field.
    • Minimum of 5years working experience with Spark, Hadoop eco systems.
    • Minimum of 4years working experience on designing data streaming pipelines.
    • Should be an expertin either Python/Scala/Java.
    • Should have experience in Data Ingestion and Integration into data lake using hadoop ecosystem tools such as Sqoop, Spark, SQL, Hive, Airflow, etc.
    • Should have experience optimizing (Performance tuning) data pipelines.
    • Minimum experience of 3 years on NoSQL and Spark Streaming.
    • Knowledge of Kubernetes and Docker is a plus.
    • Should have experience with Cloud services either Azure/AWS.
    • Should have experience with on-prem distribution such as Cloudera/HortonWorks/MapR.
    • Basic understanding of CI/CD pipelines.
    • Basic knowledge of Linux environment and commands.

Keyskills :
sparkairflowsqlsqooppython scala javadata streaming pipelines

APPLY NOW

Lead Data engineer Related Jobs

© 2019 Hireejobs All Rights Reserved