| Hyderabad Jobs |
| Banglore Jobs |
| Chennai Jobs |
| Delhi Jobs |
| Ahmedabad Jobs |
| Mumbai Jobs |
| Pune Jobs |
| Vijayawada Jobs |
| Gurgaon Jobs |
| Noida Jobs |
| Hyderabad Jobs |
| Banglore Jobs |
| Chennai Jobs |
| Delhi Jobs |
| Ahmedabad Jobs |
| Mumbai Jobs |
| Pune Jobs |
| Vijayawada Jobs |
| Gurgaon Jobs |
| Noida Jobs |
| Oil & Gas Jobs |
| Banking Jobs |
| Construction Jobs |
| Top Management Jobs |
| IT - Software Jobs |
| Medical Healthcare Jobs |
| Purchase / Logistics Jobs |
| Sales |
| Ajax Jobs |
| Designing Jobs |
| ASP .NET Jobs |
| Java Jobs |
| MySQL Jobs |
| Sap hr Jobs |
| Software Testing Jobs |
| Html Jobs |
| Job Location | Delhi |
| Education | Not Mentioned |
| Salary | Not Disclosed |
| Industry | NGO / Social Work |
| Functional Area | General / Other Software |
| EmploymentType | Full-time |
Manager/AM Data Engineer Role:Data EngineerReporting to:Head, Business Intelligence and Fraud AnalyticsFunction:TechnicalExperience:4 8 YearsLocation:DelhiRole Description Junior Data Engineer(s) will support in Data Extraction, transformation and processing operations for the Management Information Systems (MIS), Business Intelligence (BI) & Data Science workflows.They will assist the business users with various ad-hoc queries and MIS report preparation. They will also assist Data Scientists & Senior Analysts in building and maintaining Data Pipelines which feed transformed data into various analytical models & dashboards. They will assist Data architects with Data Platform health reports, data lifecycle & governance workflows. As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the design of data solutions using Hadoop based technologies along with Python & Pyspark programming.Key Responsibilities Data Extraction, Load & TransformationImplement the extraction & load query architecture as per designWrite & maintain extraction queries batch, realtime ingestion & ad-hocWrite & maintain multi-source to multi-target extraction to load pipelines Write data transformation queries as per business logicCreate and maintain Transformation pipelines/DAGs Continuously optimize queries & pipelines for higher performance Provide inputs to the Data Architect, Data Scientists, Data Analysts & Business users & resolve queries around dataCreate & maintain API s to expose data As a serviceMaintain HDFS folders, Hive/HBase tables. Maintain other data repos & data version control toolsDesign ETL flows using NIFI, Airflow & OozieProcess Large datasets using Python, Pyspark and ScalaResponsible to design and develop distributed, high volume, high velocity multi-threaded event processing systemsAutomate Data Testing & implement continuous testing as part of the DataOps process. Implement key principles of the DataOps manifestoData Steward, Governance & Lifecycle Maintenance Play the role of Data Steward for transformed datasets, Maintain Data dictionaries & Metadata informationWork with Data Architect to implement Data Governance & Lifecycle reportsCreate & Maintain housekeeping scripts & maintain the sanctity of the reporting & analytical datawarehouses Be the custodian of reporting & analytics data warehousesInteract, work & if required direct the Managed Services Provider (MSP)Monitor & track the activities of MSP in regards to day to day functioning of the data lake & normal upkeep of data warehouses.Others Learn new technologies & tool and suggest better ways of improving data operationsDesign, Discuss with Senior Management & Own your personal & professional development planCreate, Discuss & work through a training plan in the area of your interest which aligns with current & future responsibilitiesKey InterfacesExternal: Internal: External stakeholders (tax authorities, taxpayers, treasuries, banks, policy makers, process groups, etc.) Internal departments i.e. Technology, Strategy, MIS & Analysis, Services team and CISOKey Attributes & Skills:Basic Qualifications:Experience of 4-8 years as a Big Data Engineer. Graduate with any STEM fieldCore Skills
Keyskills :
big dataapache kafkamanaged servicesevent processingunstructured datainformaticadata warehousingunix shell scriptingjavapythonmanagement information systemsdata scienceversion controldata solutionsshell scriptingdata governancesqldata ext