Hadoop Administrator

Job Description

Hadoop Administrator Hadoop Administrator APPLY FOR THIS JOB Location: Pune , India Experience: 6+ years Number of Positions: 2 At Clairvoyant , were building a thriving big data practice to help enterprises enable and accelerate the adoption of Big data and cloud services. In the big data space , we lead and serve as innovators , troubleshooters , and enablers. Big data practice at Clairvoyant focuses on solving our customers business problems by delivering products designed with best in class engineering practices and a commitment to keep the total cost of ownership to a minimum. Must have: 6+ years of relevant experience - Hadoop , System administration with sound knowledge in Unix based Operating System internals. Working knowledge of Hadoop basics including HBase , Kafka , Impala , Oozie Experience in setting up and supporting Hadoop environment (Cloud and On - premise) Ability to design the structure , plan the capacity , performance fine - tune , monitor the components of the cluster. Ability to closely work with the infrastructure , networking , development teams to ensure the timely deliverables. Ability to setup , configure and implement security for Hadoop clusters using Kerberos. Experience in setting up services like YARN , HDFS , Zookeeper , Hive , Impala , HBase etc. Should have done CM and CDH upgrades Linux tuning - understands kernel parameters tuning and tweaking them. Strong scripting background (shell / bash , python / ruby , etc.) - automation , configuration management. Develop and document best practices. Ability to perform Backup and Recovery as applicable. Experience in large scale enterprise apps that involves provisioning systems using virtualization on private clouds and using physical servers. Comfortable to work in 247 rotational shifts. The interest , ability , curiosity to tinker and automate everything in a repeatable fashion with least amount of manual work involved. Strong analytical skills and Get it done attitude. Must have at least one of these: Chef / Puppet / Ansible awareness Any MongoDB , Cassandra , Hadoop , elastic search cluster administration experience AWS experience (VPC , OpsWorks , IAM ) Preferred to have understanding of the following tools: Log aggregation , ELK , Splunk , logstash , Ganglia , Nags BlueGreenDeployments - understanding and experience Measuring availability , SLAs , 99th percentile , 99.9th percentile Open source Technology background Responsible for implementation and ongoing administration of Hadoop infrastructure as part of our Global Managed Services group. Ensuring high availability of the services in the cluster and ensuring that the cluster resources are utilized optimally reducing the costs. Work closely with India and US team with some overlap hours to ensure a proper handover resulting in optimal productivity. Install and configure monitoring tools. Diligently teaming with the infrastructure , network , database , application and business teams to guarantee high data quality and availability. Managing the Hadoop File system and monitoring the services and performing backup and disaster recovery (when applicable) for Hadoop data. Coordinate root cause analysis (RCA) efforts to minimize future system issues. Provide off hours support on a rotational basis. Assist with develop and maintain the system runbooks. Create and publish various production metrics including system performance and reliability information to systems owners and management. Work with big data developers and developers designing scalable supportable infrastructure. With help of senior hadoop administrators , ensure resolution of tickets within defined SLA Actively work with senior administrator for cluster maintenance activities. Keep all the related documentation of the assigned clusters up to date. Could act as first point of contact for escalation in the assigned shift Ensure meeting of Clairvoyant standards while providing best customer service Provide 247 support to client clusters (will be working in shifts on rotation basis) Participate in creation of weekly / monthly / quarterly client cluster reports Ensure smooth transition of needed information to next shift administrators Actively participate in all planned training. Provide training and mentoring to junior administrators Actively monitor and resolve client tickets. Understand and ensure implementation of set ticket workflow Perform duties as needed within the engineering team.,

Keyskills :
linux pig hive hadoop sqoop ilesystem totalcostofownership rootcauseanalysis stronganalyticalskills rootcause dataquality opensource bigdata privateclouds elasticsearch