Software Engineer BigData

Job Description

Responsibilities Because we work on the cutting edge of a lot of technologies, we need someone who is a creative problem solver, resourceful in getting things done, and productive working independently or collaboratively. This role will be responsible for developing and deploying a game- changing and highly- disruptive advertising technology platform. This person would also take on the following responsibilities: Gather and process raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.) Work closely with our engineering team to integrate your amazing innovations and algorithms into our production systems Support business decisions with ad hoc analysis as needed Propose and investigate new techniques Troubleshoot production issues and identify practical solutions Routine check- up, back- up and monitoring of the entire Hadoop ecosystem Take end- to- end responsibility of the Big Data ETL, Analysis and processing Life Cycle in the organization Build, deploy and maintain real- time streaming pipelines and real- time analytics Manage deployments of big- data clusters across private and public cloud platforms Required Skills 2+ years of Hands- on experience working with Cloudera Hadoop Distribution platform and Apache Spark 3+ years of experience with SQL and relational databases Comprehensive knowledge of Hadoop Architecture and HDFS is a must, to design, develop, document and architect Hadoop applications Working expertise with Apache Spark, spark streaming, and Python or Scala programming Working knowledge of Map- Reduce, Hive, Kafka, HBase, Pig, Java and Impala Experience with Python, R, and Big Data tools like Oozie, Zookeeper, Sqoop, Hive, Impala to manage and administer TB of data processing Working knowledge of SQL, NoSQL, data warehousing & DBA Experience processing large amounts of structured and unstructured data Experience extracting and transforming data from remote data stores, such as relational databases or distributed file systems Strong knowledge and experience with data analytics, mathematics and statistics Seamlessly convert hard- to- grasp technical requirements into outstanding designs Knowledge of agile methodology for delivering software solutions Additional Desired Skills Excellent communication skills, ability to tailor technical information for different audiences Excellent teamwork skills, ability to self- start, share insights, ask questions, and report progress Strong enthusiasm in solving messy problems with good solution design skills Adequate experience with data analysis or software development in a Unix- based system Ability to grasp emerging technical standards and apply in right direction Benefits Leave travel allowance, medical allowance, conveyance allowance, bonus etc. are a given everywhere & here at Affinity too. However, we strive to go beyond the usual & the drab. If a young competitive work environment, casual dress code, flexible working hours, on- the- house snacks, movies, sports events, flexible leaves, 5- day work week (Mon- Fri), recreational facilities (Pool, TT, Foosball, etc.) are your cheat- codes, then youre sure to win at Affinity. We are a company, which realizes that its employees are its most important assets & hence we strive towards providing a conducive work environment which is flexible, professional, prospering, stable, innovative and truly international in scope. If you wish to build a career with a thriving, fast- paced and cutting- edge technology company that offers personal & professional growth, we sure do want to hear from you! Founded in 2006, we are worlds leading contextual ad network. We build innovative advertising media and monetization solutions for Advertisers & Publishers. Our advertising media and monetization solutions are delivered via Affinitys Global Pay- Per- Click (PPC) & Display Advertising Networks. Both these networks help 100, 000+ Advertisers reach out to their target audience by delivering over 50 Billion ad impressions each month. This is achieved through exclusive contextual ad distribution via our network of 10, 000+ premium publishers and syndication channels from across North America, Western Europe, Middle East, South- East Asia, India and Australia.,

Keyskills :
java jquery sql javascript ataanalysis sqlserver dataanalytics lifecycle filesystems webscraping bigdata solutiondesign distributedfilesystems apachespark dataprocessing sqlqueries