Hadoop Tech Lead

Job Description

*About UsMorgan Stanley is a leading global financial services firm providing a wide range of investment banking, securities, investment management and wealth management services. The Firms employees serve clients worldwide including corporations, governments and individuals from more than 1,200 offices in 43 countries.As a market leader, the talent and passion of our people is critical to our success. Together, we share a common set of values rooted in integrity, excellence and strong team ethic. Morgan Stanley can provide a superior foundation for building a professional career - a place for people to learn, to achieve and grow. A philosophy that balances personal lifestyles, perspectives and needs is an important part of our culture.Technology/Role/Department at Morgan StanleyWealth Management and Investment Management Technology (WMIMT) is responsible for the design, development, delivery, and support of the technical platform behind the products and services used by the Business.Morgan Stanley Wealth Management (WM) is a product of the acquisition of Smith Barney from Citigroup, which was completed in June 13. Its core client base is individual investors, small- to medium-size businesses and institutions, and high net worth families and individuals.The Practice Area Group (PAG) AIDT under Management Technology provides consistent, trusted and integrated business information across all platforms and delivery channels within Morgan Stanley Wealth Management (MSWM) businesses. AIDT areas of expertise include but are not limited to Enterprise Data Warehouse (EDW) and Datalake (DL), cross-business-unit Information Management, Business Intelligence and more. AIDT team synthesizes and provisions a multitude of business-critical metrics and underlying detailed information in all Core areas of MSWM. Examples of essential data domains and metrics include Assets, Revenue, Account Demographics, Financial Advisor (FA) information, Securities and Cash Transactions, and so on.As an integral part of AIDT, Data and Services team is tasked with providing insightful and accurate business intelligence on Solium, Next Best Actions and many other aspects of Wealth Management business.Technology is the key differentiator that ensures that we manage our global businesses and serve clients on a market leading platform that is resilient, safe, efficient, smart, fast and flexible. Technology redefines how we do business in global, complex and dynamic financial markets. We have a large number of award winning technology platforms that help to propel our Firm s businesses to be the top in the market. Our India technology teams are based in Mumbai and Bengaluru. We have built strong techno-functional teams which partner with our offices globally taking global ownership of systems and products. We have a vibrant and diverse mix of technologists working on different technologies and functional domains. There is a large focus on innovation, inclusion, giving back to the community and sharing knowledge, *Role DescriptionRequired Experience:7 to 14 yearsThis Role requires credibility, confidence and active participation with technology groups, COE and leadership teams across the project lifecycle. Immuta implementation is integral part of Data governance initiative.Candidate will be joining AIDT-Cloud squad.Education: Bachelor s/Master s Degree in Engineering, preferably Computer Science/Engineering.As a Hadoop Tech Lead, the candidate is expected to:Design and development of ETL and Hadoop solutions.Undertaking end-to-end project delivery (from inception to post-implementation support), including review and finalization of business requirements, creation of functional specifications and/or system designs, and ensuring that end-solution meets business needs and expectations.Development of new transformation processes to load data from source to target, or performance tuning of existing ETL code (mappings, sessions) and Hadoop Platform.Analysis of existing designs and interfaces and applying design modifications or enhancements.Providing insights and analysis findings for ad-hoc issues.Testing software components and complete solutions (including debugging and troubleshooting) and preparing migration documentation.Providing reporting-line transparency through periodic updates on project or task status.Primary skillsExperienced with the technical analysis and design, development and implementation of Data Lake and Data Warehouse solutions.Hands on experience in Hadoop, PySpark and Big data technologies.Strong UNIX Shell scripting and Python experience to support data warehousing solutions.Process oriented, focused on standardization, streamlining, and implementation of best practices delivery approach.Excellent problem solving and analytical skills.Excellent verbal and written communication skills.Experience in optimizing large data loads.Proven teamwork in multi-site/multi-geography organizations.Ability to multi-task and function efficiently in a fast-paced environment.Self-starter with flexibility and adaptability in a dynamic work environment.Should have ability to perform the root cause of Issues and be able to design solution as required.Ability to gather user requirements, interface with business partners as required.Complete team accountability: deliverables, discipline, consistency.Responsible for quantitatively mapping the success of the team.Good to have skills:Should be a good Team player.Experience with detailed Error/Exception handling and Data Validation issues.Experience in Developing, managing and maintaining data dictionary and or metadata.Nice to have experienced with common data science tools such as Dataiku, Tableau, Business Object, Tensor Flow and a good understanding of modelling platforms (Azure AutoML, Data bricks).Strong understanding of Data warehousing domain.Ability to architect an ETL solution and data conversion strategy.Should know different ways of Hadoop processing Performance Optimization.Being in Frameworks Team able to assist L1 - L2 Ops Support Team and assist/advise them on Production Job failures and should able to assist Non-Prod Ops Support Team and assist them in Non-Prod Environment s Job failures.Experience in Agile methodology.

Keyskills :
unix shell scriptingbig dataroot causedata scienceshell scriptingdata conversionproblem solvingdata validationdata governancestatements of work sow