Python & Big Data Platform Developer

Job Description

*About UsMorgan Stanley is a leading global financial services firm providing a wide range of investment banking, securities, investment management and wealth management services. The Firms employees serve clients worldwide including corporations, governments and individuals from more than 1,200 offices in 43 countries.As a market leader, the talent and passion of our people is critical to our success. Together, we share a common set of values rooted in integrity, excellence and strong team ethic. Morgan Stanley can provide a superior foundation for building a professional career - a place for people to learn, to achieve and grow. A philosophy that balances personal lifestyles, perspectives and needs is an important part of our culture.Technology/Role/Department at Morgan StanleyTechnology works as a strategic partner with Morgan Stanley business units and the worlds leading technology companies to redefine how we do business in ever more global, complex, and dynamic financial markets. Morgan Stanleys sizeable investment in technology results in quantitative trading systems, cutting-edge modelling and simulation software, comprehensive risk and security systems, and robust client-relationship capabilities, plus the worldwide infrastructure that forms the backbone of these systems and tools. Our insights, our applications and infrastructure give a competitive edge to clients businesses and to our own.Legal & Compliance technology leverages a Spark & Hadoop based big data stack to ingest and analyze billions of trades for market abuse patterns, detect fraudulent transactions, and perform sanctions screening on top of our petabyte of HDFS storage.Our infrastructure is required to deal with these mixed workloads & Scala and Python batch processes, data ingestion, SQL queries through Impala, and Dataiku and Zeppelin notebooks written by data scientists. All without impacting performance or disrupting our critical regulatory processes, and making sure our data remains secure and accessible by those who need it.We build the tooling and frameworks so that developers across the firm can write efficient, optimized and easily deployable Spark code without having to rediscover the core principles that let us scale our business problems to thousands of cores and terabytes of memory.Position DescriptionAs a Big Data Platform Developer you will be responsible for our Big Data platforms stability and the design and development of core functionality and strategic solutions that utilize big data technologies in order to meet our constantly changing business requirements. This includes the development of reusable tooling / APIs, an efficient entitlements models, and software stack to accelerate our current big data development and analytics solutions.Responsibilities include:- Designing and developing strategic technology solutions on Hadoop based on business requirements- Collaboration within the organization on current and future state architecture- Hands-on development working within a global development team- Monitoring platform processes and activities across various analytics teams to ensure platform stability- Coordination with our DevOps teams, infrastructure engineers, and production support to triage and resolve issues and efficiently manage resource capacity of our Big Data platform, *Skills Required:- 5-8 years of appropriate technical experience- Proficiency with developing software with Scala and Python- Experience with Spark for data processing- Exposure maintaining Cloudera Hadoop infrastructure such as HDFS, YARN, Spark, Impala &edge nodes- High quality software architecture and design methodologies and patterns- Strong SQL skills with commensurate experience in a large database platform- Complete SDLC process and Agile Methodology (Scrum)- Strong oral and written communication skillsSkills Desired:- Experience with Core Java- Experience with XML/JSON technologies- Experience with Data Federation or Virtualization technologies- Experience with developing Cloud-based Big Data solutions- Experience with tools / platforms / technologies like Dataiku, ElasticSearch, etc.- Experience in development on other application types (Web applications, batch, or streaming)

Keyskills :
business unitsproduction supporttrading systemssql queriessecurity systemsagile methodologyfinancial serviceswealth managementsimulation softwareinvestment bankingbig data