Software Engineer II

Job Description

Microsoft s cloud business is growing exponentially, and it is the Capacity Supply Chain and Provisioning (CSCP) team that is responsible for enabling the infrastructure underlying this growth. Key responsibilities include demand forecasting, capacity planning, supply chain management and deployment execution. To efficiently manage this scope at scale, analytics based on high quality, timely data flowing across business processes is essential. The engineering team within CSCP is undergoing a digital transformation that will enable the business to efficiently scale and is looking for passionate data engineers to solve world class problems.Within CSCP, the Usage Team is responsible for collecting telemetry across the Azure Fleet, transforming, and aggregating that data to provide a high-quality input signal for downstream systems. This is a world class big data engineering problem operating at scale. The usage system must constantly detect, analyze, and mitigate data quality issues before they impact dependent systems. The opportunity is to join this team as they build the next generation system using machine learning technologies to detect anomalies, predict missing data values, and build automation over a complex data transformation graph.We are looking for a Senior Data Engineer passionate about transforming information to intelligence, who is experienced with engineering data ingestion frameworks, data quality frameworks, and multiple projections of data - relational, dimensional and stream based using big data platforms. You should also have a working knowledge of machine learning technologies, message and event processing, proven software development skills, and experience as part of a dev ops team.If you love data and are excited about data engineering challenges, this is an opportunity to have a substantial impact in a strategically vital area of our business.

• You will work with design and build automated systems with pipelines, transformations, data quality analytics and control logic.
• Contribute to the design and development of data platform components to validate and improve data quality, detect anomalies, and predict missing values.
• Generate, process, log and correlate events across multiple sources.
• Work with engineering teams and data scientists to integrate insights with automation and deliver near term business impact.
• Use engineering best practices throughout the software development lifecycle to establish maintainable, reliable, and secure systems
• Work in a culture of continuous improvement, adaptation, reflection, and growth
• Drive end-to-end features that touch multiple engineering aspects: designing, implementing, integrating, testing, and deploying

,

• BS/MS in Computer Science, Engineering, or related technical field
• 5+ years of experience designing and developing software.
• Proficient with C#, test-driven development, refactoring and object-oriented development.
• 1+ years of experience with data aggregation platforms based on technologies such as Spark, SQL, Azure Data Lake, Hadoop, Cosmos, etc.
• Solid understanding and proven skills in raw and processed stream design, relational database design and dimensional models
• Experience using scalable data ingestion and transformation systems in batch and near real time environments.
• Experience with SQL, relational modelling, and big data tools such as Cosmos, Hive, Scope or Spark
• Experience with ETL, data modelling, and working with Business Intelligence systems.
• Solid understanding of event processing including publish/subscribe mechanisms.
• Demonstrated ability to create and ship high quality code by using engineering best practices.

Preferred Qualifications:

• Ability to understand complex distributed systems and to know your way around them.
• Experience with machine learning and predictive analytics is a plus.
• Strong scripting skills to perform data/file manipulation Python experience is a plus.
• Demonstrated ability to work efficiently, prioritize workflow, and meet demanding deadlines Proven ability to deal with ambiguity and drive for clarity

#CSCPJOBSAbility to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, colour, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations, and ordinances. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form.Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form .Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.

Keyskills :
storagesupply chain managementbig dataautomationenvironmentdata qualitysupply chainanalyticsmissing dataprogrammingsoftware development life cycledata modeling