Azure Data Engineer

Job Description

Job Title: AzureDataEngineerLocation:PAN IndiaExperience: 9 YearsWork Mode: HybridWe are seeking a skilled and motivatedDataEngineerto join our team. The ideal candidate will have a strong background indataengineering,dataarchitecture, anddataintegration. This role involves building and maintaining scalabledatapipelines, ensuringdataquality and integrity, and supportingdataanalytics initiatives. You will work closely with internal customers, product owners, core team members, and other stakeholders to ensure that qualitydatais available, reliable, and easily accessible in a cost effective efficient, scalable way.Key Responsibilities:DataPipeline Development: Design, build, and maintain robust, scalable, and efficientdatapipelines to collect, process, and store large volumes ofdatafrom various sources.DataIntegration: Integratedatafrom multiple sources, including APIs,databases, and externaldatasets, ensuringdataconsistency and reliability.DataModelling: Develop and maintaindatamodels and schemas that support efficientdatastorage, retrieval, and analytics.Database Management: Manage and optimizedatabases, ensuring their performance,availability, and securityDataQuality: Implement and monitordataquality checks to ensure the accuracy, completeness, and consistency ofdata. Perform analysis required to troubleshootdatarelatedissues and assist in the resolution ofdataissues.Automation: configuredataextraction and load jobs and automate repetitive tasks and processes to improve efficiency and reduce errors. Improve performances of jobs andpipelines by applying optimisation and automation techniques.Documentation: Maintain clear and comprehensive documentation ofdatapipelines,datamodels, anddataintegration processes. Maintenance of time allocation reports.Develop understanding of business/ processes and high-level understanding of high-qualitydigital product deliveryWays of working: Follow Agile and SDLC processes including the creation ofdatarelated userstories.Perform code peer reviews and testing as per SDLC or industry standards.Collaboration: Work closely with internal customers, product owners, various core teammembers, and other stakeholders to understand theirdataneeds and provide appropriatesolutions.Technical collaboration and oversight with Vendors as required. Create SOWs and work packets with vendors as required.Education:Bachelors or Masters degree in computer science,datascience, softwareengineering, information systems or a similar field or equivalent experience.Experience and skills:Over 9years experience as aDataEngineeror in a similar role.Experience withdatapipeline, ETL and workflow management tools (e.g.,Databricks,DataFactory).Proficiency in programming languages such as SQL, Python, R, or Scala.Strong experience with SQL anddatabase management (e.g., MySQL, PostgreSQL, SQL Server).Excellent problem-solving and analytical skills.Strong understanding ofdataarchitecture,datamodelling, and ETL processes.Ability to work in a fast-paced, dynamic environment and manage multiple tasks simultaneously.Strong communication and collaboration skills.Experience with best practice UX / UI design & development of Business Intelligencedatavisualisation / exploration tools (Power Bi, Qlik etc)Familiarity with SDLC and Agile ways of working.Familiarity with DevOps tools and practices (e.g. Jenkins, Azure DevOps, etc).Familiarity with Atlassian tools like Bitbucket, JIRA, Confluence.Preferred:GxP and Non GxP SDLC experienceKnowledge ofdatagovernance anddatasecurity best practices.MS Power Platform / power apps experienceCloud DevOps knowledge (e.g. Terraform)Certifications in clouddatatechnologies (e.g. AzureDataFundamentals)Basicdatascience experience and knowledge.Other:English language required.May require travel 10-20% of time based on project locations.

Keyskills :
pythonetlsqladfazure devopspyspark