Data Engineer with SQL

Job Description

As an Azure Data Engineer at our company, you will be responsible for designing, building, and optimizing high-performance data pipelines on Azure. Your main focus will be on delivering scalable, efficient, and reliable data solutions by developing and maintaining critical data workflows. You will work independently and collaboratively to solve complex data challenges and drive continuous improvements across our data engineering practices.Key Responsibilities:- Build and manage large-scale data pipelines with at least 4 years of data engineering experience- Utilize Python, PySpark, Spark, Databricks, Delta Lake, and SQL for data processing tasks- Work with Azure Data Factory, Azure Pipelines, Azure DevOps, and Git for version control- Optimize performance of data processing through code tuning and resource management- Implement CI/CD pipelines using Azure DevOps, GitHub Actions, or similar tools- Collaborate with data engineers, architects, and business stakeholders to deliver scalable solutions- Demonstrate strong problem-solving, debugging, and troubleshooting skillsQualifications Required:- Bachelors or masters degree in computer science, information systems, engineering, or a related field; candidates without these degrees should have a minimum of 4 years of professional experience- Strong proficiency in Python, PySpark, Spark, Databricks, Delta Lake, and SQL- Experience working with Azure Databricks and PySpark/Spark for data engineering tasks- Knowledge of data modeling and working with complex data types- Hands-on experience in performance optimization techniques for large-scale data processing- Ability to implement CI/CD pipelines with Azure DevOps, GitHub Actions, or similar tools- Excellent communication skills and the ability to work independently and collaborativelyNice to have:- Understanding of multithreading concepts and JVM process behavior in distributed systems- Knowledge of data security, compliance, and governance best practices- Familiarity with monitoring, alerting, and operational runbooks for production-grade data pipelinesIn addition to technical skills, you should have interpersonal skills such as proven experience collaborating with cross-functional teams, effective communication with stakeholders at all levels, confidence, energy, and a commitment to ethics and business integrity. As an Azure Data Engineer at our company, you will be responsible for designing, building, and optimizing high-performance data pipelines on Azure. Your main focus will be on delivering scalable, efficient, and reliable data solutions by developing and maintaining critical data workflows. You will work independently and collaboratively to solve complex data challenges and drive continuous improvements across our data engineering practices.Key Responsibilities:- Build and manage large-scale data pipelines with at least 4 years of data engineering experience- Utilize Python, PySpark, Spark, Databricks, Delta Lake, and SQL for data processing tasks- Work with Azure Data Factory, Azure Pipelines, Azure DevOps, and Git for version control- Optimize performance of data processing through code tuning and resource management- Implement CI/CD pipelines using Azure DevOps, GitHub Actions, or similar tools- Collaborate with data engineers, architects, and business stakeholders to deliver scalable solutions- Demonstrate strong problem-solving, debugging, and troubleshooting skillsQualifications Required:- Bachelors or masters degree in computer science, information systems, engineering, or a related field; candidates without these degrees should have a minimum of 4 years of professional experience- Strong proficiency in Python, PySpark, Spark, Databricks, Delta Lake, and SQL- Experience working with Azure Databricks and PySpark/Spark for data engineering tasks- Knowledge of data modeling and working with complex data types- Hands-on experience in performance optimization techniques for large-scale data processing- Ability to implement CI/CD pipelines with Azure DevOps, GitHub Actions, or similar tools- Excellent communication skills and the ability to work independently and collaborativelyNice to have:- Understanding of multithreading concepts and JVM process behavior in distributed systems- Knowledge of data security, compliance, and governance best practices- Familiarity with monitoring, alerting, and operational runbooks for production-grade data pipelinesIn addition to technical skills, you should have interpersonal skills such as proven experience collaborating with cross-functional teams, effective communication with stakeholders at all levels, confidence, energy, and a commitment to ethics and business integrity.

Keyskills :
PythonSparkSQLAzure DevOpsGitData ModelingPerformance OptimizationDebuggingTroubleshootingPySparkDatabricksDelta LakeAzure Data FactoryAzure Pipelines