hireejobs
Register for Job
Hyderabad Jobs
Banglore Jobs
Chennai Jobs
Delhi Jobs
Ahmedabad Jobs
Mumbai Jobs
Pune Jobs
Vijayawada Jobs
Gurgaon Jobs
Noida Jobs
Oil & Gas Jobs
Banking Jobs
Construction Jobs
Top Management Jobs
IT - Software Jobs
Medical Healthcare Jobs
Purchase / Logistics Jobs
Sales
Ajax Jobs
Designing Jobs
ASP .NET Jobs
Java Jobs
MySQL Jobs
Sap hr Jobs
Software Testing Jobs
Html Jobs
IT Jobs
Logistics Jobs
Customer Service Jobs
Airport Jobs
Banking Jobs
Driver Jobs
Part Time Jobs
Civil Engineering Jobs
Accountant Jobs
Safety Officer Jobs
Nursing Jobs
Civil Engineering Jobs
Hospitality Jobs
Part Time Jobs
Security Jobs
Finance Jobs
Marketing Jobs
Shipping Jobs
Real Estate Jobs
Telecom Jobs

Lead Data Engineer

1.00 to 10.00 Years   Pune   31 Jul, 2026
DIGITALCUBE CONSULTANCY SERVICES PRIVATE LIMITED
Job LocationPune
EducationNot Mentioned
SalaryNot Disclosed
IndustryIT Services & Consulting
Functional AreaApplication Programming / Maintenance
EmploymentTypeFull-time

Job Description

    Designation:Lead Data Engineer (Databricks & PySpark)Experience:8to 14 yearsLocation:MumbaiWork Mode:HybridAbout the Role:We are seeking a highly skilled and experienced Lead Data Engineer to design, build, and operate our next-generation data platform. In this role, you will champion Legacy Code Modernization efforts, migrating traditional ETL processes into modern, scalable ELT patterns on Databricks. As a technical leader, you will manage a cross-functional team, oversee end-to-end delivery, and collaborate with Infrastructure, Applications, and Cyber Security teams to drive data engineering excellence across the organization.Key Responsibilities-Data Pipeline Development & OperationsDesign and Build:Develop reliable, scalable end-to-end data workflows from ingestion through transformation to consumption on the Databricks platform.Operations & Monitoring: Implement robust error handling, alerting mechanisms, and monitoring to ensure pipeline performance and uptime.Performance Tuning:Optimize Spark job design and cluster configurations to maximize throughput and minimize cloud infrastructure costs.Orchestration:Manage complex multi-stage data workflows using Databricks Jobs and modern orchestration patterns.Legacy Code ModernizationCode Refactoring:Assess existing, traditional codebases and refactor legacy ETL workflows into highly efficient PySpark pipelines.ELT Migration: Transition legacy data structures to modern ELT lakehouse patterns on Databricks.Risk Mitigation:Maintain backward compatibility and data integrity during migrations, creating clear playbooks to minimize business disruption.Data Engineering Excellence & LeadershipGovernance & Quality:Implement data quality verification frameworks and validation checks to protect data integrity.Delta Lake Architecture:Design and optimize Delta Lake tables utilizing advanced storage features, ACID transactions, and schema evolution.Team Leadership:Manage, mentor, and foster growth for junior and mid-level data engineers, driving knowledge-sharing initiatives across Team.Delivery Management:Own project delivery by participating in agile ceremonies, sprint planning, estimation, and release management.Job RequirementsExperience & BackgroundOverall Experience:8 to 14 years of professional experience in data engineering or related backend fields.Databricks Focus:4 to 8 years of intensive, hands-on experience building scale production workflows on Databricks.Leadership:2 to 3 years of direct experience handling engineering teams, managing stakeholders, and providing production support.Essential Technical SkillsPySparkAdvanced Python programming capabilities tailored for data engineering and automation workflows.SQL: Deep proficiency in writing complex SQL queries, analytical functions, and data transformations.Delta Lake: Solid understanding of Delta Lake optimization strategies, transactional layouts, and data modeling (dimensional, data vault, or lakehouse).Workspace AI Agent: Familiarity with Databricks Workspace AI Agent features and integration workflows.Desirable Technical Skills (Nice to Have)Cloud ArchitectureDevOps/DataOps: Streaming.Mandatory CertificationsDatabricks Certified Data Engineer AssociateDatabricks Certified Data Engineer ProfessionalPreferredCertificationsDatabricks Certified Associate Developer for Apache SparkCloud platform data certifications (e.g., Azure Data Engineer Associate, AWS Certified Data Analytics, GCP Professional Data Engineer.

Keyskills :
sqlpythondelta lakedatabricks

APPLY NOW

Lead Data Engineer Related Jobs

© 2019 Hireejobs All Rights Reserved