Senior CICD Engineer

Job Description

We are seeking an experienced CI/CD Engineer or Lead to design, automate, and govern delivery pipelines across the Azure ecosystem (Azure DevOps, Databricks, ADF/Synapse) and adjacent CI/CD platforms. While the final tool choice may vary, the candidate must have significant hands-on experience with Jenkins and strong DevOps fundamentals. The role includes in-sprint testing enablement, platform governance (monitoring, administration, configuration), and cost-effective technologyselection. Excellent communication and documentation skills are essential.Key Responsibilities
• Pipeline Architecture & Delivery: Design and maintain end-to-end CI/CD pipelines using Azure DevOps, Jenkins, GitHub Actions, or GitLab CI. Implement linting and code review checks, integrate static analysis tools (SonarQube/CodeQL), architect multi-stage workflows, build reusable templates for Databricks, and manage artifact repositories.
• Azure Platform Governance & Operations: Monitor, administer, and configure Azure services (Key Vault, Storage/ADLS, ADF/Synapse, Databricks, AKS). Apply RBAC,secretsmanagement, observability (Azure Monitor, Log Analytics), and cost governance.
• Source Control & ALM: Define branching strategy (GitFlow/Trunk), enforce PR policies in Bitbucket, and integrate Jira workflows with CI/CD.
• Automated & In-Sprint Testing: Embed unit, integration, SIT, and E2E/regression tests into pipelines for in-sprint execution. Collaborate with QA to align automated test coverage with sprint goals.
• DevSecOps&Policy asCode: Integrate SAST/DAST, dependency scanning, SBOM generation, and enforce quality/security gates. Apply policy-as-code (OPA/Conftest, Azure Policy).
• Infrastructure as Code (IaC): Provision and promote infrastructure using Terraform (preferred) and/or Bicep; manage remote state and modules. Standardize container builds and AKS deployments.
• Data Storage Layer Management & Governance: Govern data storage across Unity Catalog (Databricks), Azure SQL / SQL Server, and ADLS. Implement data lineage,optimizestorage tiers, and coordinate schema migrations.
• Communication & Collaboration: Communicate effectively with cross-functional teams including QA, Development, Product Management, and Architecture. Produce clear documentation, runbooks, and present status updates to stakeholders.
• Leadership (for Lead level): Define CI/CD standards, mentor engineers, lead design reviews, and drive continuous improvement.
Required Qualifications
• Significant hands-on experience with Jenkins (declarative pipelines, shared libraries).
• Hands-on with Azure services (Key Vault, Storage, ADF, Databricks, AKS).
• Expertisein CI/CD best practices across multiple tools (Azure DevOps, GitHub Actions, GitLab CI).
• Solid understanding of in-sprint testing and test automation frameworks.
• Experience with linting, static analysis, and enforcing code quality gates.
• Scripting: Python, Bash, YAMLproficiency.
• Familiarity withIaCusing Terraform (preferred) and containerization (Docker/Kubernetes).
• Strong communicationand documentation skills.
Preferred Qualifications
• Databricks asset bundling/dbx,MLflowmodel promotion; Unity Catalog administration.
• Data quality frameworks (e.g., Great Expectations).
• GitOps(Argo CD/Flux), Helm,Kustomize.
• SRE practices; DORAmetricsinstrumentation.
Cost-effective Tools & Technologies
• CI/CD: GitHub Actions, GitLab CI, Jenkins (self-hosted),CircleCI.
• Code Quality: SonarQube/CodeQL, pre-commit, Black/Flake8,ESLint.
• Observability: Azure Monitor, Log Analytics; Cost Management tools like Azure Advisor.
• IaC: Terraform (preferred), Bicep; Secrets via Azure Key Vault.
• Data Storage: Unity Catalog, Azure SQL / SQL Server, ADLS Gen2; Catalog alternatives likeDataHubor Azure Purview.
KPIs / Success Metrics
• Lead time for changes, deployment frequency, change failure rate, MTTR (DORA).
• % of pipelines with linting, static analysis, test/security gates.
• %infraunder Terraform; environment drift incidents;secretscoverage.
• Platform cost adherence: budget vs. actual; ingestion/storage optimization achieved.
• Data governance adoption: catalog completeness, Unity Catalog policies applied.

Keyskills :
azure devopsazureadfpythonjenkinsaksservicesdatabricksci/cd