Data Platform Architect

Job Description

8 Years in Data Engineering / Architecture: Proven experience deliveringproduction-grade Lakehouse environments for high-concurrency (1,000 user)organizations. Modern Data Stack Fluency: Extensive experiencewith Databricks (Lakehouse/Unity Catalog) and high-performance warehouseslike Amazon Redshift or Snowflake. Open-Table Formats: Deep hands-on expertise with Apache Iceberg or DeltaLake, including optimization strategies for partitioning and schema evolution. Transformation & Modeling: Mastery of dbt (Core) for complex SQL-basedmodeling and PySpark or Python for sophisticated data processing. High-Efficiency Compute: Familiarity with vectorized/embedded engineslike DuckDB for specialized or cost-sensitive processing tasks. Orchestration Mastery: Advanced experience with Apache Airflow, specificallyin designing resilient, dependency-aware DAGs in resource-constrainedenvironments. Cloud Ecosystems: Expert-level knowledge of AWS (S3, EC2, IAM) orequivalent services in Azure/GCP, with a focus on storage-compute separation. Experience with open-source catalog implementations like Apache Polaris. Knowledge of Data Ops principles and automated data quality testingframeworks. Experience translating technical debt and architectural roadmaps for C-levelexecutives. Background in managing fixed-resource infrastructure (e.g., EC2/VM-basedprocessing) vs. elastic serverless models. Kubernetes (K8s). Deep understanding of Pods, Deployments, Services,

Keyskills :
awsiamsnowflakekubernetesk8sdelta lakedataopsdbt core