Metadata driven Spark Declarative Pipelines framework for bronze/silver pipelines
Notebooks to learn Databricks Lakehouse Platform
Metadata-driven framework for Databricks Spark Declarative Pipelines. Config-driven, pattern based approach to batch & streaming across the medallion architecture. Deploys via Declarative Automation Bundles. Built for simplicity, extensibility, and alignment with the Databricks product roadmap.
🧱 A comprehensive visual and reference guide to Spark Declarative Pipelines (SDP) – Apache Spark's declarative API for building maintainable, orchestrated data pipelines.
Fictional Restaurant Chain Analytics Lakehouse with medalion data layer architecture and dimensional modeling using star schema via Lakeflow Spark Declarative Pipelines in Azure Databricks workspace
Spark Declarative Pipelines UI - Standalone app for running sdp
A collection of modern Spark Declarative Pipeline examples and implementations demonstrating different data processing paradigms and frameworks.
End‑to‑end data engineering solution built in the real estate domain, using Python, Spark and Databricks Platform.
Metadata Management Framework for Spark and Spark Declarative Pipelines
Add a description, image, and links to the spark-declarative-pipelines topic page so that developers can more easily learn about it.
To associate your repository with the spark-declarative-pipelines topic, visit your repo's landing page and select "manage topics."