Airflow Unit Tests and Integration Tests
-
Updated
Nov 16, 2022 - Python
#
airflow-dags
Here are 344 public repositories matching this topic...
Gerador de DAGs no Apache Airflow para fazer clipping do Diário Oficial da União.
-
Updated
Apr 25, 2025 - Python
Classwork projects and home works done through Udacity data engineering nano degree
data-science big-data spark etl s3-bucket data-analysis redshift data-pipelines classwork emr-cluster data-lake-analytics data-engineering-pipeline airflow-dags
-
Updated
Dec 12, 2023 - Jupyter Notebook
Data Engineering examples for Airflow, Prefect; dbt for BigQuery, Redshift, ClickHouse, Postgres, DuckDB; PySpark for Batch processing; Kafka for Stream processing
airflow kafka spark pyspark prefect airflow-dags ksqldb typer-cli dbt-bigquery dbt-postgres dbt-clickhouse dbt-redshift
-
Updated
Feb 19, 2025 - Python
Sync DAG changes from Git to Airflow
-
Updated
May 14, 2025
-
Updated
Aug 18, 2021 - Python
Zero configuration Airflow plugin that let you manage your DAG files.
-
Updated
Sep 26, 2021 - Python
Orchestration of data science and earth observation models in Apache Airflow, scale-up with Celery Executor, experiment with jupyter notebook using a docker containers composition
-
Updated
Aug 23, 2022 - Python
My self-learning about Apache Airflow
-
Updated
Jul 13, 2022 - Python
🎵 LyricWave - Your AI Music Composer 🎶 Compose Unique MP4 Songs Effortlessly! LyricWave uses AI to create personalized music by harmonizing lyrics with captivating melodies and synthetic vocals. Unleash your musical creativity today! 🚀🎶
docker flask airflow mongodb docker-compose pytorch celery music-generation flask-api airflow-docker apache-airflow airflow-dags huggingface airflow-operators huggingface-transformers music-processing stable-diffusion suno-ai musicgen audiocraft
-
Updated
Mar 13, 2025 - Python
This is my Apache Airflow Local development setup on Windows 10 WSL2/Mac using docker-compose. It will also include some sample DAGs and workflows.
python docker yaml airflow sql superset data-engineering workflows airflow-docker treasuredata apache-superset airflow-dags wsl2 airflow-testing sample-dags airflow-community
-
Updated
Feb 9, 2024 - Python
Apache Airflow Guide
python airflow awesome big-data distributed data-engineering business-intelligence awesome-list airflow-plugin airflow-docker awesome-resources business-analytics airflow-dags airflow-operators
-
Updated
May 7, 2024 - Python
Orchestrate your Databricks notebooks in Airflow and execute them as Databricks Workflows
-
Updated
Aug 8, 2024 - Python
Loan Default Prediction using PySpark, with jobs scheduled by Apache Airflow and Integration with Spark using Apache Livy
docker unit-testing airflow spark docker-compose docker-image python3 pyspark classification livy loan-default-prediction airflow-plugins airflow-dags pyspark-machine-learning livy-operators livy-docker auc-roc
-
Updated
Dec 26, 2020 - Jupyter Notebook
WebApp to bring together Text Summarization and Sentiment Analysis of the stock related news to better understand the stock price trends.
python docker aws-lambda sentiment-analysis web-crawler yahoo-finance-api text-summarization aws-ec2 news-scraper gcp-storage bert-model fastapi airflow-dags newspaper3k streamlit gcp-app-engine gcp-bigquery huggingface-transformers gcp-datastudio
-
Updated
Apr 7, 2025 - Jupyter Notebook
An end-to-end Twitter Data Pipeline that extracts data from Twitter and loads it into AWS S3.
python airflow twitter etl twitter-api scheduler s3 data-engineering tweepy boto3 data-pipeline apache-airflow etl-pipeline etl-job data-engineering-pipeline airflow-dags twitter-data-pipeline
-
Updated
Aug 26, 2023 - Python
Here I added 9 projects which have been made by me during my apprenticeship in Yandex.Practicum as data engineer.
python kubernetes redis sql kafka mongodb hadoop docker-compose postgresql pyspark vertica airflow-dags
-
Updated
Dec 16, 2023 - Python
DAGs, Python Scripts, and SQL scripts for creating and updating the Saint Louis Regional Entity Database
-
Updated
Oct 26, 2020 - PLpgSQL
A starting point for a data stack using Python, Apache Airflow and Metabase.
python docker template data-science airflow stack sphinx metabase data-engineering airflow-docker data-stack airflow-dags doccano
-
Updated
Dec 8, 2022 - Python
Automate your data pipelines using Apache Airflow with this ready-to-use DAG for data integration, ETL and workflow automation.
python airflow etl data-warehouse data-engineering data-integration data-pipelines apache-airflow dags airflow-dags
-
Updated
Sep 13, 2024
Improve this page
Add a description, image, and links to the airflow-dags topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the airflow-dags topic, visit your repo's landing page and select "manage topics."