Tech Trends — Technology Trend Analysis Platform
| Challenge |
Solution |
Impact |
| Multi-source tech trends with consistent scoring |
Canonical ETL pipeline + Trend Score bridge |
Stable, explainable rankings |
| Fragile frontend assumptions |
Bridge-first JSON + CSV fallback |
Resilient UI under data changes |
| Weekly refresh without regressions |
CI gates + data contracts |
Predictable releases |
| Metric |
Value |
| Repos classified |
931 |
| StackOverflow questions (annual window) |
33,356 |
| Reddit mentions (emerging topics) |
366 |
| Technologies in ranking |
22 |
| Window |
2025-03-13 → 2026-03-13 (UTC) |
| Layer |
Technologies |
| Data |
Python, pandas, duckdb, pandera |
| ETL |
Custom pipelines + contracts |
| Frontend |
Flutter Web |
| Hosting |
GitHub Pages |
# backend
pip install -r backend/requirements.txt
python -m pytest -q
# run ETLs
python backend/github_etl.py
python backend/stackoverflow_etl.py
python backend/reddit_etl.py
python backend/trend_score.py
python backend/sync_assets.py
# frontend
cd frontend
flutter pub get
flutter run -d chrome
Automation (GitHub Actions)
- ETL Weekly Refresh (
etl_semanal.yml)
- Schedule: Monday
08:00 UTC
- Runs all ETLs, Trend Score, syncs assets, validates contracts, and publishes data.
- Dependency Security (
dependency_security.yml)
- Schedule: Monday
09:00 UTC
- Runs
pip-audit against backend/requirements.txt.
- Deploy Frontend (
deploy_frontend.yml)
- Publishes Flutter Web to GitHub Pages.
Technology-trend-analysis-platform/
├── backend/ # ETLs + Trend Score + contracts
├── datos/ # CSV outputs (legacy + latest + history)
├── frontend/ # Flutter Web UI
├── docs/ # Technical docs
├── scripts/ # CI / assets validation helpers
└── .github/workflows/ # CI + ETL automation
Samir Caizapasto
Junior Data Engineer & Analyst
If you find this project useful, please give the repository a star.
