Session-based recommendation system scaffold aligned to an SR-GNN style workflow and packaged for MLOps demonstration.

The repository is structured for a session-based recommender. The production-facing model wrapper is SRGNNRecommender, with a transition-graph baseline behind the same interfaces so the repository stays usable before a full neural SR-GNN implementation is dropped in.

• Core Framework: FastAPI, Uvicorn
• Data Processing: Polars, Pandas, PyArrow, Pandera
• Utilities: Loguru, Pydantic, PyYAML

• Framework: React, TypeScript, Vite
• State & Animation: Zustand, Framer Motion, D3.js
• Deployment: Cloudflare Pages

• Frameworks: PyTorch, PyTorch Geometric (SR-GNN, GGNN, TAGNN), Scikit-learn
• Experiment Tracking: MLflow, DagsHub

• Containerization: Docker, Docker Compose
• Orchestration: Kubernetes (AWS EKS), Kustomize
• Infrastructure as Code: Terraform
• Data Versioning: DVC (Data Version Control) with S3 backend
• CI/CD: GitHub Actions, GitHub Container Registry
• Monitoring: Prometheus, Grafana, Evidently (Model Drift & Quality)

• Testing: Pytest, Unittest
• Linting & Formatting: Ruff, ESLint
• Static Analysis: Mypy, Bandit, Pip-audit

recsys-group-project/
├── .github/workflows/      # CI: lint, test, package checks
├── configs/                # YAML configs for data, model, training, serving
├── data/                   # raw/, interim/, processed/ datasets
├── deployment/             # K8s, MLflow, monitoring assets, Terraform
├── diagrams/               # Architecture and MLOps diagrams
├── docs/                   # Problem statement and team contracts
├── frontend/               # React Vite SPA for user interface
├── metrics/                # Local metrics tracking and model selection JSONs
├── models/trained/         # Local model registry / exported artifacts
├── notebooks/              # EDA notebooks
├── pipelines/              # DVC pipelines (data, monitoring, training)
├── reports/                # Monitoring and evaluation reports
├── scripts/                # Utility scripts (e.g., DVC lock splitting)
├── src/recsys/             # Application package
├── tests/                  # Unit and smoke tests
├── Dockerfile              # Backend container definition
├── docker-compose.yaml     # Local stack composition
├── pyproject.toml          # Python package config
├── requirements.txt        # Python dependencies
└── README.md

# Install package with development dependencies
pip install -e .[dev]

# Run tests
python -m unittest discover -s tests -p "test_*.py"

# Process data, train model, and serve the API locally
recsys-process-data --stage all --config configs/data_config.yaml --params params.yaml
recsys-train --data-config configs/data_config.yaml --model-config configs/model_config.yaml --training-config configs/training_config.yaml --params params.yaml
recsys-serve --config configs/serving_config.yaml

cd frontend
npm install
npm run dev

Runtime config is merged in this order:

1. configs/*.yaml defaults
2. params.yaml overrides (experiment knobs only)

Use params.yaml for DVC experiment tuning only (model/data/training hyperparameters). Keep metadata and runtime settings (paths, registry, MLflow URI, DagsHub repo, report destinations) in configs/*_config.yaml.

Each data version has its own params file and artifact directory so it can be tracked independently in DVC. The repo provides a closed-loop comparison target that runs data build + train/eval + drift monitoring + aggregation in one pipeline.

# Run the full version comparison pipeline
dvc repro pipelines/monitoring/dvc.yaml:compare_data_versions

# Build data versions independently (no training)
dvc repro pipelines/data/dvc.yaml:data_version_v1
dvc repro pipelines/data/dvc.yaml:data_version_v2

Version definitions:

• V1 strict filter: configs/data_versions/v1_strict_filter.yaml
• V2 sliding-window safety: configs/data_versions/v2_sliding_window.yaml

Generated artifacts are stored in data/versions/, models/trained/, and metrics/.

Model profiles are defined in configs/model_profiles/*.yaml (e.g., srgnn, ggnn, tagnn).

Run fast profiles (default iteration loop):

dvc repro pipelines/training/dvc.yaml:train_matrix pipelines/training/dvc.yaml:evaluate_matrix

Run long-running profiles:

dvc repro pipelines/training/dvc.yaml:train_matrix_slow pipelines/training/dvc.yaml:evaluate_matrix_slow

Final production flow retrains the selected winner on merged train+val data:

dvc repro pipelines/training/dvc.yaml:select_best_model pipelines/training/dvc.yaml:retrain_selected_model pipelines/training/dvc.yaml:promote_retrained_model

Training outputs are intentionally split into DVC-managed model artifacts and local-only experiment metrics. Before committing, ensure only source/config/lock changes are present:

git status --short
# Expected: changes to pipelines/*/dvc.yaml, pipelines/*/dvc.lock, code, config, docs

After training and before dvc push, confirm new cache objects:

dvc status -c
dvc push

If migrating from the legacy monolithic dvc.lock, split the lock state first without recomputing artifacts:

python scripts/split_dvc_lock.py

To track experiments via DagsHub/MLflow, enable it in configs/training_config.yaml:

mlflow:
  enabled: true
  dagshub:
    enabled: true
    token_env_var: DAGSHUB_USER_TOKEN
    repo_owner: lytlong.pers
    repo_name: recsys-group-project

Export your token before running:

export DAGSHUB_USER_TOKEN=<your_personal_access_token>

Enable MLflow system metrics collection by config to track CPU/memory/GPU usage:

mlflow:
  system_metrics:
    enabled: true
    sampling_interval: 10
    samples_before_logging: 1

Install host dependencies:

pip install psutil
# Optional GPU metrics: pip install nvidia-ml-py

GitHub Actions publishes container images to the GitHub Container Registry (GHCR) after successful CI runs triggered by pushes to main.

• Image: ghcr.io/<owner>/<repo>
• Tags: main and sha-<short_sha>

For Kubernetes deployment assets, refer to the deployment/ directory.