This project builds an end-to-end computer vision pipeline for animal pose estimation from video frames. Using the DigiDogs dataset, the workflow covers frame preprocessing, keypoint annotation parsing, deep learning model training, quantitative evaluation, and pose visualization.

The full training pipeline was developed and executed on Kaggle GPU notebooks. This repository is a portfolio version designed for recruiters and technical interviewers: it highlights methodology, experiment design, measurable results, and reproducibility without shipping large datasets or model weights.

33-keypoint pose visualization on a DigiDogs frame (numbered skeleton overlay):

• Video frame processing: TIFF/PNG ingestion, frame indexing, and annotation alignment
• Animal keypoint detection: 33-keypoint regression from dog images
• Dataset preprocessing: parsing 2DCoordinates.log, invalid-point filtering, train/val split
• Deep learning training: PyTorch-based CNN baseline with GPU acceleration
• Evaluation: PCK, MAE, RMSE, loss curves, per-keypoint error analysis
• Visualization: keypoint overlays, prediction-vs-ground-truth plots, demo video frames
• Multi-scenario experiments: motion augmentation, background change, scene diversity

DigiDogs Dataset — dog video sequences with per-frame 2D keypoint annotations (2DCoordinates.log).

Keypoint setup:

• 33 valid keypoints used for training and evaluation
• Raw log contains 41 fields (including invalid placeholders / auxiliary points)

The dataset is not included in this repository due to size and licensing constraints. See KAGGLE_NOTEBOOK.md for how to reproduce on Kaggle.

1. Frame extraction & conversion — TIFF images converted to PNG for stable GPU training
2. Annotation parsing — 2DCoordinates.log parsed into structured JSON per frame
3. Coordinate normalization — images resized to 128×128; keypoints mapped to model input space
4. Model — SimplePoseCNN, a lightweight convolutional baseline for coordinate regression
5. Loss — MSE between predicted and ground-truth keypoint coordinates
6. Metrics — PCK@τ (percentage of correct keypoints), MAE, RMSE
7. Post-processing — EMA temporal smoothing for trajectory stability (optional)

DigiDogs frames + annotations
    → Parse & filter keypoints (33 valid)
    → Train/val split (80/20)
    → SimplePoseCNN training (30 epochs, Adam, MSE)
    → Evaluate (PCK, MAE, RMSE)
    → Visualize predictions & export metrics

See docs/workflow.md for a detailed workflow diagram.

Main training set (08_30_2022, 1,600 samples, 30 epochs):

Multi-scenario comparison (same model architecture):

Exported metric files are in results/.

Model weights are not included due to file size limits.

Kaggle notebook output — metrics and exported figures from the main training run:

See screenshots/README.md for the full list.

1. Upload a DigiDogs subset as a Kaggle Dataset (TIFF/PNG frames + 2DCoordinates.log)
2. Open one of the training notebooks (links in KAGGLE_NOTEBOOK.md)
3. Enable GPU accelerator (Settings → Accelerator → GPU)
4. Run all cells — outputs are saved under /kaggle/working/paper_results/
5. Download figures and metrics from the notebook output panel

For local development, install dependencies from requirements.txt and adapt paths in scripts/.

• Curated and organized DigiDogs subsets for four experimental scenarios
• Built the data preprocessing pipeline (TIFF→PNG, log parsing, JSON export)
• Implemented the PyTorch training and evaluation pipeline on Kaggle GPU
• Designed experiments: main training, motion, background, and scene diversity
• Produced quantitative metrics (PCK, MAE, RMSE) and visualization assets
• Documented the end-to-end workflow for reproducibility

• Replace the CNN baseline with stronger architectures (HRNet, RTMPose, Transformer)
• Add heatmap-based pose estimation for finer spatial localization
• Integrate temporal models (TCN, Kalman filter, temporal Transformer) for video consistency
• Expand evaluation with real-world dog videos beyond synthetic DigiDogs frames
• Package inference as a lightweight API for demo deployment

This project is released under the MIT License.