This repository provides scripts of deep neural network (DNN) feature decoding from fMRI brain activities, originally proposed by Horikawa & Kamitani (2017) and employed in DNN-based image reconstruction methods of Shen et al. (2019) as well as recent studies in Kamitani lab.
Please setup Python environment where packages in requirements.txt are installed.
# Using venv
$ python -m venv .venv
$ . .venv/bin/activate
$ pip install -r requirements.txtRun the following commands in data directory to download required fMRI and DNN features.
# In "./data" directory:
# fMRI data (collected by Shen et al., 2019)
python download.py fmri_deeprecon_fmriprep_vc
# DNN features (VGG-19)
python download.py features_imagenet_training_vgg19
python download.py features_imagenet_test_vgg19- Training:
train_decoder_fastl2lir.py - Test (prediction):
predict_feature_fastl2lir.py - Evaluation:
evaluation.py - Example config file: deeprecon_pyfastl2lir_alpha100_vgg19_allunits.yaml
# Training of decoding models
$ python train_decoder_fastl2lir.py config/deeprecon_pyfastl2lir_alpha100_vgg19_allunits.yaml
# Prediction of DNN features
$ python predict_feature_fastl2lir.py config/deeprecon_pyfastl2lir_alpha100_vgg19_allunits.yaml
# Evaluation
$ python evaluation.py config/deeprecon_pyfastl2lir_alpha100_vgg19_allunits.yaml- Training:
train_decoder_sklearn_ridge.py(example for scikit-learn Ridge regression) - Test (prediction):
predict_feature.py - Evaluation:
evaluation.py - Example config file: deeprecon_sklearn_ridge_alpha100_vgg19_allunits
# Training of decoding models
$ python train_decoder_sklearn_ridge.py config/deeprecon_pyfastl2lir_alpha100_vgg19_allunits.yaml
# Prediction of DNN features
$ python preeict_feature.py config/deeprecon_pyfastl2lir_alpha100_vgg19_allunits.yaml
# Evaluation
$ python evaluation.py config/deeprecon_pyfastl2lir_alpha100_vgg19_allunits.yaml- Training:
cv_train_decoder_fastl2lir.py(example for scikit-learn Ridge regression) - Test (prediction):
cv_predict_feature_fastl2lir.py - Evaluation:
cv_evaluation.py - Example config file: deeprecon_cv_pyfastl2lir_alpha100_vgg19_allunits
# Training of decoding models
$ python cv_train_decoder_fastl2lir.py config/deeprecon_cv_pyfastl2lir_alpha100_vgg19_allunits.yaml
# Prediction of DNN features
$ python cv_predict_feature_fastl2lir.py config/deeprecon_cv_pyfastl2lir_alpha100_vgg19_allunits.yaml
# Evaluation
$ python cv_evaluation.py config/deeprecon_cv_pyfastl2lir_alpha100_vgg19_allunits.yaml- Horikawa and Kamitani (2017) Generic decoding of seen and imagined objects using hierarchical visual features. Nature Communications 8:15037. https://www.nature.com/articles/ncomms15037
- Shen, Horikawa, Majima, and Kamitani (2019) Deep image reconstruction from human brain activity. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1006633