diff --git a/README.md b/README.md index 50ae79c..d53e556 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,5 @@ # DeepForge example projects +This repository contains example projects for DeepForge v2. For legacy DeepForge examples, check out the [v1 branch](https://github.com/deepforge-dev/examples/tree/v1.x). ## Loading Examples First, you will need to clone the project locally: @@ -7,6 +8,16 @@ First, you will need to clone the project locally: git clone https://github.com/deepforge-dev/examples ``` +If you do not have a deployment of deepforge handy, you can quickly spin up one using `docker-compose`: +``` + $ cd examples + $ docker-compose up +``` +After that, DeepForge can be used from the browser at http://localhost:8888. + +**Note**: Further instruction on deployment can be accessed [here](https://deepforge.readthedocs.io/en/latest/deployment/quick_start.html). + + Then you can load examples into DeepForge as follows: 1. Select "Import Project" diff --git a/cifar10/cifar10.webgmex b/cifar10/cifar10.webgmex index 2aa347a..b485751 100644 Binary files a/cifar10/cifar10.webgmex and b/cifar10/cifar10.webgmex differ diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..b0f83ca --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,19 @@ +--- +version: "3" +services: + mongo: + image: mongo + volumes: + - "$HOME/.deepforge/data:/data/db" + server: + environment: + - "MONGO_URI=mongodb://mongo:27017/deepforge" + - "NODE_ENV=default" + image: deepforge/kitchen-sink:latest + ports: + - "8888:8888" + - "8889:8889" + volumes: + - "$HOME/.deepforge/blob:/data/blob" + depends_on: + - mongo diff --git a/galaxy-zoo/README.md b/galaxy-zoo/README.md new file mode 100644 index 0000000..b8cf88e --- /dev/null +++ b/galaxy-zoo/README.md @@ -0,0 +1,25 @@ +# Galaxy Zoo Example +This is a simple example using the Galaxy Zoo dataset (also seen in [the Galaxy Challenge](https://www.kaggle.com/c/galaxy-zoo-the-galaxy-challenge) on [kaggle](www.kaggle.com)). This example also demonstrates how a large dataset can be imported into DeepForge and streamed to the neural network during training. + +The example project contains the training data and can be downloaded from [here](https://deepforge.s3.amazonaws.com/galaxy-data.webgmex?AWSAccessKeyId=AKIAISTYNTKQZLV6IVLQ&Expires=1633932738&Signature=8%2BlF1x%2FsF0fGE0K0DGBLjG1R9v0%3D). Or, if you prefer downloading from the console: + +``` +wget -O galaxy-data.webgmex https://deepforge.s3.amazonaws.com/galaxy-data.webgmex?AWSAccessKeyId=AKIAISTYNTKQZLV6IVLQ&Expires=1633932738&Signature=8%2BlF1x%2FsF0fGE0K0DGBLjG1R9v0%3D +``` + +In this example, we are simply cropping a 106x106 section of the image and training either a feed-forward neural network or a convolutional neural network. The architecture for the convolutional neural network is based on the paper by Sandor Dieleman. The entire citation for the paper is given below. + +``` +@article{dieleman2015rotation, + title={Rotation-invariant convolutional neural networks for galaxy morphology prediction}, + author={Dieleman, Sander and Willett, Kyle W and Dambre, Joni}, + journal={Monthly notices of the royal astronomical society}, + volume={450}, + number={2}, + pages={1441--1459}, + year={2015}, + publisher={Oxford University Press} +} +``` + +This example depends on the [deepforge-keras](https://github.com/deepforge-dev/deepforge-keras) extension. diff --git a/imdb/README.md b/imdb/README.md new file mode 100644 index 0000000..9691726 --- /dev/null +++ b/imdb/README.md @@ -0,0 +1,2 @@ +# Overview +This is a simple example of a convolutional neural network for sentiment analysis of IMDB movie reviews. diff --git a/imdb/imdb.webgmex b/imdb/imdb.webgmex new file mode 100644 index 0000000..81a3e72 Binary files /dev/null and b/imdb/imdb.webgmex differ diff --git a/iris/README.md b/iris/README.md deleted file mode 100644 index 1723bf5..0000000 --- a/iris/README.md +++ /dev/null @@ -1,3 +0,0 @@ -# Neural networks and the Iris Dataset -In this example, we train a neural network to classify the species of plants based on a couple physical characteristics. More detailed information about this dataset can be found [here](https://archive.ics.uci.edu/ml/datasets/Iris). - diff --git a/iris/iris.webgmex b/iris/iris.webgmex deleted file mode 100644 index 8a7aa51..0000000 Binary files a/iris/iris.webgmex and /dev/null differ diff --git a/mnist/README.md b/mnist/README.md index 9fe8e14..c8d954b 100644 --- a/mnist/README.md +++ b/mnist/README.md @@ -1,15 +1,2 @@ # MNIST An example which trains a classifier on the [MNIST](http://yann.lecun.com/exdb/mnist/) data set. - -This example uses [torchnet](https://github.com/torchnet/torchnet) to showcase the flexibility of both DeepForge and torchnet by extending their [example code](https://github.com/torchnet/torchnet/blob/master/example/mnist.lua). - -# Installation -Assuming torch is already installed, torchnet can easily be installed with *luarocks* -``` -luarocks install torchnet -``` - -To run this example you must also have the `mnist` package as well: -``` -luarocks install mnist -``` diff --git a/mnist/mnist.webgmex b/mnist/mnist.webgmex index 811f1a0..99d01ac 100644 Binary files a/mnist/mnist.webgmex and b/mnist/mnist.webgmex differ diff --git a/redshift-tutorial/README.md b/redshift-tutorial/README.md new file mode 100644 index 0000000..6c90760 --- /dev/null +++ b/redshift-tutorial/README.md @@ -0,0 +1,7 @@ +# Redshift Tutorial + +This collection of Deepforge projects provides an example of how to build and use a project centered around using Convolutional Neural Networks (CNNs) to classify or analyze astronomical images. In this example, the analysis is focused on predicting [cosmological redshift](https://en.wikipedia.org/wiki/Redshift) using a simplified version of the approach used in [Pasquet et. al. (2018)](https://arxiv.org/abs/1806.06607). + +The Redshift-Application project provides a collection of potentially useful pipelines for a user that wants to create and utilize a redshift estimator. This project may be useful as a starting point for more specialized projects. More in depth descriptions of the available pipelines can be found [here](https://deepforge.readthedocs.io/en/latest/examples/redshift.html). + +The Redshift-Tutorial project contains a sequence of increasingly complex example pipelines that provide guidance on how to build a project from scratch. This project is geared toward new users of the deepforge framework and has an accompanying written tutorial, which can be found [here](https://deepforge.readthedocs.io/en/latest/examples/rs-tutorial.html). \ No newline at end of file diff --git a/redshift-tutorial/Redshift-Application.webgmex b/redshift-tutorial/Redshift-Application.webgmex new file mode 100644 index 0000000..32051bf Binary files /dev/null and b/redshift-tutorial/Redshift-Application.webgmex differ diff --git a/redshift-tutorial/Redshift-Tutorial.webgmex b/redshift-tutorial/Redshift-Tutorial.webgmex new file mode 100644 index 0000000..8e997f1 Binary files /dev/null and b/redshift-tutorial/Redshift-Tutorial.webgmex differ diff --git a/redshift/README.md b/redshift/README.md new file mode 100644 index 0000000..afbe78d --- /dev/null +++ b/redshift/README.md @@ -0,0 +1,32 @@ +## REDSHIFT +Redshift is a spectral measurement of how far from the earth the galaxy is moving. The [sdss](https://www.sdss.org/) is a survey conducted by a telescope in the night sky that takes pictures of various regions in the sky. In this example, we present the preprocessing techniques for SDSS images as well as the code to the model presented by this [paper](https://arxiv.org/abs/1806.06607). If you use this model, please cite the paper: +``` +@ARTICLE{2019A&A...621A..26P, + author = {{Pasquet}, Johanna and {Bertin}, E. and {Treyer}, M. and {Arnouts}, S. and + {Fouchez}, D.}, + title = "{Photometric redshifts from SDSS images using a convolutional neural network}", + journal = {\aap}, + keywords = {galaxies: distances and redshifts, surveys, methods: data analysis, techniques: image processing, Astrophysics - Instrumentation and Methods for Astrophysics}, + year = "2019", + month = "Jan", + volume = {621}, + eid = {A26}, + pages = {A26}, + doi = {10.1051/0004-6361/201833617}, +archivePrefix = {arXiv}, + eprint = {1806.06607}, + primaryClass = {astro-ph.IM}, + adsurl = {https://ui.adsabs.harvard.edu/abs/2019A&A...621A..26P}, + adsnote = {Provided by the SAO/NASA Astrophysics Data System} +} +``` +Since, the exported webgme project is quite large, you can download it from [here](https://vanderbilt365-my.sharepoint.com/:u:/g/personal/umesh_timalsina_vanderbilt_edu/EZ0ME7hGyUlHqu6n1YmqrIsBdRmfQaKGB1yE7rpHAw5M-w?e=azaP34) and run the pipelines. There are three pipelines as shown in the screen-shot below: + +![pipelines](../screenshots/redshift-pipelines.png) + +The model designed in deepforge is as follows: + +![inceptionModel](../screenshots/inceptionblocks.png) + + +For more details on the implementation please visit this github [repo](https://github.com/umesh-timalsina/redshift). diff --git a/screenshots/inceptionblocks.png b/screenshots/inceptionblocks.png new file mode 100644 index 0000000..1ad873c Binary files /dev/null and b/screenshots/inceptionblocks.png differ diff --git a/screenshots/redshift-pipelines.png b/screenshots/redshift-pipelines.png new file mode 100644 index 0000000..6558395 Binary files /dev/null and b/screenshots/redshift-pipelines.png differ diff --git a/starterkit/README.md b/starterkit/README.md deleted file mode 100644 index 57d943c..0000000 --- a/starterkit/README.md +++ /dev/null @@ -1,28 +0,0 @@ -# DeepForge Starter Kit -The deepforge starter kit provides the following example pipelines operations demonstrating various features of deepforge including: -- hello world example -- using operation attributes -- plotting operation results (in real time) -- displaying tensors as images -- operation references -- multi-threading - -When viewing a pipeline, the details of any operation (with the exception of Input/Output operations) can be inspected by clicking on the operation and then selecting the code button (labeled with ``) in the top right. - -Each pipeline can also be executed within deepforge using the play button option in the floating action button (floating button in the lower right). The play button is available after hovering on the button in the lower right. - -## Loading Starter Kit -First, you will need to clone the project locally: - -``` - git clone https://github.com/deepforge-dev/examples -``` - -Then you can load examples into DeepForge as follows: - -1. Select "Import Project" -2. Name the new project -3. Select the file to import ('starterkit.webgmex') -4. Click "Create" to create a new project from the example! - -![screenshot](../screenshots/steps.gif "") diff --git a/starterkit/starterkit.webgmex b/starterkit/starterkit.webgmex deleted file mode 100644 index 9fcb4fd..0000000 Binary files a/starterkit/starterkit.webgmex and /dev/null differ diff --git a/xor/README.md b/xor/README.md deleted file mode 100644 index dd4d51e..0000000 --- a/xor/README.md +++ /dev/null @@ -1,20 +0,0 @@ -# XOR example -This is one of the canonical examples used in deep learning. This has been adapted from the [torch example](https://github.com/torch/nn/blob/master/doc/training.md). - -This example trains a neural network to predict the XOR operation on two numbers where the number is true if it is < 0 and false if it is > 0, ie: - -``` --1, -2 => false --1, 2 => true -1, 2 => true -``` - -This can be rewritten as - -``` --1, -2 => (> 0) --1, 2 => (< 0) -1, 2 => (< 0) -``` - -As you can see this, happens to correspond to multiplication between the input numbers and then checking the sign for the "truthyness". This approach is used in this example. diff --git a/xor/xor.webgmex b/xor/xor.webgmex deleted file mode 100644 index 2385194..0000000 Binary files a/xor/xor.webgmex and /dev/null differ