Requirements:

• Python3 (Anaconda)
• MySQL
• Java
• Bower

Step 1: Install MySql and configure database

1. Make sure MySql is installed. (We use version 5.6.33)
2. Create a new database for scke data with create database scke
3. Create the necessary tables. See table configuration notes in database_management.py for instructions.

Step 2: Install Java

Make sure you have Java installed!

Step 3: Retrieve the source code

Clone (or fork!) the project and move into the main directory

1. git clone https://github.com/hclent/Science-Citation-Knowledge-Extractor.git
2. cd Science-Citation-Knowledge-Extractor.

Step 4: Get the javascript with Bower

A lot of the Javascript is already provided in the repo, but there are still a few packages you will need. bower install bower.json to get them.

Step 5: Download the supplementary files (You need these!)

These supplementary files are stored on CyVerse's Data Commons

The files in the Data Commons that you will need are:

• The py-processors BioNLP Server jar, processors-server.jar
  • Move this file to ~/anaconda3/envs/scke/lib/python3.4/site-packages/processors!!!
  • This jar is used to annotate the texts. It is initialized to use 10G of memory but you can increase by editing this line of multi_preprocess.py.
  • If you you are running SCKE with either large documents or on a large amount of documents 10G will not be enough memory for the jar file! We give SCKE 100G of memory to run it in production. If you experience problems, try increasing the memory to 25G, 50G, or more.
  • IF YOU SEE THE ERROR: 'NoneType' object has no attribute 'to_JSON', YOU NEED TO INCREASE THE MEMORY. This error indicates that the annotation process has failed to annotate the data and produce json.
• Our trained FastText word vectors file, scke_lower.vec

We also make use of nltk's stopwords. You can download these easily by entering the python shell and

import nltk
nltk.download()

Type d stopwords, download the stopwords, then q to quit and exit().

Step 6: Setting up Python

In order to set up the site, you will need the same Python environment that the site uses to run. Here are instructions for how to re-create our environment.

1. Install (Anaconda) for Python3
2. Create a Python environment with the repository's requirements.txt libraries (e.g conda create -n scke python=3.4 anaconda --file conda_requirements.txt)
3. Activate the environment (source activate scke)
4. Install the rest of the requirements in the environment with pip install -r pip_requirements.txt

Step 7: Creating and loading the config file

1. Next you will need to create a config file, that tells the files in the app where to look for the database and cached data, etc. For complete instructions, please see the example configExample.cfg.
2. Make sure that this line ofconfigapp.py is pointing to the correct location of your config file!
   • NB: Also while you're updating this, the line above it should also be pointing to the static directory. Since this is where the app is initialized, it can't use the config file yet.

Step 8: Run SCKE

Once you have accomplished steps 1-8, you are almost ready to run SCKE!

1. Make sure your conda environment is active (source activate scke)
2. Run python app.py

It takes a few minutes to start up since it must establish a connection with our processors-server.jar and load the scke_lower.vec file. You can tail -F .app.py to watch the log file go.

To view the website go to either 0.0.0.0:5000/ if you are running it locally or if you are running this on a vm go to <THE_IP_ADDRESS>:5000/

Debugging?

If you have any problems, feel free to contact me! But first, take a look instead of your .app.log file to see where things might have gone amiss.