Install Applications
The software needed to run ggKbase includes: ruby, rbenv, rbenv-vars, sidekiq, puma, redis, mariadb, elasticsearch.
rbenv (ruby env mgr) + ruby
# Full doc: https://github.com/rbenv/rbenv git clone https://github.com/rbenv/rbenv.git ~/.rbenv echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc echo 'eval "$(rbenv init -)"' >> ~/.bashrc source ~/.bashrc # To verify type rbenv
Install ruby-build plugin, useful for install / uninstall commands
git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build
Install rbenv vars plugin, needed for environmental variables
git clone https://github.com/rbenv/rbenv-vars.git $(rbenv root)/plugins/rbenv-vars
rbenv install 2.7.5 # as of 2/7/2023 rbenv rehash rbenv local 2.7.5 # set it to be global gem update --system # this will install bundler
pyenv (python env mgr) + python
git clone https://github.com/pyenv/pyenv.git ~/.pyenv echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.bashrc # install python pyenv install 3.6.5 pyenv local 3.6.5 pip install --upgrade pip
keychain
sudo apt install keychain
# generate a private key from your machine
mkdir $HOME/.ssh/
chmod 0700 $HOME/.ssh/
ssh-keygen
# Append the following to your ~/.bashrc file
vim $HOME/.bashrc
------
/usr/bin/keychain $HOME/.ssh/id_rsa
source $HOME/.keychain/$HOSTNAME-sh
------
System Libraries
System libraries
These are installed by the system administrator if you don’t have SUDO access
sudo apt install libtokyocabinet-dev # needed for tokyocabinet sudo apt install libxml2-dev # needed for libxml-ruby sudo apt install libmysqlclient-dev # needed for mysql2 sudo apt install libcurl4-openssl-dev # needed for patron sudo apt install sqlite3 libsqlite3-dev # needed for sqlite3 sudo apt install libbz2-dev # needed for tokyocabinet # all in one line sudo apt install libtokyocabinet-dev libxml2-dev libmysqlclient-dev libcurl4-openssl-dev sqlite3 libsqlite3-dev libbz2-dev -y
redis
sudo apt install redis-server # test redis redis-cli ping # Expected response is PONG
mariadb (a variant of mysql)
# install 10.2 on 18.04 (bionic)
sudo apt-get install software-properties-common
sudo apt-key adv --fetch-keys 'https://mariadb.org/mariadb_release_signing_key.asc'
sudo add-apt-repository 'deb [arch=amd64,arm64,ppc64el] http://mirror.rise.ph/mariadb/repo/10.2/ubuntu bionic main'
sudo apt update
sudo apt install mariadb-server
Set up database and import data
mysql -uroot CREATE USER 'ggdbmgr'@'localhost' IDENTIFIED BY 'banfieldlab'; GRANT ALL PRIVILEGES ON * . * TO 'ggdbmgr'@'localhost'; FLUSH PRIVILEGES; create database ggkbase_development; exit #exit database scp gg:/home/bcthomas/.... # NEED NEW TRAINING DB gunzip NEW TRAINING DB mysql -uroot ggkbase_development < NEW TRAINING DB
elasticsearch (search engine)
Best is to download the latest version of Elasticsearch tar xvf elasticsearch-x.x.x.tar.gz cd elasticsarch-x.x.x # Edit the configuration files they are vim config/elasticsearch.yml # change cluster application name # elasticsearch.yml and vim config/jvm.options # increase memory to 31g # Install Java if it has not been installed yet sudo apt install default-jre # Fire up elasticsearch bin/elasticsearch
Set up ggKbase instance
git clone project
git clone git@github.com:banfieldlab/ggkbase.git # if you didn't set up keychain above, add the private key to the SSH agent eval `ssh-agent -s` ssh-add -k ~/.ssh/id_rsa
git update local master branch
# Assuming you are in ggkbase development directory git pull origin master # If/when you run into conflict over Gemfile.lock, run the following before pulling git checkout Gemfile.lock
install ruby gems
bundle config set path 'vendor/bundle' bundle config gems.contribsys.com a3ebaba2:eed19071 # for installing sidekiq bundle install
add temporary folders
# in ggkbase/ folder, add the following temporary folders mkdir tmp mkdir data mkdir log
set up rbenv-vars
cp .rbenv-vars.example .rbenv-vars #import variables to change are # Secret key base SECRET_KEY_BASE # must be unique from all other keys # Database configurations DB_USERNAME= DB_PASSWORD= DB_HOST= DB_POOL= DB_DEVELOPMENT= DB_PRODUCTION= # ES ES_HOST= ES_PORT=9200 ES_INDEX_NAME= ES_DOCUMENT_TYPE=
update database
RAILS_ENV=production bin/rails db:migrate
install yarn
To manage the javascript packages
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list sudo apt update sudo apt install yarn
launch ggKbase server
# Start redis ./redis_init.sh start # When started successfully: Starting redis... all set # Start sidekiq RAILS_ENV=production bundle exec sidekiq # Check whether sidekiq started successfully tail -f log/sidekiq.log # The last line should be: Sidekiq Pro 3.3.3, commercially licensed. Thanks for your support! # Start puma (application server) RAILS_ENV=production bin/rails server
install python plugins for ES import
If all goes well, you should be able to load up the site at localhost:3000.
# go to the elasticsearch import script directory cd ggkbase/script/elasticsearch pip install -r requirements.txt
elasticsearch indexing
# Create the ES index defined in rbenv-vars RAILS_ENV=production bin/rails es:create:index # To index the whole database RAILS_ENV=production bin/rails es:index