How to start on a new machine with a "Settings repo" - Hypergol

I have a small repo with all my config files and a couple of scripts. On any new instance, I can clone this repo and execute the scripts and get my usual environment and shortcuts. Because I have GitHub on my local machine and I set up ForwardAgent (see GCP instructions), I can access my repo’s from the remote machine as well which makes cloning really easy. The directory tree of the repo looks like this:

├── .bashrc
├── .inputrc
├── .jupyter
│   ├── custom
│   │   └── custom.css
│   └── jupyter_notebook_config.py
├── data
│   ├── Scratch.ipynb
│   ├── config.py
│   ├── make_venv.sh
│   ├── requirements.txt
│   └── start_jupyter.sh
├── git-completion.bash
├── make_settings.sh

Let’s go through the contents of each file. These are just my examples, customise each as you feel to maximise convenience.

.bashrc

I add the following two lines to .bashrc

export PS1="\u@\h:\w$ "
source ~/git-completion.bash

As you can see the git-completion.bash is in the repo as well, you can get a copy from here. The other line formats the prompt.

.inputrc

The entire content of the file is:

set completion-ignore-case on
set show-all-if-ambiguous off
TAB: menu-complete

This makes tab-completion case insensitive and if there are multiple options, pressing TAB repeatedly cycles through them.

custom.css

.notebook_app .container {
  width: 75% !important;
}

.terminal_app .container {
  width: 99% !important;
}

div.cell.selected {
  border-left-width: 1px !important;
}

This is where the default width of the notebook is selected, change the 75% as you wish. The other one changes the width of the terminal.

jupyter_notebook_config.py

I only set the following values.

c.NotebookApp.open_browser = False
c.NotebookApp.ip = '0.0.0.0'
c.NotebookApp.port = 8881

The port number must be the same that you will choose in the firewall option in the GCP Instance setup. If you plan to use multiple notebook servers on the same machine, you need different ports for them.

Scratch.ipynb

import sys
sys.path.insert(0, '..')
%matplotlib inline
from config import *

I use this file as a scratchpad. It loads all the usual config like import numpy as np and many others like matplotlib. It also initialises matplotlib. If I need a new notebook, I copy-paste these lines, and I don’t pollute my notebooks with imports.

make_venv.sh

python3 -m venv .venv
source .venv/bin/activate
pip3 install --upgrade pip
pip3 install setuptools==47.1.1
pip3 install wheel
pip3 install -r requirements.txt
python3 -m spacy download en_core_web_sm

This creates the virtual environment for the notebook server (not for the Hypergol project, that has its own env). It also downloads a spacy model because I use it regularly, and this is the best place to do so. Don’t forget to make the file executable with chmod u+x make_venv.sh before you put it in your repo.

This also uses requirements.txt, add any packages you regularly use like jupyter, numpy, scipy and many others. VM Instances have a very good network connection, so installing even a lot of packages will be very fast.

start_jupyter.sh

source ~/data/.venv/bin/activate
screen -dmS jupyter && screen -S jup -X stuff "source ~/data/.venv/bin/activate && cd ~/data && jupyter notebook^M"
sleep 5
jupyter notebook list

This starts the notebook server in a screen session in the background. In case you lose connection to the instance your notebook server still runs. It also shows the token you need to use in your local browser to access the notebook server. If you want to see the logs of the server you can join from a remote terminal with screen -x jupyter (make sure you don’t close the window with ctrl-D because that ends the screen session). Also, don’t forget to make this file executable with chmod.

make_settings.sh

cp .bashrc ..
cp .gitconfig ..
cp .inputrc ..
cp -r .jupyter ..
cp -r data ..
cp git-completion.bash ..

This just copies everything relevant to the root folder. Again, don’t forget chmod. After executing the script, close the terminal and open a new one to make sure the config files are applied.

So the steps are:

$ ./make_settings.sh
# close and open a new terminal
$ cd data
$ ./make_venv.sh
$ ./start_jupyter.sh

In your local browser go to https://ip_address:port/tree/Scratch.ipynb and use the token you see in the terminal. And you are ready to go!

Join our Discord server for more tips and tricks.