This post aggregates several online resources to help Hypergol’s users to get going as fast as possible on GCP. See the end of the article for sources. The following steps will be discussed below:

  • Create a new Google project
  • Create a project-wide SSH key
  • Create a new cloud instance
  • Create fixed IP address and open ports
  • Prepare to log in for the first time
  • Create an extra disk and attach to the cloud instance
  • How to connect to the created instance
  • Set up on the new instance
  • Open a remote repository in VS Code

Google provides $300 credit for the first time the service is activated, which will cover your needs for a while. See pricing here, and you can make estimations of how far this will last with their calculator. It is vital to get used to use VM machines as throwaways and build habits that enable this (easy setup, backup everything to external disks and GitHub).

Create a new Google project

First, go to console.cloud.google.com and set up a new project. If you are on a free tier, you can only have four of these, so you need to delete one, to do that go to console.cloud.google.com/cloud-resource-manager. Be careful this operation is irrecoverable.

So back at the project dashboard: console.cloud.google.com. Click the current project’s name next to the Google Cloud Platform and click “New Project” on the top right of the pop-up window. Then follow the instructions.

Create a project-wide SSH key

If you don’t want to copy your public key each time you create a new instance, you can set up one to be copied over every time you create a new VM. Follow the steps here on GitHub: Generating a new SSH key and adding it to the ssh-agent. The article also tells you how to add the SSH key to your GitHub account if you need that as well.

Now we will take the public key and add to the project. Click on the hamburger menu (three lines in the top left corner) in the dashboard and select “Compute Engine” and then “Metadata”. On the “SSH Keys” tab click “Edit”, then “+ Add Item”. After that on your local machine execute the following shell command:

$ cat ~/.ssh/id_ed25519.pub

Make sure you use the .pub extension, and if you selected a custom filename at the key generation, use that. Copy the text (starting with ssh-ed25519 and ending with your email) into the “Enter public SSH key” textbox and click “Save”.

Create a new cloud instance

Go back to “Compute Engine” and then “VM instances”. Click “Create instance” (“+” icon next to VM instances). Then go through the following steps:

  • Name the instance: Pick something memorable and hope it brings luck. :)
  • Select a region: for example, us-central1 (best to stick to the same with all of your assets as cross-region data traffic costs more).
  • Select a machine configuration: This cannot be changed later so small ones for experimentation, large ones for a heavy workload. Again, these should be off when they are not used and have a small hard drive, so they are relatively cheap.
  • Select boot disk: I recommend Ubuntu 20.04 LTS (at the time of this writing, make sure you do not select the minimal edition or you won’t have git) as a default choice. The standard 10GB is a bit small; you can make it bigger to 100GB. Because instances can be created on a whim, you can consider using an SSD which is much more expensive but will help you with heavy I/O loads (edit: I can confirm that you need an SSD if you use Hypergol on a lot of cores). You can always back up to the secondary disk that we will make in the next point. The disk costs are charged even if the machine is off, so be careful here.
  • Setup firewall: THIS IS VERY IMPORTANT! If you miss this, you won’t be able to connect to the instance’s notebook server from your browser.
  • Make your machine preemptable: Click on the blue “Management, Security…” link and under “Availability policy” at “Preemptibility” switch to “On” in the dropdown.
  • Security: If you haven’t created a project-wide SSH key, place your current one under “Security”. (If you did create then make sure that the “Block project-wide SSH keys” is unchecked).
  • Keep boot disk: To keep the boot disk after you deleted the instance you need to check out “Boot disk Deletion Rule”. I would keep important data on a separate disk or in Google Cloud Storage and all relevant code in GitHub. This will give more flexibility because the whole project can move to different instances.
  • Create the instance:: Click “Create”.

MAKE SURE YOU TURN OFF THE INSTANCE WHEN YOU ARE FINISHED, OR IT WILL COST YOU DEARLY! You can click on the three dot’s next to the instances on the right and select “Stop”. This is also the place where you delete an instance with “Delete”. If you decided not to keep the boot disk, it will be deleted as well so back up anything important from it.

Create fixed IP address and open ports

Now that you have the instance you need to enable ports so your browser can connect to the notebook server running on the VM. To do that:

  • Under “VPC Network”, select “External IP addresses.
  • Change the type from Ephemeral to Static.
  • Under “VPC Network”, select “Firewall”.
  • Click “Create Firewall”.
  • Select a name.
  • At “Targets” select “All instances in the network” (Of course you can look for other settings here).
  • At “Source filters” select “IP ranges”.
  • At “Source IP ranges” select add “0.0.0.0/0”.
  • Under “Protocols and ports” tick “TCP” and select a port (I recommend 8881 but you can specify range).
  • Click “Create”.

Prepare to log in for the first time

Add the following to your ~/.ssh/config on your local machine. The IP address can be seen at “External IP” at “VM instances”. The user name can be seen at “Metadata” under “SSH Keys”. The machine can be any string you will connect to the VM with ssh machine.

Host <machine>
  HostName <ip address from above>
  User <username>
  IdentityFile ~/.ssh/id_ed25519
  ForwardAgent yes

If you want to use GitHub from the instance, the ForwardAgent yes will enable sending your credentials safely to the VM so you can connect to Github from there. When you develop on the instance, the repo will be on that machine, and you open it from there in VS Code. Then you will use the remote machine’s terminal to push your code to GitHub.

Create an extra disk and attach to the cloud instance

If you selected an SSD for your boot disk or you expect to store a lot of data, it makes sense to keep all that data on a separate disk. In this way, if you want a different VM (less CPU or memory), you can attach this disk to the new machine and keep working.

At “Compute Engineer” click “Disks” then “Create Disk” and select the size and the type of the disk. No need to be SSD but make it a sizeable (500GB default is a good starting point). Then go back to the instances click on the instance’s name you want to attach it to and click “Edit”. At the “Additional disks” click “+ Attach existing disk” and select the one you just created. Make sure the “Keep disk” option is selected in “Deletion Rule”. Then click “Save” at the very bottom to finalise the instance.

Continue with instructions in the following article: How To Mount Extra Disks on Google Cloud VM Instance

MAKE SURE YOU SELECT THE RIGHT DISK, OR YOU WILL LOSE ALL INFORMATION AT FORMATTING

The lsblk command’s output shows the correct disk.

Set up on the new instance

Start the machine if it is not running yet. Login to the machine with:

$ ssh machine

where the "machine" is what you selected for the Host field in ~/.ssh/config. If you haven’t chosen a passphrase (pressed enter at the key generation), you will be taken straight to a remote terminal. In the terminal, execute the following commands:

$ sudo apt update
$ sudo apt install python3 python3-dev python3-venv python3-pip

This will install the latest python on your VM. For the next step, I have a “settings” repo in my GitHub account that is detailed in this post Settings repo. I recommend to follow the instructions there to create your own.

Open a remote repository in VS Code

To develop efficiently, the best workflow is to open a remote repository in VS Code, edit the code and use a remote terminal for git commands. To do so:

  • Install the Remote - SSH extension in VS Code.
  • Click on the newly installed Monitor icon which will show the SSH targets in the sidebar.
  • Click “+”.
  • Use the ssh machine where the machine is the same what you set up in ~/.ssh/config.
  • Click the “Add directory” icon next to the server’s name; this will open a new VS Code window.
  • Click “Open Folder” and wait for sync.
  • Select the correct directory.

And you are good to go. Don’t forget to commit your changes and push to GitHub as the server might be deleted and your progress lost.

Join our Discord server for more tips and tricks.

Sources: