Gibdos Talks FOSS

A Beginners Guide To Selfhosting Part 3

In Part 3 of our Beginners Guide To Selfhosting, we will install Docker on our VPS. Docker is a container system, that seperates your services (and their dependencies) from your servers operating system. This makes it easier to run different services that might require components that are incompatible with other services components.

We will also install our first Docker container: Caddy - A reverse proxy server that will connect your sub-domains to your service (music.domain.com -> Navidrome, book.domain.com -> AudioBookShelf, etc.).

Install Docker

To install Docker, open a Terminal window, connect to your VPS and enter the following commands

# Add the official Docker GPG key
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc


# Add the repository
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# Check if docker is running correctly
sudo docker run hello-world

If you Docker installation was successfull you should see

Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
17eec7bbc9d7: Pull complete
Digest: sha256:6dc565aa630927052111f823c303948cf83670a3903ffa3849f1488ab517f891
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

in your Terminal.

The compose.yml

With Docker you can use a compose.yml file for each of your services. It contains instructions on how to

An example compose.yml looks like this

services:
  service_name:
    container_name: container_name1
    image: service:tag
    restart: unless-stopped
    ports:
      - "8080:80"
    volumes:
      - ./outside_folder:/inside_folder
    environment:
      - TZ=Europe/Berlin
    depends_on:
      service_name2:
  
  service_name2:
    container_name: container_name2
    image: another_service:tag
    restart: unless-stopped
    ports:
      - "8081:80"
    volumes:
      - ./outside_folder2:/inside_folder2

With the .yml file format, the spaces are important. Each sub-setting is indented by 2 spaces. If a compose.yml does not work, make sure it's not a missing space somewhere.

Let's go through each part in more detail.

services
All compose.yml files begin with this line. It is mandatory and groups all underlying services together.

service_name & service_name2
The name of each individual service. Services can refer to each other with their respective name.

image
All services require an image. This image is a mini virtual machine that contains the services itself, as well as the bare minimum for it to function.

restart
Tell Docker if the container should (re-) start automatically after an error or a reboot of the VPS.

ports
Each service listens to a specific port inside its own container. Since many services use the same port (80 for http, 443 for https, etc.), this part tells Docker which port on the VPS itself (8080, 8081) will be used to link to the port inside the container (80).

A reverse proxy (t.ex. Caddy) will then use an internet adress (music.yourdomain.com) and forward it to the servers port 8080, which connects to the service containers internal port 80. Think of it as a chain like this

music.yourdomain.com → YOUR_IP:80 (Caddy) → YOUR_IP:8080 → service_name:80

volumes
Since a docker container is a miniature virtual machine, it contains its own folder structure, apart from your server. Without telling Docker where these files and folders should be placed on your servers filesystem, anything inside a container would be deleted and lost once the container is stopped.

In our example file, the service expects a folder named /inside_folder and we tell Docker to create an outside_folder next to the compose.yml file and save all data from the containers /inside_folder there.

environment
Optional settings for a service, like the timezone (TZ) or database access information. Depends on the service / image.

depends_on
If a webservice requires a database, you can tell Docker to only start the webservice if and when the database is up and running.

There are many more settings you can find inside a compose.yml and going through all possibilities is far outside the scope of this guide. The onces mentioned here are the bare minimum required to get a service container up and running.

If you are curious, you can find all possible settings in the Compose Reference Documentation.

Your Docker Directories

To keep my Docker container organized I usually use the /opt/ folder on my VPS. This folder is meant for optional programs that do not come with Ubuntu Server itself.

To create the docker directory inside /opt/ do the following

# Create a new directory inside /opt/ named docker
sudo mkdir /opt/docker

All services will live entirely inside their respective subfolders. We will use compose.yml files for each of our services and ensure all files and folders will be contained within their respective services subfolder.

This keeps our services organized and makes them easier to backup.

Install Caddy Reverse Proxy

Now that Docker is running and our folder structure is organized, we can install our very first Docker container: Caddy.

Caddy is the first thing people will connect with when entering your internet address into their browser. It will connect your services to their respective sub-domains and ensure a fast and secure webservice for all your Docker containers.

To install Caddy, connect to your VPS and do the following

sudo mkdir /opt/docker/caddy
cd /opt/docker/caddy

# Create the service instructions
nano compose.yml

In .yml files, spaces are important. As such, make sure that you copy the following content exactly.

services:
  caddy: # Name of your service
    container_name: caddy # Name of the container
    image: caddy:alpine # Caddy image from Docker Hub
    restart: unless-stopped # Service will always attempt to restart, unless manually stopped
    ports:
      - "80:80" # Port 80 is the default http port
      - "443:443" # Port 443 is the default https port
      - "443:443/udp"
    volumes:
      - ./conf:/etc/caddy # Contains your Caddyfile and other configuration
      - ./site:/srv # Can be used to directly host HTML, CSS, etc.
      - ./caddy_data:/data # Contains your HTTPS certificates
      - ./caddy_config:/config # Contains further Caddy configurations

Paste it into your open compose.yml on your VPS with CTRL+SHIFT+V. Save your file with CTRL+O -> Enter and close nano with CTRL+X.

To start your new service, we will need to pull the image (caddy:alpine) from Docker Hub. To do that, just enter

# pull downloads the latest version of your tag (:alpine)
sudo docker compose pull

Once the image is pulled you can start your service with

# -d makes it run in the background
sudo docker compose up -d

This will create the folders caddy_config, caddy_data, conf and site.

But since we haven't set up a Caddyfile for our services, we'll shut the container down again with

sudo docker compose down

To create our Caddyfile, enter the following

cd conf
sudo nano Caddyfile

Then paste the following content into your Caddyfile

{
        email YOURMAILADRESS
}

(security_headers) {
  header {
    Content-Security-Policy "default-src 'self' 'unsafe-inline' 'unsafe-eval'"
    Strict-Transport-Security "max-age=31536000; includeSubDomains"
    X-Content-Type-Options "nosniff"
    X-Frame-Options "SAMEORIGIN"
    Referrer-Policy "no-referrer"
    X-XSS-Protection "1; mode=block"
    Permissions-Policy "camera=(), microphone=(), geolocation=(), accelerometer=(), autoplay=(), fullscreen=(), gyroscope=(), interest-cohort=(), magnetometer=(), payment=()"
    -Server
    -x-powered-by
  }
}

yourdomain.com {
  file_server
  encode zstd gzip
  import security_headers
}

sub.yourdomain.com {
  reverse_proxy XXX.XXX.XXX.XXX:PORT {
   header_up X-Real-IP {remote_host}
  }
  encode zstd gzip
  import security_headers
}

Here's a detailed explanation of each part

email
For your HTTPS certificate

security_headers
A list of best practise security headers, which help protect your website and visitors.

yourdomain.com
This is for your HTML / Websites in the /opt/docker/caddy/site/ folder. It's entirely optional. If you want to link to a service instead, just use the same structure as sub.yourdomain.com.

sub.yourdomain.com
Your sub-domain. Connects your service to the corresponding docker container with the line reverse_proxy XXX.XXX.XXX.XXX:PORT where XXX is your IPv4 and PORT is the left side of the -ports: section in your compose.yml. If your service has 3000:80, you would use 3000 here.

header_up X-Real-IP {remote_host}
Makes sure the service has access to the visitors actual IP, instead of the internal Docker IP, which Caddy proxies to.

encode zstd gzip
Uses standard compression for data and bandwidth.

import security_headers
Adds the aforementioned security_headers to your service.

Once you've added your first entries into the Caddyfile, we can start the Docker container again with

cd /opt/docker/caddy
sudo docker compose up -d

Manage your running containers

Here's a list of commands to manage your running containers

# Lists all currently running container
sudo docker ps

# Show the logs of a running container
sudo docker logs CONTAINER_NAME # You can try it with caddy, t.ex.

# Run a command inside a Docker container
sudo docker exec -it CONTAINER_NAME COMMAND

# Update your container images (use where compose.yml is located)
sudo docker compose pull

# (Re-) Start container with new image
sudo docker compose start -d

# Reboot container (in case of error)
sudo docker compose down && sudo docker compose up -d

# Shutdown container
sudo docker compose down

Backup & Restore a Service

Backup

It is important to regularly backup your services. The easiest way is to create a .tar.gz archive file of the entire service container and copy it to your local PC. From there you can backup the archive to an external drive (and ideally an additional off-site location).

To backup a services, connect to your VPS through your terminal and follow the commands

# Change to your service folder
cd /opt/docker/SERVICE_FOLDER

# Stop the running service
sudo docker compose down

# Switch to /opt/docker
cd ..

# Create a compressed archive of your service in your HOME folder
sudo tar -czvf ~/SERVICE.tar.gz SERVICE_FOLDER

# Start your service again
cd SERVICE_FOLDER
sudo docker compose up -d

Once you have the archive in your HOME folder on your VPS, you need to transfer it to your local computer. Open a new terminal window and follow these commands

## NAME refers to the Host NAME you chose in your ~/.ssh/config
# Copy the file through SSH to your current local folder
scp NAME:~/SERVICE.tar.gz ./

Restore

To restore your service with your archive, use the following commands

### If SERVICE.tar.gz is no longer on your VPS
## on your local machine
scp /PATH/TO/YOUR/SERVICE.tar.gz NAME:~/

## on your VPS
# Change to your docker service directory
cd /opt/docker

# Make sure the old service is no longer running with 
sudo docker ps

# Delete your old service folder
rm -rf SERVICE_FOLDER

# Restore your archive file
sudo tar -xf /PATH/TO/SERVICE.tar.gz -C /opt/docker/

# Start the restored service
cd SERVICE_FOLDER
sudo docker compose up -d

# Check if the service is running again
sudo docker ps

It's important to make sure that your backups and restores work as intended. As such, I highly recommend going through it at least once, when setting up a new service.

You can waste many hours by having to transfer multiple GBs of backup data around, only to end up with a broken backup / restore.

This backup / restore method will always include your entire service. There are more advanced backup solutions, which only transfer new and changed files, but I will go over these in my Intermediates Guide To Selfhosting.

And with that we are finished for today. In Part 4 we will set up your Domains and Sub-Domains, so you (and anyone else) can actually reach your services through their browser.

A Beginners Guide To Selfhosting Part 4