Automating Docker Volume Backups

Automating Docker Volume Backups

Backing up production databases regularly is very important. I am self-hosting Leanote, an open-source note taking application server and that required some kind of automated daily backups.

Docker Background

Docker stores volumes in the /var/lib/docker/volumes directory. The naming convention for docker volumes is <directory name>_<volume name> where the directory name is the name of the directory containing docker-compose file and volume is the volume name as specified in the docker-compose file. On Linux based systems, each volume directory is directly accessible from a root account. This makes for a simple backup process.

Automating Docker Volume Backups

To backup a volume, we can simply compress the volume directory using tar and then back up the archive file to version control1.

To set up automated backups of your important data:

  1. Copy and paste the script below to a new backup.sh file in your ~/backups directory.
  2. Run git init inside your backups directory (and set up a remote link for external backups)
  3. Run crontab -e and append the following line: 0 1 * * * /bin/bash /home/pi/backups/backup.sh This runs our backup script daily at 1am.

Backup Script

#!/bin/bash
#Purpose: Backup docker container/s
#Version 1.0

# BASIC CONFIG START
items=(mongo)                           #space separated list of words. Used in file names only.
vol_names=(leanote_data)                #space separated list of volume names. Same order as items array.

DESDIR=/home/pi/backups                 # backup directory

# BASIC CONFIG END

# CUSTOMIZE THESE
TIME=`date +%m-%d-%y-%H-%M-%S`
FILENAME=$TIME-BACKUP.tar.gz
SRCROOT=/var/lib/docker/volumes
# CUSTOMIZE END

cd $DESDIR
for i in "${!items[@]}"; do
  echo "[$i]: Backing up ${items[$i]} (Volume: ${vol_names[$i]}) -------------------------- "
  ITEM=${items[$i]}
  SRCDIR=$SRCROOT/${vol_names[$i]}
  DIR=$DESDIR/$ITEM/$ITEM-$FILENAME
  echo "     Source:      $SRCDIR"
  echo "     Destination: $DIR"
  sudo tar -cpzf $DIR $SRCDIR
  echo "Content Listing (and integrity test):"
  tar -tzf $DIR
  git add $DIR
  git commit -m "$ITEM backup $TIME"

done

# Push all commits at the end
git push

This script compresses a given volume, moves the resulting archive to a subdirectory in backups and commits that file to version control. You can use this same script to backup multiple volumes by adding more elements to the items and vol_names arrays.

Congratulations! You can now rest assured that your data is backed up automatically. To confirm backups work, check your git repository or your local mail server. Cron sends output logged to STDOUT to the user executing the script (pi@raspberrypi). If your Cron logs show mail delivery errors, then you need to install postfix.

Access Cron emails using the mutt command (install if unavailable). Mutt provides a simple way to check the script outputs and confirm it is working as expected.

Do not stop here! Try this script in a non-production environment and restore a backup of some test data (see next section).

Backing up to AWS

See the following modified script to backup to AWS instead. YOu can set up a lifecycle rule to automatically delete backups older than 31days. Some sort of lifecycle is required as to not exceed the free usage limits.


#!/bin/bash
#Purpose: Backup docker container/s
#Version 1.1
#START

# BASIC CONFIG START
items=(influxdb)                           #space separated list of words. Item is descriptive, used in file names only.
vol_names=(influxdb)                #space separated list of volume names. Same order as items array.

DESDIR=/home/daniel/backups/ubuntu                 # backup directory

# BASIC CONFIG END

# CUSTOMIZE THESE
TIME=`date +%Y-%m-%d-%H-%M-%S`
FILENAME=$TIME-BACKUP.tar.gz
SRCROOT=/var/lib/docker/volumes

pushd $DESDIR

for i in "${!items[@]}"; do
  echo "[$i]: Backing up ${items[$i]} (Volume: ${vol_names[$i]}) -------------------------- "
  ITEM=${items[$i]}
  SRCDIR=$SRCROOT/${vol_names[$i]}
  DIR=$DESDIR/$ITEM/$ITEM-$FILENAME
  echo "     Source:      $SRCDIR"
  echo "     Destination: $DIR"

  docker run -v ${vol_names[$i]}:/volume -v$DESDIR/$ITEM:/backup --rm loomchild/volume-backup backup $ITEM-$FILENAME

done

popd
/home/daniel/.local/bin/aws s3 sync $DESDIR s3://bucket-name

Restoring a Volume Backup

Before you relax and let your backup script do its work, it is important you convince yourself that the resulting archive contains not only the correct files but that they are picked up correctly by Docker when extracted and moved back into the /var/lib/docker/volumes directory.

Run the backup script, and then use the script below. We can extract this archive using sudo tar -zxvf <archive> command. This reproduces the same directory structure where the files were originally located. In our case, var/lib/docker/volumes/<volume name>. To restore the volume, move the <volume_name> directory into /var/lib/docker/volumes.

# cd into extracted file
cd var/lib/docker/volumes
mv <volume name>/ /var/lib/docker/volumes
  1. This is ok, in my opinion, for small databases up to a few megabytes in size. For larger backups, a remote FTP share would be more appropriate. ↩︎