Data Backup
To backup your Cadmus data, you must:
- backup 1 or 2 Mongo databases (data and logs). You can leave out logs if you are not interested in keeping audit records. Conventionally they are named as follows:
cadmus-PRJcadmus-PRJ-log
- backup 2 or 3 PostgreSQL database(s) (indexes, user accounts, and optionally graph). Conventionally they are named as follows:
cadmus-PRJ(the index anyway can be rebuilt from data if required)cadmus-PRJ-authcadmus-PRJ-graph(present only when using the graph)
If you expose the database services from your Docker containers, as it is usually the case, you just have to use the corresponding database tools to dump databases. Just be sure to use the right port, as often ports are remapped when several services run in the same host machine.
Note that usually in Docker omitting
portsfor database services altogether is the safest option. Anyway, exposing the database ports to the loopback address (127.0.0.1) on your host machine is better than exposing them to all interfaces (- 27017:27017). Since only the API containers need to access the databases, we could remove theportssection entirely from the database services. The API services will still be able to connect using the service name and internal port (e.g.cadmus-ndp-mongo:27017) because they share the same Docker network. This is the most secure configuration as the databases are completely isolated from the host machine’s external network interfaces. Anyway, as we need to access the DB locally for backup, it is acceptable to use the loopback address.
Linux
In Linux, which is the most typical host, you can just have a .sh batch file saving your data somewhere, and then launch it periodically with crontab. Here is an example, which saves all the databases into separate, compressed files, with their date.
Backup
#!/bin/bash
# Backup script for Cadmus databases.
# You can launch this by editing cron with crontab -e, using a line like this (daily dump at 3 AM):
# 00 03 * * * /home/crontab-scripts/cadmus-dump.sh
# --- Configuration ---
# Set the base directory for backups
BASE_BACKUP_DIR="./backup"
# Get the current date in YYYYMMDD format for the directory name
DATE_DIR_NAME=`date +%Y%m%d`
# Define the full path for today's backup directory
TODAY_BACKUP_DIR="${BASE_BACKUP_DIR}/${DATE_DIR_NAME}"
# Ensure the base directory exists
mkdir -p "${BASE_BACKUP_DIR}"
# Create the date-stamped folder for today's backup
echo "Creating backup directory: ${TODAY_BACKUP_DIR}"
mkdir -p "${TODAY_BACKUP_DIR}"
# --- MongoDB Dump ---
echo "Dumping MongoDB databases..."
# Dump cadmus-PRJ (main data)
mongodump --port=27017 --db cadmus-PRJ --archive="${TODAY_BACKUP_DIR}/cadmus-PRJ-mongo.gz" --gzip
# Dump cadmus-PRJ-log (logs)
mongodump --port=27017 --db cadmus-PRJ-log --archive="${TODAY_BACKUP_DIR}/cadmus-PRJ-log-mongo.gz" --gzip
# --- PostgreSQL Dump ---
echo "Dumping PostgreSQL databases..."
# Note: Using '-h 127.0.0.1' requires your DB services to be configured with '127.0.0.1:PORT:PORT' in docker-compose.yml
# Dump cadmus-PRJ (main data)
pg_dump --username=postgres -h 127.0.0.1 cadmus-PRJ | gzip > "${TODAY_BACKUP_DIR}/cadmus-PRJ-pgsql.gz"
# Dump cadmus-PRJ-auth (accounts)
pg_dump --username=postgres -h 127.0.0.1 cadmus-PRJ-auth | gzip > "${TODAY_BACKUP_DIR}/cadmus-PRJ-auth-pgsql.gz"
echo "Backup completed successfully in ${TODAY_BACKUP_DIR}!"
⚠️ Note that for PostgreSQL you should set the password in the home folder of your user (get it via echo $HOME) in a file named .pgpass with this content:
127.0.0.1:5432:*:postgres:YOURPASSWORDHERE
Be sure to set its permissions:
chmod 0600 /root/.pgpass
💡 If you want to download files, just connect to your VM (using your account credentials) via SCP (use your VM IP and port 22). You can either use GUI like WinSCP or command line tools.
Cleanup
This script can be periodically run to cleanup the backup folder from oldest dumps.
#!/bin/bash
# Script to delete old local backups
# --- Configuration ---
# Set the base directory where your backup folders are located
BASE_BACKUP_DIR="./backup"
# Number of latest backup folders to KEEP (e.g., 7 days)
KEEP_DAYS=7
echo "Starting cleanup: retaining the last ${KEEP_DAYS} backup folders in ${BASE_BACKUP_DIR}"
# Find all directories in BASE_BACKUP_DIR, sort them by name (which is the date),
# skip the latest 'KEEP_DAYS' directories, and delete the rest.
# The date format (YYYYMMDD) ensures alphabetical sorting is chronological.
find "${BASE_BACKUP_DIR}" -mindepth 1 -maxdepth 1 -type d | sort -r | tail -n +$((KEEP_DAYS + 1)) | while read dir_to_delete; do
echo "Deleting old backup folder: ${dir_to_delete}"
rm -rf "${dir_to_delete}"
done
echo "Cleanup completed."
Transfer
Of course, you can then transfer your files somewhere else, e.g. (this script is placed in backup folder):
#!/bin/bash
# Transfer script to upload the latest backup archive using lftp
# --- Configuration ---
BASE_BACKUP_DIR="./backup"
DATE_DIR_NAME=`date +%Y%m%d`
ARCHIVE_NAME="cadmus-backup-${DATE_DIR_NAME}.tar.gz"
# NOTE: For security, use SFTP if your server supports it:
# Change 'ftp://' to 'sftp://'
FTP_HOST="ftp.myserver.net"
FTP_USER="user"
FTP_PASS="password" # Replace with a real password or read from a secure source
# --- Archiving ---
echo "Creating single archive for upload: ${ARCHIVE_NAME}"
# Create a tarball of the daily folder
tar -czf "${BASE_BACKUP_DIR}/${ARCHIVE_NAME}" -C "${BASE_BACKUP_DIR}" "${DATE_DIR_NAME}"
# --- Upload ---
echo "Starting upload using lftp..."
# Use lftp to connect, set prompt off, put the file, and quit
lftp -e "set cmd:prompt ''; put ${ARCHIVE_NAME}; bye" -u "${FTP_USER}","${FTP_PASS}" "${FTP_HOST}"
# Check the exit status of lftp
if [ $? -eq 0 ]; then
echo "Upload completed successfully!"
# Clean up the single archive file after upload (keep the original folder)
echo "Removing local archive file: ${ARCHIVE_NAME}"
rm "${BASE_BACKUP_DIR}/${ARCHIVE_NAME}"
else
echo "ERROR: Upload failed! Keeping archive for inspection."
fi
echo "Transfer process finished."
This script just uses some FTP account. Another popular option is using GDrive as your backup target. To this end, you could use a script like this:
#!/bin/bash
# Backup and upload script using rclone to Google Drive
# --- Configuration ---
BASE_BACKUP_DIR="./backup"
DATE_DIR_NAME=$(date +%Y%m%d)
ARCHIVE_NAME="cadmus-backup-${DATE_DIR_NAME}.tar.gz"
RCLONE_REMOTE="gdrive" # Name of your rclone remote
RCLONE_DESTINATION="backups" # Folder in Google Drive
# --- Archiving ---
echo "Creating archive: ${ARCHIVE_NAME}"
tar -czf "${BASE_BACKUP_DIR}/${ARCHIVE_NAME}" -C "${BASE_BACKUP_DIR}" "${DATE_DIR_NAME}"
# --- Upload ---
echo "Uploading to Google Drive via rclone..."
rclone copy "${BASE_BACKUP_DIR}/${ARCHIVE_NAME}" "${RCLONE_REMOTE}:${RCLONE_DESTINATION}" --progress
if [ $? -eq 0 ]; then
echo "Upload successful. Cleaning up local archive..."
rm "${BASE_BACKUP_DIR}/${ARCHIVE_NAME}"
else
echo "ERROR: Upload failed. Archive retained for inspection."
fi
echo "Backup process complete."
💡 Use
rclone moveinstead ofcopyif you want to delete local files after upload.
To setup rclone in the VM:
curl -O https://downloads.rclone.org/rclone-current-linux-amd64.deb
sudo apt install ./rclone-current-linux-amd64.deb
To test the installation and see the GDrive files listed:
rclone ls gdrive:
If you need to create the target folder:
rclone mkdir gdrive:backups
To configure rclone with your GDrive account you need a headless setup via its --copy-config method. These are the instructions for the machine with access to your GDrive account:
- install rclone from https://rclone.org/downloads.
- run
rclone configand create a remote namedgdrive: choose Google Drive and follow the OAuth flow in browser, then save the configuration. - locate the configuration file you saved (e.g. in Windows it’s usually under your user folder
.config\rclone\rclone.conf) and copy it to the VM hosting Cadmus, e.g. via SCP:
scp ~/.config/rclone/rclone.conf user@your-vm:/home/user/.config/rclone/rclone.conf
Crontab
You could launch all these scripts sequentially:
00 03 * * * /home/crontab-scripts/cadmus-dump.sh && \
/home/crontab-scripts/cadmus-transfer.sh && \
/home/crontab-scripts/cadmus-cleanup.sh
⚠️ Ensure all scripts are executable:
chmod +x /home/crontab-scripts/*.sh.
Host Setup
The host would require the database client tools and FTP utility. Should you need to install them, you can follow the procedures outlined here.
MongoDB Client
(1) Import the MongoDB Public Key (⚠️ change the key and version as required for all these commands!):
wget -qO - https://www.mongodb.org/static/pgp/server-8.0.asc | sudo apt-key add -
(2) Add MongoDB repository to the list:
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu $(lsb_release -cs)/mongodb-org/8.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-8.0.list
(3) Update the package list:
sudo apt update
(4) Install the MongoDB Database Tools (which includes mongodump):
sudo apt install -y mongodb-database-tools
Note that this does not include
mongosh. If you also need the shell, follow this procedure (change version numbers as required):
curl -fsSL https://pgp.mongodb.com/server-8.0.asc | sudo gpg --dearmor -o /usr/share/keyrings/mongodb-server-8.0.gpg
echo "deb [ signed-by=/usr/share/keyrings/mongodb-server-8.0.gpg ] https://repo.mongodb.org/apt/ubuntu $(lsb_release -sc)/mongodb-org/8.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-8.0.list
sudo apt update
sudo apt install -y mongodb-mongosh
💡 To restore a whole MongoDB database, use a command like:
mongorestore --drop --archive="cadmus-PRJ-mongo.gz" --gzip --db cadmus-PRJ. To restore a single collection, e.g. the facets, in an easy way, export thefacetscollection via a tool like Studio3T: you will get a folder named after the database, including a couple of.gzfiles for thefacetscollection. Upload this folder to the VM (e.g. via SCP) and run a command like this from the PARENT folder of the folder containing the uploaded folder:mongorestore --drop --nsInclude "cadmus-PRJ.facets" --gzip ./xfer. So, if the folder was uploaded to/opt/xfer, you must run this command from/opt.
PostgreSQL Client
(1) add the APT repository:
echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" | sudo tee /etc/apt/sources.list.d/pgdg.list
(2) import its GPG key:
curl -fsSL https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/postgresql.gpg
(3) Update the package list (if not recently done):
sudo apt update
(4) Install the client package (change client version accordingly):
sudo apt install -y postgresql-client-17
To open the client shell:
psql -h 127.0.0.1 -p 5432 -U postgres
LFTP Tool
Install the lftp tool:
sudo apt install -y lftp