Database Backup
| Name | Location | Frequency |
| Production data | On Server | online |
| Copy 1 | On Server | 5 minutes transation log |
| Copy 2 | On Backup Server | 5 minutes transaction log |
| Copy 3 | AWS S3 | 5 minutes transaction log |
| Methodology |
backup scripts crontab |
Server System Backup
| Name | Location | Frequency |
| Production data | On Server | online |
| Copy 1 | Backup Server | 5 minutes transation log |
| Copy 2 | AWS S3 |
weekly monthly |
| Methodology | rsync, cron | 5 minutes |
| target |
/opt/wildfly/configuration /opt/wildfly/deployments
|
During thte backup process, backup files are stored in a temporary directory. In the case of a base backup:
/home/backup/base_backup/temp
Once the backup process is finished, the backup files are moved to a staging area:
/home/backup/base_backup/staging/YYYYMMDD
where YYYYMMDD is the date of the base backup.
A chron scriipt, backup_archiver.sh copies the files from the staging area to an S3 bucket as defined in the file
/home/backup/scripts/backup.conf
After successfully copying the files to S3, the files are moved to a “done” directory:
/home/backup/base_backup/done/YYYYMMDD
Files older than “file-age-limit” are deleted from the “done” directory.
Backup directories
Temp - holds files while the backup process is running
Staging - holds files for copying to Amazon S3
Done - holds the files after copying to Amazon S3
| Script | Description |
| base_backup.sh | create a base backup by tarring the database data directory |
| wal_archiver.sh | copy wal archives every 10 minutes |
| backup_archiver.sh | copy files to s3 every 5 minutes, remove old files |
| backup.conf | parameters for the backup processes |
Note:
The backup scripts are available in the git server. /* put the proper project reference here */
Backup processes are run periodically. To do this, entries are made in the /etc/crontab file.
Sample crontab entries:
#add /home/backup/scripts to the path to shorten commands
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/home/backup/scripts
#LOGDIR to environment
LOG_DIR=/home/backup/logs
# don't bump into each other
# weekly backups run on Saturday and Sunday except on the 1st of the month.
# Monthly backups run only on the 1st of the month.
# Base backups and wal files are retained for 60 days in S3
0 1 * * 6 postgres [ $(date +\%d) != "01" ] && pg_dump.sh WEEKLY >> $LOG_DIR/backup-pg_dump.log 2>&1
#monthly pg_dump
0 1 1 * * postgres pg_dump.sh MONTHLY >> $LOG_DIR/backup-pg_dump.log 2>&1 \
#weekly base-backup
0 1 * * 0 postgres [ $(date +\%d) != "01" ] && base_backup.sh >> $LOG_DIR/base_backup.log 2>&1
# Archive files to S3, SMB or SCP then move to done directory
# files are kept in done directory for X days
*/5 * * * * postgres backup_archiver.sh >> $LOG_DIR/backup_archiver.log 2>&1
Restore the base backup to the data directory. Put the decompressed wal files in a recovery directory. Set the database recovery mode and start the database to apply the wal files.