Backups How To

Restoring from rsync backups

sudo rsync -va --stats --log-file ~/restore.log --exclude "/*/.*/" --exclude "/*/.*" /media/user/Backups/server/ /home/
this will restore everything except top level . files and .foders

How to configure cygwin rsyncd server

get rsyncd from cygwin install rsyncd service
cygrunsrv --install "Sync" --path /cygdrive/c/cygwin/bin/rsync.exe --args "--config=/cygdrive/c/cygwin/etc/rsyncd.conf --daemon --no-detach --port 1823" --desc "Synchronization service" --disp "Syncd" --type auto
to remove later use
cygrunsrv --remove "Sync"
Edit c:\cygwin\etc\rsyncd.conf

use chroot = false
strict modes = false
log format = %h %o %f %l %b
log file = /cygdrive/c/Project/rsyncd.log
[Project]
path = /cygdrive/c/Project
comment = Project
read only = false
auth users = userx
secrets file = /etc/rsyncd.secrets

C:\cygwin\etc\rsyncd.secrets
userx:secret
Restart and connect to it:
rsync -arbuz userx@server:Project .

How to do incremental backup with rsync

The idea is to have a full rsync backup that hard links any file that has not changed since the last backup, thus preserving space. To do this use the preserve hard links argument. Then, once the backup is completed re-link the new folder, so it becomes a master. Here is example for backing up all user folders, with full logging

DATE=`date +%Y-%m-%d.%H-%M-%S`
echo "*** Starting the home folder backup for $DATE" | tee -a ~/bin/backup.log
# verbose, archive (-rlptgoD), dont cross file systems and preserve hard links
sudo rsync -vaxH --numeric-ids --exclude-from ~/bin/backup.exclude --delete --delete-excluded --stats --log-file ~/bin/backup.log --prune-empty-dirs --link-dest=/media/Backups/home/server /home/ /media/Backups/server/home.$DATE 2>~/bin/backup.errors

if [[ ($? -eq 0) || ($? -eq 24) ]]; then
    #  24     Partial transfer due to vanished source files
    echo "*** Home folder backup for $DATE successful" | tee -a ~/bin/backup.log
    ln -snf /media/Backups/server/home.$DATE /media/Backups/server/home
    echo $DATE 0 server >>~/bin/backup-status.log
else
    error=$?
    echo "*** Home folder backup for $DATE finished with $error" | tee -a ~/bin/backup.log
    echo $DATE $error server >> ~/bin/backup-status.log
    echo "Errors collected during the last run from stderr:"
    cat ~/bin/backup.errors
    exit $error
fi

How to set up rsync as a backup server (rsyncd)

Edit /etc/default/rsync and set

RSYNC_ENABLE=true

Create /etc/rsyncd.conf with something like the following

[backup]
   path = /media/Backups/folder/
   comment = Backup
   uid = user
   gid = user
   read only = false
   auth users = rsyncuser 
   secrets file = /etc/rsyncd.secrets

Where /etc/rsyncd.secrets contains a shared secret (password) for rsyncuser Start rsyncd with

sudo service rsync start

How to do a full system backup with rsync and a bare metal restore

Backup can be done with a single command like this:

sudo rsync -vaz --delete --delete-excluded --stats --password-file ~/bin/backup-system.secrets --include-from ~/bin/backup-system.include --exclude-from ~/bin/backup-system.exclude --log-file ~/bin/backup-system.log / destination

destination can be either a mounted disk or a remote system that rsync supports, such as an sftp or an rsync server (rsyncd): rsync:rsyncdealer@backupsserver/systembackup// The include file is as follows:

# include all folders but not files to simplify bare metal restore
- /var/cache/system-tools-backends
/var/cache/**/
/var/log/**/
/var/run/**/

The exclude file is shown here for a typical Debian based linux system. Some folders will be missing on RPM or Suse systems. Also note that /home is excluded from the system backup.

*~
/proc/**
/sys/**
/dev/**
/media/**
/tmp/**
/home/
/lib/udev/devices/*
/var/lib/dpkg/info/*
# skip inaccessible folders even by root
# strangely only the non-slash terminated string below works properly
/var/cache/system-tools-backends
# remove all files but not folders to simplify bare metal restore
/var/cache/**
/var/log/**
/var/run/**
/var/tmp/**
/var/spool/cups/**

To restore simply copy backed up files to a new drive, using a working system. Make sure to preserve file attributes:

rsync -ghop ''source'' ''destination''

Make sure your MBR and boot records are ok if you are running on an intel platform.

How to automatically back up a desktop that is not always on

If you know that you do not connect your system to different networks then your best and the easiest way to back it up is to add an entry to anacron. Ana[chronistic]cron will start jobs that are overdue, unlike cron that just ignores past items. Anacron itself is started by Ubuntu via Upstart either on boot (/etc/init), on wakeup (/etc/apm/event.d/) and daily via cron (/etc/cron.d). That is if anacron is not already up. To use it create symbolic link

sudo ln -s /home/user/bin/1backup-as-user /etc/cron.daily

and

sudo ln -s /home/user/bin/1backup-system-as-user /etc/cron.weekly

Since your anacron scripts are kicked in under the root authority you need to make the scripts assume your authority. For example backup-as-user would read like this:

sudo -u user -H /home/user/bin/backup

How to automatically back up a laptop when on home network

Create a bash script that can calculate dates and launch backups if a set number of days has passed. Something along these lines:

#!/bin/bash
# This script is run when the network is up
# it checks for the backup server to be available and then starts appropriate backups based on the date that they last ran
# Schedule (every n days)
HOMEBACKUP=1
# The datacalc routines
date2stamp () {
    date --utc --date "$1" +%s
}

stamp2date (){
    date --utc --date "1970-01-01 $1 sec" "+%Y-%m-%d %T"
}

dateDiff (){
    case $1 in
        -s)   sec=1;      shift;;
        -m)   sec=60;     shift;;
        -h)   sec=3600;   shift;;
        -d)   sec=86400;  shift;;
        *)    sec=86400;;
    esac
    dte1=$(date2stamp "$1")
    dte2=$(date2stamp "$2")
    diffSec=$((dte2-dte1))
    if ((diffSec < 0)); then abs=-1; else abs=1; fi
    echo $((diffSec/sec*abs))
}

launchBackup (){
    if [ ! -e ~/bin/$1.timestamp ]; then
        echo "1979-01-01 01:01:01 PST" > ~/bin/$1.timestamp
    fi
    if [ $(dateDiff -d "now" "$(cat ~/bin/$1.timestamp)") -ge $2 ]; then
        echo "*** Launching the $3 backup" | tee -a ~/bin/backup-launcher.log
        ~/bin/$1
        if [[ $? -eq 0 ]]; then
                echo $DATE > ~/bin/$1.timestamp
        else
                echo "*** Error $? during the $3 backup. Check the logs." | tee -a ~/bin/backup-launcher.log
        fi
    fi
}

echo ---------------------------------- >> ~/bin/backup-launcher.log
DATE=`date "+%Y-%m-%d %H:%M:%S %Z"`
echo "*** Starting backup launcher for $DATE" | tee -a ~/bin/backup-launcher.log
launchBackup backup $HOMEBACKUP "home folder"
echo "*** Backup launcher for $DATE successful" | tee -a ~/bin/backup-launcher.log

Then configure the network manager to run this script every time a connection to your home network is made, as described in this section