Wednesday, 14 October 2009

Building a customised quiet NAS using Ubuntu server 9.04: Part 3 (NFS software and backup scripts)

We require a static IP address for our NAS. Edit /etc/network/interfaces and modify the eth0 lines:


# The primary network interface
auto eth0
iface eth0 inet static
address 192.168.1.77
netmask 255.255.255.0
gateway 192.168.1.254

ifdown eth0; ifup eth0

Install some software:


$ sudo apt-get install postfix nfs-kernel-server apache2 unzip zip make usbmount


Edit /etc/exports:


# /etc/exports: the access control list for filesystems which may be exported
# to NFS clients. See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4 gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes gss/krb5i(rw,sync,no_subtree_check)
#
# insecure added to allow pre-Leopard OS X clients to mount share.
/communal *(rw,sync,no_subtree_check,no_root_squash,insecure)



$ sudo /etc/init.d/nfs-kernel-server restart

$ showmount -e localhost

Export list for datastore:
/communal *


Next consider the backups, running from a cron job. Remember that the requirement was for a job which would detect if a USB disk was mounted and if so, kick off a backup. This cron job would check for the existence of a USB disk every 15 minutes. We need some inbuilt controls - if the job takes longer than 15 minutes, a problem occurs as concurrent backups will kick off. This requires a locking mechanism.

Firstly we need a USB mounting mechanism:

Edit /etc/usbmount/usbmount.cfg to allow us to mount hfsplus filesystems:


#############################################################################
FILESYSTEMS="ext2 ext3 ufs hfs hfsplus"



$ sudo crontab -e

0,15,30,45 * * * * /usr/local/scripts/backup.sh


backup.sh looks like this:


#!/bin/bash
# Name: backups.sh
# Author: tim.gibbon@lunarlite.co.uk
# Version: 0.1
# Description: Backup utility for xyz

############SETUP SOME LOGFILES#############################
DAY_OF_YEAR=`date +%j`
LOGFILE_DATA="/home/tng/BACKUP/backuplog_${DAY_OF_YEAR}.txt"
LOCKFILE="/tmp/backup.pid"
BACKUPDEVICE=`mount | grep usb | awk '{print $3}'`
BACKUPDEVICEFOUND=`mount | grep usb `
BACKUPDEVICE_EXIT_STATUS=$?
MAILTO="tim.gibbon@lunarlite.co.uk"
############SETUP SOME LOGFILES#############################

# Sanity check our lockfiles
/usr/local/scripts/check_old_backup_dead.sh

#If already running then stop
if [ -e ${LOCKFILE} ]
then
echo "Already running. Exiting" >>${LOGFILE_DATA}
exit 1
fi




#Create backup directory mount points if they dont exist
if [ ${BACKUPDEVICE_EXIT_STATUS} -ne 0 ]
then
echo "No backup device exists" >>${LOGFILE_DATA}
exit 1
fi


# We have a backup device and no lockfile. Time to start backing up.
echo $$ >${LOCKFILE}
>${LOGFILE_DATA}
echo "Backup device exit status is ${BACKUPDEVICE_EXIT_STATUS}" >> ${LOGFILE_DATA}
#echo "Found disk" >> ${LOGFILE_DATA}
echo "Backing up to ${BACKUPDEVICE}" >> ${LOGFILE_DATA}


STARTTODAY=`date`
echo "####################################################" >> ${LOGFILE_DATA}
echo "BACKUP STARTED $STARTTODAY" >> ${LOGFILE_DATA}
echo "####################################################" >> ${LOGFILE_DATA}
echo "Disk capacity before backup" >> ${LOGFILE_DATA}
df -h ${BACKUPDEVICE} >> ${LOGFILE_DATA}


#####Do the actual work. -x = Don't cross filesystem boundaries
/usr/bin/rsync -a -l -x / ${BACKUPDEVICE}/ >>${LOGFILE_DATA}


ENDTODAY=`date`
echo "####################################################" >> ${LOGFILE_DATA}
echo "BACKUP FINISHED $ENDTODAY" >> ${LOGFILE_DATA}
echo "####################################################" >> ${LOGFILE_DATA}
echo "Disk capacity after backup" >> ${LOGFILE_DATA}
df -h ${BACKUPDEVICE} >> ${LOGFILE_DATA}

#Backup finished here, lets cleanup
umount ${BACKUPDEVICE}

#Make sure can read the logfiles
chown tng:tng ${LOGFILE_DATA}


#Remove the lockfile
rm -f ${LOCKFILE}

#Send mail to inform everyone backup complete and USB disks needs removing
echo "." >> ${LOGFILE_DATA}
mailx -s "Backup complete" ${MAILTO} <${LOGFILE_DATA} # exit exit 0


Check old backup dead script looks like the following:


$ cat /usr/local/scripts/check_old_backup_dead.sh


#!/bin/bash

LOCKFILE="/tmp/backup.pid"
LOG="/home/tng/BACKUP/remove_lockfile.txt"

if [ ! -e ${LOCKFILE} ]
then
echo "NO LOCKFILE EXISTS - exiting" >${LOG}
exit 0
fi

BACKUP_PID=`cat ${LOCKFILE}`

echo "Looking for ${BACKUP_PID}" >${LOG}

ps -ef | grep ${BACKUP_PID} | grep -v grep

FOUND_BACKUP_PID=$?

if [ ${FOUND_BACKUP_PID} -eq 0 ]
then
echo "Found PID ${BACKUP_PID} running" >>${LOG}
exit 0
fi

echo "We found a lockfile, but no backup running" >>${LOG}
echo "We are removing the lockfile" >>${LOG}
rm -f ${LOCKFILE}



Configure postfix quickly.

I then installed gnuplorer to keep a copy of the essential files and track changes made to the server since installation. This was modified slightly [gnuplorer.cfg] to pickup the backup logs (/home/tng/BACKUP/). A monthly cron job was added to run gnuplorer and rsync the job output for reference purposes.

We now have a self-contained resilient NAS that will run a backup to external media which can then be mounted under OS X. The final steps are to allow the OS X clients to mount the NFS mounts. Some tweaks are advisable - increasing the rsize and wsize mount options. These will be detailed in the next blog entry.

No comments: