Showing posts with label system configuration. Show all posts
Showing posts with label system configuration. Show all posts

Tuesday, March 04, 2008

Data/Documents backup script

I used to always create directory under my home called data. I'd store files which I need to do work and my working files. Then, this would be the only directory I had to synchronize with Unison (http://www.cis.upenn.edu/~bcpierce/unison/)between machines. Since Fedora has included a directory called Documents, which has a similar purpose (and so has Mac OS X), I've just changed the name of this special directory from data to Documents.



That brings me to the topic of this article--how do I manage if I mess up a file in this directory, and I synchronize the mistake to other mirrors? That would mean none of the computers I work on has a copy anymore. The answer is rdiff-backup (http://www.nongnu.org/rdiff-backup/). I used to make just a user cron job to call rdiff-backup and update an incremental backup repo on my behalf, but I wanted this to be more mechanical (what if I added a new user, etc.). So I wrote the following script which replaces an old script I used to have which would only make a mirror with rsync.



Note that, on my configuration, the incremental backup repo is on the same device as the original files, but both are mirrored in an overnight backup job. This means I have (at least) four mirrors of the same files, which is sort of a waste but I don't really worry about it since it's about 200 MB / user. Ideally, you would keep just the local mirror on the disk, and the rdiff-backup repository on the backup device (or system over network). The downside here would be, if the backup device (or remote system) crashes you loose the ability to restore a previous increment.



Following is the script for maintaining incremental backup repositories with rdiff-backup for all users. Caution: the backup path should be as read-only as possible, but it is not if the user logs in to the system. For this reason, you'll have to count on them not to modify the files, or keep the files on a filesystem they cannot mount. For simple remote access over samba (the kind of access in my setup), you would have to configure a per-user share (so Joe can't see Susan's backup, for instance) and make each one read-only (so Joe cannot corrupt his repository by changing the files there).




#!/bin/sh
#
# Make incremental backups of 'Documents' directories for users
#
# Ryan Helinski

LOGFILE=/var/log/rdiff-backup.log

# Close stdout and stderr
1>&-; 2>&-;

# Direct stdout and stderr to logfile
exec 1>>$LOGFILE; exec 2>>$LOGFILE;

echo Begin $0, `date`;

# For debugging
#set -x

# Script-specific configuration
HOMESPATH=/home
BACKUPPATH=/opt/backup
RDIFF_OPTIONS="--print-statistics"
DOCSDIR=Documents

# Max increment life
# The time interval is an integer followed by the character s, m, h, D, W,
# M, or Y, indicating seconds, minutes, hours, days, weeks, months, or
# years respectively, or a number of these con-catenated.
MAX_LIFE=4W

for USERHOME in $HOMESPATH/* ; do

# USERDIR=`basename $USERHOME`;
USER=`stat -c "%U" $USERHOME`;

if [ -d "$USERHOME/$DOCSDIR" ] ; then
echo "Updating $BACKUPPATH/$USER/$DOCSDIR";

# Create backup repo if it doesn't exist
if [ ! -d "$BACKUPPATH/$USER/$DOCSDIR" ] ; then
echo "Creating rdiff-backup repo";
mkdir -p "$BACKUPPATH/$USER/$DOCSDIR";
chown $USER "$BACKUPPATH/$USER";
chown $USER "$BACKUPPATH/$USER/$DOCSDIR";
fi

# Update repo
rdiff-backup $RDIFF_OPTIONS "$USERHOME/$DOCSDIR" \
"$BACKUPPATH/$USER/$DOCSDIR" ;

# Clean up repo
#rdiff-backup --remove-older-than $MAX_LIFE --force \
# "$BACKUPPATH/$USER/$DOCSDIR" ;

fi

done

echo End $0, `date`;
echo

exit


To actually remove old backups, uncomment the code in the "Clean up repo" section.

Thursday, August 30, 2007

Configure OpenSSH to automatically authenticate

Sometimes entering your pass-phrase to gain SSH access can get redundant, or you may want an automated script to be able to authenticate as you (as described at http://sial.org/howto/rsync/).

I've been trying to do this for some time now--apparently if you read the man pages carefully enough you can figure this out. I thought I'd need a more advanced SSH client, but OpenSSH already has everything you need--the ssh-keygen program is key. Following is a brief walk-through of how to configure connection from client to server for SSH. Replace the words client and server as appropriate.

client$ ssh-keygen -t rsa


It will ask where to save, you want the default since this is where ssh will look. It will ask for a pass-phrase, and since we're trying to get around having to enter a pass-phrase, we don't want to protect our authentication, so just hit enter twice. Then it will give you a fingerprint and put files in your ~/.ssh directory.

Copy the ~/.ssh/id_rsa.pub to the server machine taking care not to overwrite the same file there:

client$ scp ~/.ssh/id_rsa.pub user@server.address:.ssh/client.pub


Now SSH to the server and add the public key to the authorized keys file like so:

server$ cd ~/.ssh
server$ cat client.pub >> authorized_keys
server$ rm client.pub


The SSH server won't let you use the key unless the file is secured. That is, the keys should only be readable by you.

server$ ls -l authorized_keys
-rw-rw-r-- 1 user users authorized_keys
server$ chmod og-rw authorized_keys


Now we should be able to open an SSH session simply by:

client$ ssh user@server.address


where you can omit the 'user@' part if the user names are the same on the local and remote machines.

Finally, note that doing all this means if you leave your machine open, an intruder has more doors they can open! This is why you should always lock and/or time-out your screen. If the key has no passphrase, then if someone copies it, they can gain access to the machine you opened with this procedure. Therefore, you should restrict access to your machine by configuring /etc/hosts.allow and /etc/hosts.deny. If you're concerned, don't let the passphrase for the key be null (use a passphrase). Taking these steps can help protect your key.

Tuesday, July 17, 2007

Mount USB drives on boot

On Fedora Core 7, leaving a USB drive (a simple flash disk) connected doesn't seem to be mounting after reboot even though it's properly listed in the /etc/fstab. I went through a few solutions to this, like this one, but the only one that worked for me was modifying the /etc/rc.sysinit script as I explain below.

I seem to remember creating an entry in the fstab, and allowing the auto option (which is part of the default specification), would make your USB partition mount on boot. The issue with FC7 seems to be that the usb-storage kernel module isn't being loaded when the fstab is first loaded, and that the USB devices aren't initialized yet by /etc/rc.sysinit. This seems to be common with all newer distros, in accordance with the move to udev, hal and other components, which facilitate plug & play user mounting and they don't need them to be available before entering a GUI. This is great for workstations, but I have a "headless" server configuration, therefor I'm not using GNOME to mount anything on-demand, but rather they should be available statically.

After a few days of trial & error, and inspiration from this post, I added the following lines to the end of my /etc/rc.sysinit. Note that changes to this particular file may be removed by system updates (Using yum) as I explain below.
# Try to mount all USB partitions
# This should be moved to `rc.local' and be prepended with a wait
modprobe usb-storage
mount -a

I believe this is the most most robust method, since simply adding the entry to the fstab will allow it to be mounted after the USB devices are initialized. It is preferred to use the /etc/rc.local script to issue this nature of commands (local/custom configuration), but this script seems to be run before the /etc/rc.sysinit script exits, meaning the USB partitions may not yet be available. Commanding the /etc/rc.local script to wait for the /etc/rc.sysinit script to exit may be a solution.

Finally, note that you can guarantee consistency of the partition before mounting it by carefully editing the corresponding fstab line in the normal way, see man 5 fstab for more.