rsync backup script

There are many backup solutions available for Linux, several of them in fact using rsync under the covers. However none of them quite seemed to fit my requirements (or I didn’t dig deep enough into the options).

Instead I have put together my own shell script to back up my home directory. It takes advantage of Linux filesystems’ capability to give a single file more than one name (known as hardlinking), and rsync’s –link-dest option. In essence I have a hybrid incremental/full backup system, which only takes as long to run as an incremental system.

My main inspiration comes from this page by Mike Rubel. My script is available at http://funkyhat.org/stuff/scripts/funkybackup

Update (17-05-2010): The script now writes out the date of the most recent backup to a file if the rsync process completes successfully. This means that if a backup is interrupted part way through it won’t be used as the base for linking, so my backup space won’t get bloated artificially.

Drupal Multisite in lighttpd

I’m currently investigating switching from Apache to Lighttpd, and in the process I decided I’d have a go at simplifying my virtual host configuration. Lighttpd’s config files are already simpler than Apache’s, but I figured I could move from a separate config file for each virtual host to specifying settings in groups. Perhaps this is also possible with Apache, but I’ve not tried as one file per vhost was working fine.

I also need more advanced URL rewriting than lighttpd’s mod_rewrite offers, as some Drupal modules rely on it (Apache has such URL rewriting inbuilt in its mod_rewrite). This part of the switch is simple, someone else has already done the hard work: installing lighttpd-mod-magnet and setting up this script is all that needs to be done (instructions are in the script, near the top).

I wasn’t happy with either of the vhost modules provided by lighttpd, they appear to only work well if all of your sites are laid out in your filesystem in exactly the same way. I could have worked with that and created symlinks from my drupal dir to each site’s name in the vhost dir, but that feels messy, and would mean I could not set up access rules for the /sites directory.

Enter include_shell. This beautifully simple Lighttpd configuration option enables configuration to be generated by a script. My /etc/lighttpd/conf-available/15-drupal6.conf has one line:

include_shell "/etc/lighttpd/shell/drupal6"

The drupal6 script is where the magic happens:

#!/bin/sh
# Script for generating drupal multisite niceness

# Space separated list of disabled sites:
DISABLED=""

# Space separated list of sites with custom config files:
CUSTOM=""

# Location of custom config files:
CUSTOMDIR="/home/matt/Websites/testing/customconfig"

DRUPAL_DIR="/home/matt/Websites/testing/drupal6"
DRUPAL_LUA_PATH="/home/matt/Websites/tools/drupal/drupal.lua"

#Don't edit this, edit the other DISABLED="" VAR up there ^
DISABLED="all default "$DISABLED

for VHOST in `find ${DRUPAL_DIR}/sites/ -mindepth 1 -maxdepth 1  \( -type d -or -type l \) -exec basename "{}" \;` ; do
    for SITE in $DISABLED ; do
        if [ $SITE = $VHOST ]
        then
            #Just calling continue here can't work, we're inside a child loop
            DIS=y
        fi
     done

    if [ "$DIS" = "y" ]
    then
        DIS=""
        continue
    fi

    echo "\$HTTP[\"host\"] =~ \"$VHOST\" {
          var.vhost_name = \"$VHOST\"
          server.document-root = \"$DRUPAL_DIR\" 

          magnet.attract-physical-path-to = (\"${DRUPAL_LUA_PATH}\")

          \$HTTP[\"url\"] =~ \"^/sites/.*\" {
          \$HTTP[\"url\"] !~ \"^/sites/(`echo $VHOST| sed 's:\.:\\\.:g'`|all)/.*\" { url.access-deny = (\"\") }
          }"
          # This allows access to /sites/all and /sites/(this domain) which is all we need access to

    for SITE in $CUSTOM ; do
        if [ "$SITE" = "$VHOST" ]
        then
            cat $CUSTOMDIR/$SITE
        fi
    done

    echo "}"

done

The variables in the script should be self explanatory, though the $CUSTOM variable is untested – it’s just there in case I have a need for it.

Lighttpd parses the output of the script as part of its configuration. As I’ve included a magnet configuration in each vhost using this script it’s not needed anywhere else (you just need to enable the magnet module).

This setup won’t work with drupal installations in subdirectories, but I don’t use subdirectories for my drupal sites, so that’s not an issue for me.

Hopefully others will find this useful ⢁)

Update: I’ve replaced the alias lines with a url.access-deny statement, which actually works, with help from darix on irc.freenode.net/#lighttpd

Putting -rt kernels first in grub2

I’ve installed the linux-rt realtime kernel package in Ubuntu 9.10 so I can play with the pro audio stuff that uses JACK properly. It seems that the -rt kernel doesn’t get as many updates as the -generic one, resulting in it having a lower release version (currently -generic is at 2.6.31-18 and -rt is at 2.6.31-9).

This is unfortunate because it means that -generic kernels will be placed first in the grub boot menu, and be chosen as the default boot option. I could change the default boot option each time a new kernel is installed, but I’m lazy.

There are a couple of solutions to this, firstly I could remove all of the linux-generic kernel versions, but I’d rather not just in case the -rt kernel causes issues, or some apps don’t play nice with it.

The other solution is to figure out how to re-order the grub menu. With grub2 (in Ubuntu and Debian at least) the menu config file should not be edited by hand, and is instead generated by running the scripts in /etc/grub.d/ in order. I didn’t work out exactly how the scripts order the kernels, but I figured out if I copied the file responsible for generating the linux entries in the boot list and modified my copy so that it only found -rt kernels I could force the -rt kernels to the top.

Here are the commands needed to do this:

cd /etc/grub.d
sudo cp 10_linux 09_linux-rt
sudo sed -i 's:\(vmlinu[^ ]*\)-\*:\1-*rt:g' 09_linux-rt #<-edits 09_linux-rt and makes
sudo update-grub                                        #  it only find -rt kernels

This does result in the -rt kernels being listed twice, and I could modify 10_linux as well so that this didn’t happen, but I’m happy with the way it is.

Update: 02/09/2010 fixed sed command (thanks to canakas on ubuntuforums.com for pointing out it wasn’t working)

Persistent session with ConnectBot

One of the coolest applications available for Android is ConnectBot – though I say this as an owner of a G1, which has a hardware keyboard. I doubt it would be as useful if I were restricted to an on screen keyboard only. Something which makes ConnectBot even more useful is GNU screen (of course screen extremely useful all the time, not just when used in conjunction with ConnectBot).

Something which I’ve added to make all this even more usable is a script which reconnects a specific screen session when I connect to one of my servers using ConnectBot.

#!/bin/sh

# Do stuff when I connect from my phone

if screen -ls | grep g1gir
then
 screen -x g1gir
 exit 0
else
 screen -c .screen-g1gir -S g1gir
 exit 0
fi

Nice and simple :-). This takes advantage of ConnectBot’s post-login automation setting to launch the script, which I have placed in ~/.local/bin/ on my server. I’m also loading a custom screen configuration which starts some programs for me (that’s what -c does) but this part is optional.

Trial before punishment, please.

From the Open Rights Group:

“This week, plans to disconnect music fans who illegally download music will be announced by the government, as part of their Digital Economy bill.

This plan won’t stop copyright infringement and with a simple accusation could see you and your family disconnected from the internet – unable to engage in everyday activities like shopping and socialising.”

I will be writing to my MP, and have signed the petition at number10.gov.uk. I think you should too.

RPM – DEB hybrid?

Something that came up during the live recording of Linux Outlaws and Ubuntu-UK Podcast at OggCamp earlier today was the huge number of Linux distributions available, and whether this meant that developers, advocates and other people are spread too thinly between projects, and whether 3rd party and proprietary vendors are put off by the enormous range of options they are expected to support.

I think that people should be free to work on whatever they want to. Someone that is not happy with the way a particular project works is not as likely to make useful contributions as someone who is happy with it, from my point of view at least.

The point I raised in relation to 3rd party vendors especially is whether it would be possible to create (perhaps with some modification to one or other system) a type of hybrid package that contains the program’s files as well as the relevant package management information for both the deb and rpm package managers. I’m aware this is something that would be most useful for closed source software vendors, as distributions can do their own packaging of programs where licensing permits it, but as closed source vendors are interested in producing software to run on Linux I think there is nothing wrong with making that easier for them.

Someone (Popey?) pointed out that universal package managers have been tried several times and not worked, but I’m not sure this is quite the same idea. It’s not trying to create a universal package manager, just a universal package.

I’ve not yet done any investigating to see how feasible this would be, perhaps someone with a reasonable knowledge of both packaging systems could tell me why it would(n’t) work. I’d be interested to hear from anyone else what they think about this.

Credit Card Frauds!

I have to say that I am impressed with the service that Egg provides.

Someone managed to get hold of my credit card details and used my card to buy £60 worth of Vodafone mobile credit, but before I’d even checked my balance and found out I got a phone call from Egg’s fraud department to check whether the transactions were legit. They cancelled my card and sent a new one out very quickly, and after I signed the form to say I didn’t authorise the payments they were removed from my account. This is one of the reasons I try to always use a credit card rather than a debit card to make purchases.

However, having looked at the egg site, it looks like the new accounts they are offering are as good as the one I have, the interest rates are not as good and there is a £1 monthly fee. So I’m not sure if I would recommend them to someone or not…

cupcakes are nice

Well, I upgraded to cupcake a few days after my last post. I rooted my phone as soon as I got it, so I didn’t get the “over the air” update automatically, I installed the ADP cupcake build instead.

I had to wipe the data from my phone to do that but that didn’t really bother me, the Market stores a list of the apps you’ve downloaded so it’s nice and easy to install them all again. The on screen keyboard is quite nice, good predictive text, I use it only for short bits of text though, the hardware keyboard is much faster. The Google Mail client isn’t any different as far as I can tell, still only top posting :-(.

I can now use my A2DP Bluetooth headset/headphones properly, but I still can’t send or receive files to other phones/computers, which sucks a bit. Battery life does seem slightly improved compared to Android 1.4, but it’s still quite terrible.

I’ve started using PMix, an MPD client for Android, it works well enough, but is a little slow in places, hopefully this will improve with later releases :-). I also found the Locale app, which lets you configure your ringtone, volume, background, screen brightness and wifi state based on your location, date, time and battery state. Very cool! No more switching to silent when you get to work and then forgetting about it at the end of the day and missing calls. Another potentially useful app is Pintail, which responds to a txt message containing a PIN you decide with the phone’s current location, so if you lose it or leave it at a friend’s house you can at least figure out where it is.

That’s it for now, perhaps I’ll think of something more interesting to blog about than my phone one day… never know!

gigglephone

I broke the screen on my old phone, and decided that was as good an excuse as any to get myself an android phone.
So here I am, blogging on my G1.
So far on the whole I’m really loving it, I can ssh in to my server from anywhere, so I’ve been playing around getting irssi working just how I like it, but more importantly (or is it really just extremely geeky?) I can get a shell and fix it if something goes wrong and I’m not near my PC.
My favourite apps at the moment are ConnectBot, Shazam, ShopSavvy and Banshee Remote (which isn’t finished yet, but still very cool).
Perhaps I’ll start keeping a list of all the apps I’m using on here.

My major gripes with android at the moment are the lack of decent bluetooth support (handsfree/headset only support I think, am I missing any?), and top-posting only in the google mail app. I hear bluetooth will be extended in cupcake (the next major release), with any luck google mail will be improved as well, so I’ll have to find something else to complain about :-).

Domain Renewal Group – the new Domain Registry of America

I got a letter in the post today from the “Domain Renewal Group”
It looked suspiciously similar to the “Domain Registry of America” scammer’s letters I’ve received in the past. And without too much effort I discovered they are indeed one and the same.

If you do receive a letter from one of those two groups, your best bet is to just throw it in the bin (assuming you haven’t already been convinced to switch to them as your domain name registrar).