If there is anything that the recent GMail outages taught us, it's that losing access to your email for a few hours should not mean that the end of the world has arrived. I think it also served as a reminder that if you rely 100% on the cloud for access to your data, then be prepared to be disconnected from it every once in a while. Fortunately, it's quite easy to take advantage of free email services from the likes of Google while also making sure that should the worst happen, you can still access your data. With more and more communication, both professional and personal, being conducted through email, it's no wonder that people do get nervous when they suddenly lose control over several years worth of contacts and emails. This short article will show you how easy it is to back up your GMail automatically to a remote server which you can then access in the event that GMail goes down or you lose access to your account. We'll be using nothing but free open-source software. If you already have a net-connected machine with Ubuntu installed then you're set to go. Otherwise, consider recycling an old pentium-or-better-cpu machine with a network card—the processing demands are relatively low and the most important resource is storage space for all the emails and attachments. Although these instructions are for Ubuntu, they should also work on Debian or any other Debian-based distro. To set this up you obviously need a GMail account, although any email account which can be accessed using the POP protocol can be backed up using the same method. We'll be using the excellent Fetchmail to retrieve messages from the GMail account while leaving them untouched in your GMail inbox, Procmail to deliver and filter them into a local Maildir-format mailbox, and Dovecot to access the emails remotely, all on the latest release of Ubuntu, Hardy Heron (8.04 LTS).

1. Setup GMail

• Log in to your GMail account
• Click 'Settings>Imap/Pop' and enable POP access
• You can chose whether to only download messages which arrive from now on or to download all existing messages. The latter is useful if you have already been using your GMail account for a while and would like to back all existing messages to a secure alternative location.

2. Install packages

• Fire up a terminal window and run the following command: sudo apt-get install dovecot-imapd procmail fetchmail mutt
• This should automatically download and install everything you need. The mutt package is optional and is a useful tool for checking if your IMAP server is working properly and that your emails are being downloaded (not to mention a nice command-line email client!).

3. Configure Fetchmail

• We need to define our GMail accounts in fetchmail's configuration file. Open the file /etc/fetchmailrc for editing as root. The file won't exist yet so we'll have to create it.
• Copy & paste the following into the file:
  set daemon 60 set syslog set postmaster "localuser" poll pop.gmail.com protocol pop3 username "username@gmail.com" there with password "yourpassword" is localuser here keep ssl mda "/usr/bin/procmail -d localuser"
• Change localuser to your local username on the linux box and the email and password to match your gmail account. If you want, you can add as many other accounts as you want here. For each account, just copy the text starting from 'poll...' and change the login details.
• The first line tells fetchmail to check for new emails every 60 seconds. Edit to match your preferred update frequency.
• Edit /etc/default/fetchmail as root and change the line START_DAEMON=no to START_DAEMON=yes

4. Configure Procmail (~/.procmailrc)

• We're using procmail to deliver email from fetchmail to a mailbox in Maildir format. This is necessary due to the lack of Maildir support in Fetchmail, which supports the mbox format. The main difference between the formats is that Maildir stores each email in a single file, making it very efficient when you have a lot of emails. In contrast, mbox format stores all emails in a single file.
• Create ~/.procmailrc and put the following text in it:
  MAILDIR=$HOME/Mail/ DEFAULT=$MAILDIR VERBOSE=on LOGFILE=$MAILDIR/procmail.log :0 $MAILDIR
• Set permissions on the file with: chmod 640 ~/.procmailrc
• This file simply tells procmail where to store the emails (in ~/Mail) and where to store it's logs. Procmail is capable of a lot more than this, and it supports a powerful filtering language. You can specify rules in your ~/.procmailrc file to filter emails based on sender, subject, etc.

5. Configure Dovecot

• We're almost there! We can retrieve emails from the GMail servers and deliver them locally. Now we need a way to access the email remotely. For this we're using Dovecot, and we need to tweak the configure a little.
• As root, open up /etc/dovecot/dovecot.conf and update the following lines:
  protocols = imaps mail_location = maildir:~/Mail

6. Setup maildir

• We need to create the local mailbox with: maildirmake.dovecot ~/Mail

7. Start services

Okay, now everything is set up we need to start up the services: sudo /etc/init.d/fetchmail start sudo /etc/init.d/dovecot start

8. Enjoy!

That's it! Run tail -f /var/log/syslog to check fetchmail is receiving messages and also check ~/Mail/procmail.log to see that procmail is not throwing up any errors. The final thing you need to do is make sure you can access your email server remotely. Dovecot is running over SSL on port 993, so just configure your router to enable external access to that port. Then just configure Outlook , Thunderbird, etc. to access IMAP over SSL and you have full access to all your emails on your own server. Note: if you chose to setup GMail POP access to allow all existing messages to be downloaded and you have lots of messages in your account, it will take a few runs of fetchmail to collect them all. Leave things running overnight and check back in the morning!

Optional step: Setup mutt

• Create the file ~/.muttrc and put the following in it:
  set mbox_type=Maildir set folder="~/Mail" set mask="!^\\.[^.]" set mbox="~/Mail" set record="+.Sent" set postponed="+.Drafts" set spoolfile="~/Mail"
• Now just run mutt and read your email!

Related Links

• Set Up a Debian or Ubuntu Machine as a Maildrop
• Dovecot
• Fetchmail
• Procmail

Permalink | Leave a comment  »

After playing Google App Engine for a few days, I've realised the power that even the basic framework provided by Google, webapp, provides. Working with Python adds a level of flexibility that I never had when working with PHP many years ago (although this was before OO had really infected PHP). While working on my first real application for app engine I soon came across the problem of creating sensible key names for objects. If you aren't familiar with the data object models in app engine, each object has a unique key, generated automatically when the object is created, and a key name (property: key_name) which can be specified by the user. The key name can be very useful if you need object identifiers which are human-readable, such as when you use them as part of your apps URLs. All data objects are defined as Python classes which subclass the db.Model class, provided by Google. A simple example, from the app I'm working on, is an object which holds information about an author:

class Author(db.Model):
  first_name = db.StringProperty(required=True)
  middle_name = db.StringProperty()
  last_name = db.StringProperty(required=True)

Creating a new author is a simple matter of calling the class and passing property values either as keyword arguments or by accessing the property through the new instance:

a = Author(first_name='John',
           last_name='Smith')
a.middle_name = 'Thomas'

We can specify the optional key_name property when we create the object:

a = Author(first_name='Alan',
           middle_name='Simon',
           last_name='Jones',
           key_name='alansimonjones')

However, it's more than likely that if we want to use the key_name property for a class of data objects, we will generate it procedurally based on the initial property values of the object. Above, we create the key name using the authors names. You could create a function which wraps the creation of the author object which does this for you:

def newAuthor(first_name, last_name, middle_name = ''):
  parts = [first_name, middle_name, last_name]

  # Define the key name from the authors names.
  # We split() and join each name to remove any spaces
  key_name = ''.join([''.join(x.split()) for x in parts]).lower()
  return Author(first_name=first_name,
             last_name=last_name,
             middle_name=middle_name,
             key_name=key_name)

but this is quite a lot of boilerplate code to solve a simple problem. Moreover, you need to make sure everyone working on the code is aware that this function must be used to create new author objects, rather than by calling the object class directly. A more elegant solution is to encapsulate the creation of the key_name property in the class definition itself. This is bread and butter stuff for any Pythonista, and is handled by defining a constructor for our author object, using it to define our key name and then to manually call the constructor of the db.Model class. The last step is necessary as once we define our own constructor, the constructor of the super class is not called automatically. To add to the things we need to handle, it's important to know that the db.Model class uses dynamic positional and keyword arguments. If that makes no sense to you, don't worry, it's simpler than you may think. Python provides an additional way of specifying standard function arguments using the *variable syntax, and keyword arguments using the **variable syntax. If used in the definition of a function, the *variable will be a tuple of positional arguments, and **variable will be a dictionary relating keywords to variables. Therefore, we must make sure our new constructor has the same parameters as db.Model to ensure 100% compatibility. Here is the updated author class, with the new __init__() function defined. It has been slightly altered because we access the keyword values using the kw dictionary and we need to check whether each of the properties have been specified or not by examining kw.keys():

class Author(db.Model):
  first_name = db.StringProperty(required=True)
  middle_name = db.StringProperty()
  last_name = db.StringProperty(required=True)

  def __init__(self, *args, **kw):
    parts = ['first_name', 'middle_name', 'last_name']

    # Define the key name from the authors names.
    # We split() and join each name to remove any spaces
    key_name = ''.join([''.join(kw[x].split()) for x in parts if x in kw.keys()]).lower()

    # Insert the new key_name property into the keyword arguments
    kw['key_name'] = key_name

    # Call the constructor of the base class
    db.Model.__init__(self, *args, **kw)

We can now create new author objects and the key names will be generated automatically and will follow our specified format:

a = Author(first_name='John', last_name='Smith')
# Will print 'johnsmith'
print a.key().name()

a = Author(first_name='John', middle_name='Alan Mike', last_name='Smith')
# Will print 'johnalanmikesmith'
print a.key().name()

Being able to alter the properties of an object or preconfigure core properties at the time of instantiation can be useful in all sorts of situations, but this particular example is one which I can see myself using a lot.

Permalink | Leave a comment  »

Anyone who needs to access a unix/linux/osx machine from windows via ssh will be familiar with PuTTY, the free ssh client. PuTTYGen, available to download here, is a handy utility for creating RSA/DSA public/private keys for authentication. If you have generated your RSA keys using PuTTYGen, and would like to use them with OpenSSH on Ubuntu, you just need to follow a few simple steps:

• Install putty with: sudo apt-get install putty
• Create the public key file:
  puttygen /path/to/puttykey.ppk -L > ~/.ssh/id_rsa.pub
• Create the private key file:
  puttygen /path/to/puttykey.ppk -O private-openssh -o ~/.ssh/id_rsa

You should now be able to log into an SSH server using your private key. To install your public key simply copy from ~/.ssh/id_rsa.pub on your local machine to ~/.ssh/authorized_keys on the remote server.

Permalink | Leave a comment  »

If you use Matlab for audio research / development, you'll be familiar with the pretty poor native support for audio playback. As part of my research, I wanted to play multichannel (>2) files at a variety of sampling rates and bit depths. In addition, I also wanted to use a low-latency audio layer, such as ASIO, to take direct advantange of my audio hardware. So, with a bit of coding I managed to put together a Matlab extension which allows for the playback of multichannel sound files using any soundcard with an ASIO driver. The plugin uses ASIO, PortAudio, and libsndfile. Samples are read from the audio file as needed, so playback start time is not dependant on the length of the file - useful in time-critical applications. Memory usage is also low because the whole file doesn't need to be loaded into memory, only enough to fill the audio buffer. You can download it over at the Matlab file exchange my bitbucket page and the source is included under the GPL. Comments and suggestions welcome!

Permalink | Leave a comment  »

It's quite common in University departments for roomfuls of very capable PCs to sit idle from 5pm until 8am everyday and all weekend. Thankfully, it's possible to turn such a wasted resource into a powerful cluster of Linux machines using freely available open source software. In fact, even a group of machines in an office, or at home, can be transformed into a similar state with the minimum of effort. By booting the PCs (or nodes) over the network, no changes are made to the current setup of the machines, so reverting back to their normal state is as simple as rebooting. This guide will bring you through the process of creating a High-Performance Computing (HPC) cluster using the latest version of the Debian GNU/Linux operating system, Diskless Remote Boot in Linux (DRBL), and Sun's Open Source N1 Grid Engine 6.

Requirements

Creating a Linux cluster doesn't require a huge amount of resources, either in hardware or in time, but there is a bare minimum of hardware that you need to get started:

1. A moderately powerful (at least a Pentium 4 with 1Gb RAM) machine to act as the master node. This really needs to be a dedicated machine, as this will allow users to submit jobs whether the cluster is currently running or not.
2. The master node obviously needs a network card, but if you want to give the master or the nodes internet access, or are planning to have more than about 40 nodes, you will need two or three network cards. Even high-end Gigabit ethernet cards are cheap these days, and the way your nodes will communicate with the master can make or break a successful cluster.
3. Network infrastructure to connect all of the nodes together. A 100Mbit network is the absolute minimum, particularly if you will have more than 10 nodes. Gigabit ethernet is recommended.
4. One or more node PCs which support PXE booting (network booting). If the bios doesn't have an option to boot from the network card, or you don't see any messages about netbooting when you start the PCs, then you need to install a card which supports PXE. These nodes don't need to have a hard drive, but if they do (and you have some spare space) then it can be used for a swap partition (virtual memory in Windows-speak). It helps if the nodes contain similar hardware (such as is found in most offices or University computer rooms), as it reduces the amount of configuration required (almost nobody likes to rebuild the Linux kernel 20 times).

Download the script files

You will need to download some scripts which help setup GridEngine with DRBL (Python is required to run them, but is installed by default in Debian)

Install Debian

For this project, I used the latest release of Debian, 'Etch'. You can download it here. I'm not going to go through the installation process, as you can find plenty of excellent documentation over at the Debian website. I suggest heading straight for the installation manual. Thanks to the excellent installation scripts for Diskless Remote Boot Linux (DRBL), you can choose whatever installation type you want when installing Debian. For example, if you want to use graphical configuration utilities to set up your system, the select a desktop install, which will include the X window system. If you only plan to access the master over SSH then choose a lightweight console-only install.

Install DRBL

Again, I won't duplicate the excellent and thorough installation guide available from the DRBL website over here. However, for the purposes of this guide it is important that you choose the Single System Image (SSI) mode when installing. This means that all the nodes download the same system image over the network which provides the minimum filesystem to boot Linux, and the rest of the system, including /home, /usr and /opt are NFS mounted (accessed over the network) on the master node. Therefore, any software and user files on the master node are available to all of the cluster nodes. To deploy new software across the cluster, the most dramatic step you need to take is to reboot the nodes, something that DRBL will even do for you. A final point here is to take extra care when configuring the network setup of your nodes. During installation, DRBL will assume that any network card configured with a private address (e.g. 192.168.*) will contain nodes. If you will have over 40 machines on the network, consider splitting them up into two groups, and install two network cards in the machine with different subnets, e.g. 192.168.1.1/255.255.255.0 and 192.168.2.1/255.255.255.0. Having a multi-homed master node will increase the amount of data that you can push out to nodes.

Install Grid Engine

Before you go any further, make sure tht DRBL is installed, and that you can boot your nodes over the network from your master node. All following steps assume a working DRBL Single System Image setup with one or more nodes. Now that you have a network of machines, the next step is to install software which will manage the distribution and scheduling of jobs on the cluster. There are several alternatives to choose from, including Torque, SLURM, and Condor.

Note: I strongly suggest that you do not install or use the Condor project on your system(s). Not only is the source code only available on request, and only to those that the maintainers feel are 'suitable', but by default the program will report information about your installation back to the Condor servers on a regular basis. There is no mention of this during the installation procedure. To add to this, the license requires that any publication which includes data that has been analysed or processed using Condor must include a reference to the Condor project. Given that there are plenty of open source alternatives available, I don't recommend using this.

UPDATE: As of 6.9.5, Condor is now released under the Apache Licence, v. 2.0. Therefore, some of the above concerns have been addressed. In particular, the source code is now available, but only when you agree to fill in personal information. However, I am still unsure whether usage information is reported to the maintainers by default. Caveat Emptor.

For this guide, we will be installing Sun's N1 GridEngine 6. GridEngine is simple to install on Linux, requires only a minimum of installed libraries, is relatively easy to integrate with an existing DRBL setup, and provides a wide range of powerful tools to manage and use your cluster. There is also extensive documentation available online. Installing GridEngine will involve three setups:

1. Installing the qmaster on the master node
2. Installing the execution host on the master node
3. Configuring the nodes and GridEngine to work together

Installing qmaster

This step simply involves installing qmaster on the master node as per the online documentation. However, before starting the installation script, there are a few things you will need to do:

• Make sure the hostname of the master (or any of the nodes) doesn't also point to localhost or localhost.localdomain. SGE objects very strongly if it does. So,if your /etc/hosts file looks like this: 127.0.0.1 localhost mymaster 192.168.0.1 mymaster.mydomain.com mymaster then you will want to change it to: 127.0.0.1 localhost 192.168.0.1 mymaster.mydomain.com mymaster
• Create a file containing the hostnames of your DRBL nodes. This will be used during the GridEngine installation process, when you will need to supply a textfile which contains one hostname per line. This information is available from your DRBL installation at /etc/drbl/IP_HOSTS_TABLE. You can generate it using: create_drbl_hostlist.py /path/to/output/hostlist

During the qmaster installation process you can safely accept most of the defaults, but make sure you choose to:

• Install as root user You don't have to do this, but it simplifies the process of getting SGE to run on the DRBL nodes. Please note that in recommending this, I am presuming that your cluster network is private, and the nodes won't be accessible by non-privileged users (i.e. not you).
• Do opt to verify file permissions
• Select to use BerkleyDB, but without a spool server
• Use the ID range suggested in the manual: 20000-20100
• Accept to install startup scripts
• Accept to load a file which contains the hostnames of your nodes. Here you enter the full path to file you created before running the install script.
• Use normal scheduling

Once it's installed, source the SGE settings file in your .bashrc with the command: source sge-root/cell/common/settings.sh where sge-root is the SGE installation directory.

Install the execution host on the master server

We don't actually want to make the master node an execution node (although you may want to), as it's job is to manage and schedule jobs on the other nodes. However, as the nodes rely on the master node's filesystem for their applications, we will set up the execution daemons temporarily so that we may run them remotely on the nodes. Again, installation is as per the documentation, but there are some important choices to make:

• Specify a local spool dir: /var/tmp/spool
• Accept to install startup scripts

After the install is finished:

• Move /etc/init.d/sgeexecd to somewhere safe
• Run update-rc.d -n sgeexecd remove (because we don't want to start at boot time on the master node)
• Move sgeexecd script back into /etc/init.d (which will make it available to the nodes once we rerun the drblpush script)
• Add sgeexecd to /opt/drbl/conf/client-extra-service where /opt/drbl is your DRBL installation directory. This will make sure that the execution host daemon is started automatically when the nodes boots so they will be available to run jobs.
• On the master node, save the current execution host config as a template with: qconf -sconf hostname > node.conf.template where hostname is the hostname of the master. Open the resulting file with a text editor and if the first line is 'hostname:' then remove it (thanks Josh). The config includes the custom local spool dir /var/tmp/spool which we want to be set for each of our nodes, so that the spool is storted locally in ram, and not on the master. If all of the nodes saved their spool information to the master via NFS, it would place a greater strain on the master node and the network infrastructure. Storing the information locally means that network bandwidth is saved for booting nodes and transferring user data/scripts for execution on the cluster.
• Finally, run: create_sge_nodeconfig.py /path/to/node.conf.template This script will create a new GridEngine configuration for each of the DRBL nodes (as defined in /etc/drbl/IP_HOSTS_TABLE) based on the template we saved. Note: it is not strictly necessary to have a seperate configuration for each of the nodes, as the main purpose is to set the spool location to /var/tmp/spool. This is also possible by modifying the global configuration parameters of the GridEngine. However, I personally prefer having the ability to change the configuration of individual nodes if necessary without having to create a new configuration for them each time. As we use scripts to create and update the configurations, it doesn't require any effort to adopt either setup.

Configure the nodes

At the moment, a job queue ('all.q') has been created on the master, because we ran the execution host install script. For this cluster, we can either add all our nodes to that queue or create a seperate queue for each node. For this setup we will have a seperate job queue on all execution hosts. We now want to remove the all.q queue from the master, remove it as an execute node, and add queues to all of our DRBL hosts:

• Save the queue config for the master with: qconf -sq all.q > node.queue.template
• We'll now modify the queue config template so that we can easily recreate a new config for each node with a script. Open the queue conf template in your favourite text editor and make the following changes: qname %HOST%.q hostlist %HOST% slots 1 Note: if you have more than 1 processor on all your nodes then you can change the number to reflect this shell /bin/bash Set this to whatever shell you want to use for jobs
• Run create_sge_nodequeues.py /path/to/queue.template This will create a new queue on each node with one slot per processor, enabling jobs to be submitted and executed.
• Delete the all.q queue on the master as we won't be using it: qconf -dq all.q
• And now, remove the @allhosts group because we have made individual queues for each node, so we don't need it either: qconf -dhgrp @allhosts
• And last we remove the master as an execution node. It was set as such by running the execution host installation script, but we can undo that with: qconf -de master

If you now run qhost -q | more you should see a list of all your DRBL nodes and below each should be a queue definition. Running qstat -f should now list the various queues that were created, and show how many slots each of them have and how many slots are filled at the moment. Now, rerun drblpush -i or drblpush -c /etc/drbl/drblpush and restart one of your nodes. ssh into the node when it boots and confirm that sgeexecd is running with: ps ax | grep sge. If it isn't, check for startup messages in /tmp/execd*. Back on the master, rerun qhost -q | more. The node that you have rebooted should be listed with extra information than the others, such as free memory, number of processors, and load. You can now boot the rest of your nodes to create a diskless cluster with GridEngine and DRBL!

Setting up MPI/MPICH support on GridEngine

If you want to configure MPI/MPICH support, there are two steps which need to be completed. First, you must add a new parallel environment to GridEngine which sets up the mpi/mpich startup and shutdown scripts that are provided with SGE for use with MPI/MPICH jobs. Second, you must add your new parallel environment to each of your already existing job queues. For this example we will be setting up mpich as our parallel environment. If you want non-tight MPI support, just replace mpich with mpi throughout:

• Make a copy of the mpich.template in the mpi directory
• Fill in the information that is missing (marked with <>) in the template, such as number of slots (just put 999) and the SGE root dir (for me it's /opt/n1ge6)
• Now add the parallel environment with: qconf -Ap /path/to/modified/mpich.template You can check it exists with qconf -spl and qconf -sp mpich
• Next, modify the node.queue.template that we created in a previous step, and change the 'pe_list' from 'make' to 'make mpich'. This will enable parallel jobs to be run on the queue.
• Finally, update all our queues (one for each DRBL node) with: update_sge_nodequeues.py /path/to/modified/node.queue.template
• You can then check the updated queue definitions with: qconf -sq <queuename> where queuename is one of the queues on a DRBL node.

Using QMon

QMon is the graphical interface to the Sun GridEngine system. To run it, you need some extra libraries installed. Unfortunately, they haven't (and probably won't) have a release candidate for Debian, so you need to get it manually from the Debian servers:

• Download the package for your install here.
• Install with dpkg -i package-name.deb
• Type qmon to start it.

Permalink | Leave a comment  »

Recently I came across this article on the Google blog about the launch of Google Web History. If you have a Google account and their browser toolbar installed, it will keep track of all the websites you visit, allow you to search through them, and give you stats on what you do online. It's like del.icio.us but completely passive and encompasses anything you do with your web browser. Thinking that it sounds rather big brother-esque? Me too. But that doesn't stop people from using it. It seems that people really do trust Google with vast quantities of information about their lives. I can just about cope with using their Gmail/Gtalk services, but I'm very conscious about what personal things I send over email, and I say on IM. The concept of keeping everything about your 'digital life' in a central place which is easily accessible and linked together strikes me as one of those great ideas that you would find listed in a 1970s 'Things technology will do for us in the year 2000' article along with replicators and hoverboards. However crazy it may seem to most of us, there are some people who take this to extremes, recording every bit of information about their lives in a digital form. While it is possible to understand how collecting so much data about yourself is an intruiging concept (remember the Truman Show?), it's a whole other thing to hand that information over to a large for-profit company, even if they supposedly don't do evil.

Permalink | Leave a comment  »

At the last meeting of the Board of the National Youth Orchestra of Ireland, a decision was made to amalgamate their two main orchestras: the NYOI (for players between 14-18) and the NYSOI (for players between 18-24). The full announcement can be found on the organisation's website. This move has come as a complete shock to many people, especially those who have had the opportunity to perform as a member of one of the orchestras. The NYOI has a proud history of providing young Irish classical musicians with the chance to perform repertoire of the highest standard on world-class stages with international conductors. I have been fortunate enough to be one of those people who have benefited from this organisation, and it is critical to the future of classical music in Ireland for this organisation to continue into the future. I believe that a decision which cuts the number of places for young musicians in the orchestra in half, from 200 to 100, is a huge mistake, and many up and coming young musicians in Ireland will have to pay the price for it. Since the decision, four board members have resigned and an online petition against the amalgamation has received over 1,100 signatures. They include some renowned musicians and important figures in Irish cultural life. Despite this, it would seem that the Board of the NYOI are utterly convinced that this is the only course of action that will resolve the current recruitment problems they refer to in their announcement. This was made clear in a discussion on today's Liveline show on RTE Radio 1, involving Gerard Kelly, a Board member of NYOI, and Donagh Collins, who resigned from the Board in protest at this decision. You can listen to a recording of the broadcast here. (Please note that the recording begins shortly after the start of the NYOI feature, and contains a cut after a minute or so due to a problem connecting to RTE's streaming server. I'll put a complete version online as soon as the show archive is available. Updated to include the full interview.) I feel really angry about the way in which the Board of the NYOI has made this decision, with no open consultation with anyone outside the organisation. This move to join the two orchestras will not only halve the number of places available to young musicians, but it will also impair the ability of the NYOI to provide an environment in which young and inexperienced players can develop over time into experienced orchestral musicians. Perhaps at this difficult time for the organisation, the Board of the NYOI should consider focusing their efforts on finding a new Chief Executive instead of making rash decisions which will have a strong detrimental effect on classical music in Ireland. This is a tremendous mistake and if you agree, please sign the petition now!

Permalink | Leave a comment  »

After reinstalling windows recently I once again had to battle with the process of installing Django on Windows XP. All I remember about the last time was several hours of googling (if that isn't a word ti aught to be) and cursing at the *nix-centric installation notes on the django website. So this time I decided to take notes so that I could put together a semi-coherent guide to installing Django on windows for development purposes. Here goes... As I am interested in keeping up with the latest features of Django I'll be using svn to grab the latest snapshot. To use any the official releases instead just download and extract them into a folder of your choice and skip forward to step 3. Step 1 - Set up your Python paths First, check that the main Python folder and the scripts folder are in your system path. To do this, right-click on My Computer, and select Properties -> Advanced -> Environment Variables. Edit the 'PATH' variable and add your Python and Python/scripts paths if necessary. Close any open command-line windows and open a new one (Start->Run->cmd.exe). If you can run 'python' and see a python prompt then you're ready to continue. While you are editing your environment variables I would also suggest adding '.PY' to the PATHEXT variable. This allows you to run any python script without having to add the '.py' extension to the end of the command, e.g. 'django-admin.py' becomes just 'django-admin' Step 2 - Get Django First, make sure you have python and svn installed and in your PATH so that you can run them from any folder. Then run the following:

svn co http://code.djangoproject.com/svn/django/trunk/ django

This will check out the latest development code and place it in a folder called 'django'. Using svn means that we can update the code to the latest version at anytime by moving into the 'django' folder and running

svn update

Nice, eh?! Step 3 - Install setuptools Move to the folder where you checked out django and run:

python ez-setup.py

This will install setuptools which is needed to easily install Django in your current Python installation. Step 4 - Install Django Finally we want to link Django to the current Python install (version 2.4 is recommended here). To do this, once again make sure you are in the django folder, and then execute:

python setup.py develop

This will put the django-admin.py script into your Python scripts folder and will create a link to your new django source code install in a .pth file in site-packages (the usual location to install Python modules, etc.). This is the magic bit which allows you to check out django in any location on your PC that you want and be able to access it as a Python module without any messy configuration files. And that's it! You should now be able to run 'django-admin.py' without any errors.

Permalink | Leave a comment  »

I recently came across a problem when adding user accounts to a Django application I'm currently working on: I wanted to access information about the current user, e.g. whether they were anonymous or logged in, so that I could display a login message or a link to their homepage respectively. This sounds like a relatively simple thing to accomplish but the problem was that I didn't want to have to look up the current user's information in every view method and pass it to the template. Luckily Django provides a way of dealing with this very situation by providing a special template 'context'. From the Django documentation:

A context is a "variable name" -> "variable value" mapping that is passed to a template.

The special context here is called DjangoContext (no great shock there) and it provides a few new variables to the template (again from the official documentation):

• user -- An auth.User instance representing the currently logged-in user (or an AnonymousUser instance, if the client isn't logged in). See the user authentication docs.
• messages -- A list of auth.Message objects for the currently logged-in user.
• perms -- An instance of django.core.extensions.PermWrapper, representing the permissions that the currently logged-in user has. See the permissions docs.

To use DjangoContext start by importing it using the following line at the top of your views.py file:

from django.core.extensions import DjangoContext

It was at this point that the problem arose: the documentation is fairly cryptic about exactly how the context object should be used when building a template. It was even more confusing because I am not using the standard method of rendering a template which involves 1) loading a template, 2) creating a context object, and 3) rendering the template using the render() method of the template. Instead I'm using render_to_response again from django.core.extensions. This function takes the template name and list of template variables as a dict and does all the work for you in one line. Nice. But how do I take advantage of the DjangoContext context which provides all that lovely user information while still using render_to_response too keep my code nice and clean? Like this:

return render_to_response('template_name', {'var1': var1, 'var2': var2}, context_instance=DjangoContext(request))

This renders the template template_name, passing it the variables var1 and var2 but also tells Django to use the new context instance DjangoContext. In the template could then check if the user was logged in with:

{% if user.is_anonymous %}

Simple!

Permalink | Leave a comment  »