Backstage: Get with the Program

Posted in: Software, Tech

There was a neat idea recently on the BBC backstage list to produce a tag cloud of words in subject lines. There was an implementation of this but as usual with almost everything posted on Backstage no code.

It's frustrating that the BBC go to all the trouble of making their data available and then developers horde their little snippets of code. I just don't understand it. Big kudos to MighTyV who do open source their code. And deservedly won the BBC Backstage competition.

So, without further ado, some code to implement tag clouds off mailing list subject lines:

APIs Archive BBC BetaNews Calendar Celebdaq Clips Creative DRM Feeds Google Grauniad Hourly How IM Mashup Newspaper RSS Random UK Weather Would alternative anyone backstage best bot bots can data feeds flashes for homepage iMP idea indexing media models module monday news page posted radio serve someone stations use via

(I generated from a mailbox earlier in the year which has more topical subject lines.)

#!/usr/bin/perl

use warnings;
use strict;

use Email::Folder;
use HTML::TagCloud;

my $mbox = shift || "$ENV{HOME}/Mail/Lists/Backstage";

my $cloud = HTML::TagCloud->new;
my $folder = Email::Folder->new($mbox);
my %word_count;

foreach my $subject (map { $_->header("Subject") } $folder->messages) {
	my @words = grep !/^(to|you|we|the|and|would|on)$/,
	            grep /^\w+$/, split ' ', $subject;
	$word_count{$_}++ for @words;
}
$cloud->add($_, 'http://realprogrammers.com/', $word_count{$_}) for keys %word_count;

print $cloud->html_and_css(50);

(I'm happy to comment this for anyone wishing to play with it.)

Posted by Paul Makepeace at November 1, 2006 12:49 | TrackBack