Twittering OCR

'things that go bump in the tunnel'

As I've beem following the LHC restart I've written a parser for the vistar status feed to send it to twitter. The basic method is:

Grab URL (see image) then do some imagemagick hackery to cut out the corner. Resize larger (helps with the OCR) and save as tiff. Run the image through OCR software, compare the output to the last run, if different then upload to twitter.

ie

curl -o $IMG $SRC
convert $IMG +repage -crop 509x205+1+533 -resize 1000x -threshold 39000 $IMG
convert -monochrome $IMG $TIFF
mv $OUT.txt $OUT.old # make a backup of old
tesseract $TIFF $OUT

# Strip out ready for Twitter
DATE=`date +%d-%m-%Y`
sed -i "s/Comments $DATE /#LHC Status /" $OUT.txt

diff -q $OUT.txt $OUT.old
if [ $? -eq 1 ] ; then
# Post to Twitter.
curl --basic --user lhcstatus:password --data status="`cat $OUT.txt`" http://twitter.com/statuses/update.json
fi

and lo: http://twitter.com/lhcstatus

swisscom VDSL & dyndns

Those of you who follow my twitter stream will have noticed that I managed to 'lose' my home machine today. It was online and active, I just didn't have its IP address (somwhere in the swisscom pool)

I thought I'd set up the embedded dynamic-dns client on my router, a Swisscom provided (motorola) netopia VDSL modem:

Terminal shell v1.0
Copyright ©2008 Motorola, Inc.  All rights reserved.
Netopia Model 7347-84 AnnexA VDSL Wireless Ethernet Switch, PBC
Running Netopia SOC OS version 7.8.2 (build r3)
VDSL Capable

Turns out that you have to save your username not your email address in the client, even though both are valid in the dyndns.com web login.

so the correct settings for those googling is:

Netopia-7000/XXXXXX> conf

Config Mode v1.3
Netopia-7000/XXXXXX (top)>> 
set dynamic-dns option dyndns.org
set dynamic-dns ddns-host-name "DYNHOSTNAME"
set dynamic-dns ddns-user-name "USERNAME"
set dynamic-dns ddns-user-password "PASSWORD"

subsitituting the things in caps for your own
then do a validate, save, restart and lo:

later on in 'view log' I see:

10/5/09 08:01:05 PM L3      DDNS: initializing service
10/5/09 08:01:23 PM L3      DDNS: Hostname updated. good

tada!

Studies on a hashtag

I'm at the EGEE09 conference this week, and they're also plugging use of tagging in social networking such as the #egee hashtag in twitter and the 'egee09' tag in flickr. As a small experiment I created an IRC channel on freenode (#EGEE09) and announced it with a hashtag on twitter. Within 3 minutes there were an additional 3 users, only one of whom was in my normal twitter cloud, ie the other 2 came in purely from the hashtag (and one of those was under a minute later)

so, twitter is certainly becoming a respectable way of broadcasting announcements to an interested group of people who are subscribing to a hashtag.

oh and as an aside, the IRC channel paid off -- I discovered that GridPP had some nice laptop bags via it and secured one for myself during coffee.

Anyway, back to more important work -- listening to how the EU proposes to carve up the funding for call7

BeautifulSoup + savannah

At work, we use savanne to power http://savannah.cern.ch where we keep track of all the glite-middleware (amongst other things)

However, one of the problems with savanne/savannah is that it doesn't have a nice machine readable API, but requires the user to click with a browser.

Cue Beautiful Soup and lo, the python SavannahCLI was born.

Well, in its early years it was unloved and handed round from developer to developer, until it arrived as a pedantic teenager to yours truly who decided to get rid of the horrible 'operation, item, name, value" syntax and replace with something more, um, fluid instead. (--patch 1234 --set foo=bar). It also had a huge set of hard coded lookup tables for converting select items to names and vice versa -- WHY?

thanks to some googling, I came across this which, with a little refinement gave me:

def enum_forms():
  # we search within the table tag so that we lose the LHS searchbox
  selects = soup.find('table').findAll('select')
  for sel in selects:
     sel_name = sel['name']
     print "SELECT NAME=%s" % sel_name 
     for o in sel.findAll('option'):
         print "  %s, %s" % (o['value'],o.renderContents())

hey, it's not finished yet, but sure saves a whole pile of hard-coded badness.

SPA3102 and Swisscom

I've had a Linksys (nee sipura) spa3102 for a while (worked fine in the UK with sipgate) but havent got round to configuring it for switzerland yet.

Idiots guide seeing as I had trouble finding this on the net:

1) factory reset: ****, 73738#, 1#, hangup
2) make sure you have a computer in the NAT socket ('Ethernet') - I use mine bridged or into a main switch
3) http://192.168.0.1 and reset Router > WAN Setup > Enable WAN Web Server > YES
4) Router > LAN Setup > Networking Service > BRIDGE

http://www.3amsystems.com/wireline/daa-search.htm?country=Switzerland helps a little
as does http://switzernet.com/public/070723-config_UA/spa-3102.php

Things I've changed are
Voice > Regional > TimeZone > "GMT +1"
Voice > Regional > FXS Port Impedance > "270+750||150nF"
Voice > Regional > Caller ID Method > ETSI FSK
Voice > Line 1 > Dialplan > (<#:>xx.<:@gw1>|xx.<:@gw0>)

This dialplan is because I wish to use swisscom pstn most of the time, but prefix with # if I want the call to go over the VOIP service

Gimmie, Gimmie, Gimmie (a patch after midnight)

...won't somebody help me chase those bad docs away.

As I recently mumbled on twitter (sorry, but 'tweet' is even naffer than the bad title of this post) I've started poking at Yum around the edges. Partially as I'm trying to find a 'Better Way' than the groupinstall we have to do for 64/32 bit package co-existence on the current middleware, but also as I'm narked off at the complete lack of manpages (on SL5) for yum-utils.

actually, thats a bit harsh - as of 1.1.16 (el5) there are minimal docs, including yaim-utils.1 but nothing for each individual utility (see also http://yum.baseurl.org/wiki/YumUtils/RepoDiff). So, once again I'd ideally like something that can parse wiki markup and give me the output without having to keep pasting into a test page and hitting preview.

wikilint anyone?

Anyway, 1st patch sent off for approval, but my home server has dropped off the net so no more updates for the day.

[obligatory swiss status report]
Hot. :-) Sunflowers are blooming. nice n fresh in the evenings with thunderstorms. Sadly not had a chance to see if my shiny new camera can catch lightning, but I'm reasonably impressed with the highspeed capture. see sample

SVN Stats munging

As an aside I recently looked at a piece of GPL'd software and analysed the subversion stats for it.
Each datapoint represents a checkin to the tree and I assigned a numerical ID to each ssh account. shows which developes 'stick' with a package and which abandon their updates.

Bonus points for working out what the code is :-)

In search of Feduntu

I've been a happy Ubuntu user for a while on my HP Compaq 8510p laptop, but a few things recently really started annoying me.

1) regular ubuntu system upgrades don't include openafs-modules for that kernel, which means I have to recompile and install them before I can access cern.ch AFS space.

2) it *really* doesn't work well with the ATI chipset in my laptop - full screen of a video hangs the machine with fglrx (not checked launchpad for bug no)

3) since the last update it takes 2 *seconds* to minimise / maximise / alt-tab with windows. c'mon, this is a dual core machine..... (see bug #351186

4) The fingerprint reader didn't work.

5) ... neither did userfriendly xinerama.

OK, so most of these problems are caused by the video driver (closed source fglrx) but since Fedora 11 came out I thought I'd compare and see what the opposition had (especially useful since I use RHEL clones at work - scientific linux 5 or 4)

Installer -- worked wonderfully *once I'd managed to get the iso onto a USB stick -- I was using unetbootin (on Jaunty) but I hadn't formatted the stick as fat32 -- didn't work with an ext3 formatted stick. The LVM selection was a bit fiddly (I have /boot(sda1) and linuxvg(sda2) and within the VG I have homelv,rootlv and afscachelv (was ext2 as I used XFS for / and /home)

F11 wanted me to migrate from xfs to ext4 for / -- OK not an issue, but its SElinux policy seems screwy wrt the /home

I get home directory does not exist, logging in with HOME=/
but then can immeadiatly cd to $HOME. harumph.

also the Fan runs constantly, so battery life is low, Especially since it suspends as soon as *one* of the batteries goes flat (see bug 220962)

in short, Fedora 11 fixes some of my issues. looks really nice, but just doesn't do what I want (*decent* laptop distro that lets me work for a long time). So I'm afraid I shall revert to Ubuntu over the weekend - it's still buggy for me, but in less critical ways (hopefully a fresh install of 9.04 will blow away any issues of the video driver rather than the upgrades.

Maybe again in Fedora 11.1 ?
Why not windows? well, I *like* using the CLI for most of my work, Putty + gvim + development languages are good but I'd need a virtual machine to test any linux stuff out on laptop. I don't actually have a need of any windows-only software, so why pay the licence.

[OK - tomtom home is the only thing, but thats another story...]

twitter

OK, so it's been a few weeks since I started using twitter. The best uses have been suitably demonstrated by those fine folks at b3ta but I thought I'd rant my findings so far.

In it's original use as some sort of "hello, I'm on the train" - it totally fails to impress me. Why bother? I'm following a few friends, and no offence to them, but the parts of their personal lives they choose to share are um, not gripping. If I want to know what X is doing, I'd rather a) phone em up or have some other realtime converation or 2) look for a longer than 140 characters blog posting.

However where I *do* find it useful is for a day to day scratchpad of stuff I've done to gather together for my weekly report. I'm sure the public at large don't give a monkeys what I'm up to, and my use of bug #1234 no doubt screws up hashtags something rotten, but hey, for me it works.

The alternative is some sort of cgi elog thing, but the advantage of twitter is the open API that other people have already spent time working with. Twitterfox 'just works' for me.

Ho Hum, there we go, somewhat over my 140 charaters, but if you're really sad you can follow me on http://twitter.com/elwell2000, or look over on the right of the page.

Scientific Sailing

*why* can't the UK do something like this?

I remember years ago when Curly Mills kindly donated Alba Venturer to OYT Scotland we had a long discussion about how important teaching basic science / engineering was. Big plus points for the Italians for doing the Right Thing (TM).

bacula

OK, so I work in IT. I *should* do regular backups. I know what happens when disks die. Finally (after aaaages of delaying) I set up my external USB HDD to work with Bacula. Not a moment to soon, 3 days in and I get asked do I have a copy of X file from the PC at home. clickity click, restore, find file, select restore to different machine, run. and lo...

  Job:                    RestoreFiles.2009-04-30_14.27.48.19
  Restore Client:         shinybox-fd
  Start time:             30-Apr-2009 14:27:50
  End time:               30-Apr-2009 14:28:06
  Files Expected:         1
  Files Restored:         1
  Bytes Restored:         42,212
  Rate:                   2.6 KB/s
  FD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Restore OK

tada! scp'd and mailed. :-)

No. Just 'NO' OK!

I may be proved wrong on this, but why do a remake of *such* a classic, the Edge of Darkness. I mean, the late Bob Peck vs 'Mel Gibson ... a homicide detective for the Boston Police Department."

Ho Hum.
/OldFart

munin SNMP

Blogging this as the munin docs suck a bit -- lovely and technical but no HOWTOs

anyway I have an out of the box debian munin install on my epia box (hostname mythic) and I wanted to also plot the network traffic on my swisscom VDSL router (which is SNMPable)

Magic incantation to /etc/munin/munin.conf is:

[mythic.localnet]
address 127.0.0.1
use_node_name yes

[router.localnet]
address 127.0.0.1
use_node_name no

(where I have a decent /etc/hosts for localnet) and then do

munin-node-configure-snmp router.localnet | sh

New title

Decided to rename my blog in honour of Globus-gatekeeper

it's a grid thing....

X marks the spot

Passed a google streetview car on the way home tonight - it was heading southbound on the A1 at ~15:28.
View Larger Map

Video killed the MSN star...

Flippin closed protocols. Discovered that there's a sneaky tickbox in the account settings on the Acer Aspire one messenger client.

The Webcam features are disabled when you enable the "Use a Windows Messenger compatible service" in the settings.
If you go into the Account settings for your MSN account and make sure that isn't ticked, then reconnect and try again.

(thanks to an acer forum for that one.)

Virtualbox and KVM

At the moment I'm using virtualbox on my Ubuntu 8.10 laptop for running windows (very occasionally) and some SLC test machines for work. Oh and a few other Linux Distros too. Anyway, much as I like the point n clickiness of virtualbox I thought I'd see what the 'built in' KVM could do, esp as libvirt looks like a nice abstraction layer.

However on starting virtualbox machines I get the error:

VirtualBox can't operate in VMX root mode. Please disable the KVM kernel extension, recompile your kernel and reboot (VERR_VMX_IN_VMX_ROOT_MODE).

well, thats not strictly necessary - a simple sudo rmmod kvm_intel kvm and relaunching the virtualbox guest.

Easy in Eze

Went down to the coast last weekend for the Nice Carnival. Stayed in Eze - nice medievel town on the hill. Stitched a pile of images together to try and get the scale of the hills, but managed to leave a gap sadly.

koha!

Spent a bit of time tinkering with koha in my spare time recently. Threw in a few trivial patches (once I'd got my head round git). However Some notes here for those installing on Debian Lenny (current 'testing'

1) Nearly all the required perl modules are available directly from the repos, the exception being SMS::Send (which needs Class::Adapter) -- Thise can be installed with the ever so handy dh-make-perl which converts your CPAN into .debs (sadly it still doesn't help with other conversion issues).

# dh-make-perl --build --cpan Class::Adapter ; dpkg -i libclass-adapter-perl_1.05-1_all.deb
# dh-make-perl --build --cpan SMS::Send ; dpkg -i libsms-send-perl_0.05-1_all.deb

2) I'll write up the rest as a HOWTO soon....

up up and away!

Spent weekend up at Leysin and Chateau D'oex for the baloon festival. Sat was grey and snowy / windy so not much flying. Sunday was superb tho - pics on flickr

KITT updated?

OK Wishlist time.

I'd like some sort of navigation system for the car. Yes I could buy a tomtom or suchlike, but whats the geek factor in that. Nope, what I'd really like is some sorta carpc. Now that Atom based motherboards are cheap, and SSD flash (certainly in the small storage capacity I'd need) are cheap(ish) this is becoming a sort of sketch plan. I'd still need to get a nice touchscreen and interface knocked up, and have a connection to the car amp (manufacturer fitted nice system)

So in no particular order:
* Touchscreen fitted 'neatly' in car
-- it needs to have big 'bloke sized' buttons on the screen for doing tasks, so I don't have to look away - do you look down at the car radio?
-- some sort of scriptable interface: eg 'upload all tracks since the last upload'
* GPS receiver
-- Datalogging for openstreetmap
-- Navigation (doh!)
-- Geofencing to allow comms when parked at home
* Wifi and or GPRS/HSDPA connectivity
-- limited storage in the car, download $music from home store - treat it as a small player rather than a full rsync copy.
-- kismet anyone?
-- remote upload of tracks
-- download of traffic reports
* OBD connection for datalogging.
-- correlate engine conditions to position - if you're revving the engine is that because you're going up a twisty hill or because you're a numpty.
* Accelerometer (wiimote velcro'd to dashboard?)
-- Get relative acceleration of car, not just position

Oh, and it should be cheap :-)

TODO: 2009

Following mrbens posting, I thought I'd knock up a quickie list of my own for comparison next year

• Learn French. This has to be a priority seeing as I wish to stay out here.


• Get a more permanent job - EGEE funding only lasts till next May...


• Get the mini-itx working in some media-server form


• Do stuff with Asterisk


• Improve Skiing


• Do more mapping with OpenStreetMap around the area.

That should keep me occupied for a while :-)