dark clouds

del.icio.us is closing. If you use it, you may want to head over to https://secure.delicious.com/settings/bookmarks/export and grab your bookmarks...

Fedora Packaging

OK, so I needed to get my touchatag reader working under fedora, and RFIDIOt needs pyscard. Much yak-shaving later I've come up with a .spec file for pyscard, and its now ready for review.

spec file - http://dl.dropbox.com/u/6594808/Fedora/pyscard.spec
src rpm - http://dl.dropbox.com/u/6594808/Fedora/pyscard-1.6.12-1.fc14.src.rpm

oh well, lets see who'll pick up my Review Request

ssh-copy-id

I normally have my ~/.ssh/config so that it only uses my key or gssapi to login to other sites, however this poses a problem when reinstalling a server and don't yet have my key on it

ssh -F /dev/null root@hostname

works, but this wasn't working with ssh-copy-id.

Turns out that the ssh-copy-id script is braindead and only expects one or two (with the 1st beginning with -i) arguments.

lo and behold, some single quotes did the trick:

aelwell@pcitgtelwell:~$ ssh-copy-id '-F /dev/null root@concerto'
Are you sure you want to continue connecting (yes/no)? yes
root@concerto's password:
Now try logging into the machine, with "ssh '-F /dev/null root@concerto'", and check in: ....

tada!

upgrade fail

No I'm not hiring an expensive car, but I certianly won't be using carhire3000 if I do...

Dear McDondalds,.... [RANT]

Dear McDonalds. If I wanted to wait over 15 minutes (from placing order to collection) for a simple meal (burger, drink and fries) I'd have gone to a REAL restaurant. Note, advertising your franchises as 'McDonalds Resturants' does not IMHO make it a real resturant. Oh, and teach your staff that >900 seconds on your timeclock is indeed 15 mins, so when I complain, point to said display (twas one of those place order and collect screens) and complain its been over 15 mins "No it isn't" is the Wrong Response. (see LART for possible reeducation tool)

Oh, and tell one of your other stores about NTP. Setting a computer clock is not difficult. Apparently I ordered at 12:39:40 in that store, but they managed to authorize a bank transaction at 12:32:16 (6 minutes before ordered. Impressive. Telepathic payments. Still, at least I got served at 12:33:20. again, before I'd ordered...

VoIP en France

We've moved to France, with all the normal hassle over paperwork, etc. Done the signups for utilities, then had to choose a phone company.

France has the rough equivalent of a LLU line 'degroupe' [no I haven't got punctuation dorted yet] see Wikipedia. and many of the ADSL modem/router/wotsits include an ATA in them.

So - at home we went with Orange, cos like the incumbent should be fairly reliable right.... ha!. If the VOIP line dies (we don't have a 'real' POTS line) then you have to do a factory reset of the livebox2, re-add in all the login details and any other customisations, and wait.

Oh, and they don't publish the SIP info for the ATA, so I can't even use a hardware voip phone for those times when internet up but the livebox voip service is down.

I've gone with (hopefully - still waiting for the confirmation) free.fr for the flat. lets see if they're any better...

more data musings

(the advantage of traveling by public transport once in a while is you can sit and faff on laptop)


More data musings

The only guarantee about user entered data is that, given enough entries it'll be inconsistent :-(

take for example an openstreetmap xapi query to pull out '/api/0.6/*[amenity=post_box]'

which is nice dataset of ~85k enties which I'll use for some simple analysis

So, the UK has ~40k postboxes, of which according to draco the breakdown of entries from the count are sources as follows:
13.5k - osm, 26.7k - website.

so of those 13504 UK postboxes in OSM, how many are royal mail run (hint - most of them!)
does the data match?

$ grep "operator" ~/Downloads/data.osm | sort | uniq -c | grep -i royal
      1     <tag k='operator' v='Post Office: Royal Mail'/>
      1     <tag k='operator' v='royal mail'/>
      1     <tag k='operator' v='Royal mail'/>
   5065     <tag k='operator' v='Royal Mail'/>
      1     <tag k='operator' v='RoyalMail'/>
      1     <tag k='operator' v='Royal MAil'/>
      1     <tag k='operator' v='Royal Mail Warwick'/>
      2     <tag k='operator' v='Royal York'/>

not bad - only a few CaSe sEnsiTive issues to sort out

What about other operators, say La Poste?

$ grep "operator" ~/Downloads/data.osm | sort | uniq -c | grep -i poste
      1     <tag k='operator' v='Bureau de poste'/>
      1     <tag k='operator' v='De Post  -  La Poste'/>
      7     <tag k='operator' v='la poste'/>
     21     <tag k='operator' v='la Poste'/>
     12     <tag k='operator' v='La poste'/>
    917     <tag k='operator' v='La Poste'/>
      1     <tag k='operator' v='La Poste Belgique'/>
      6     <tag k='operator' v='La Poste - De Post'/>
      1     <tag k='operator' v='La Poste Suisse'/>
      1     <tag k='operator' v='Le Poste'/>
      1     <tag k='operator' v='poste'/>
      5     <tag k='operator' v='Poste'/>

again - it's the 'long tail' problem. So, out of the ~85k entries how many unique operators?
404 (how apt for a web service)

and of those how many are singles? 222 - OVER HALF!

m m m metadata!

OK, in a semantic web kinda way, I've been looking at some of the clever machine tag integration that flickr are doing, and thinking about how these things *should* automatically link up.

Take for example http://www.dracos.co.uk/play/locating-postboxes/ and look at http://www.flickr.com/groups/postboxbypostcode/

There are many excellent postbox groups already but they all have, to my mind one problem - no structure enabling anyone to find anything. This group will only contain boxes which have their postcode in the title or tags, enabling easy searching.

if you can't find it, or reference it, it's useless.

I therefore propose to tag the postbox pics with 'ukpostbox:XXX_YYY where XXX is the 1st part of the postcode and YYY is the box ID

This means that 1) things like locating-postboxes could bring up a set of pics of the boxes, flickr could automatically link to posting times (uhm but that assumes the royal mail has an API ha ha ha ha ha). If people have followed the Tagging guidelines on openstreetmap, then you can even link them directly together.

Maybe sir Tims vision wasn't so bad after all :-)

WebDAV

Here at CERN we use AFS for our home directories on Linux, but the windows stuff all uses DFS with https webdav voodoo.

Discovered on Ubuntu 10.04 that if I save a nautilus bookmark with:

davs://USERNAME@dfs.cern.ch/dfs/Users/U/USERNAME dfshome

it totally fails to connect. The correct syntax is simply:

davs://dfs.cern.ch/dfs/Users/U/USERNAME dfshome

and put in *both* your username and password when prompted.
Oh, and they're stored in ~/.gtk-bookmarks incase you need to edit them :-)

transmission + munin

OK, so things like the Rocks Avalanche Installer (pdf) show how useful bittorrent can be for providing a scalable install service. How do you know the status of bittorrent installs though? transmission-remote -l is all well and good if you're on the console but what about long term graphing? Cue Munin and a quick bit of python:

#!/usr/bin/python
# Munin plugin for monitorring transmission bittorrent status
# Andrew Elwell 
# $Id: $
import sys, os

if len(sys.argv) == 2 and sys.argv[1] == "autoconf":
        print "yes"
elif len(sys.argv) == 2 and sys.argv[1] == "config":
        print 'graph_args --base 1000'
        print 'graph_title Transmission torrent status'
        print 'graph_vlabel Count'
        print 'graph_category Transmission'

        print 'Stopped.label Stopped'
        print 'Stopped.draw AREA'
        print 'Idle.label Idle'
        print 'Idle.draw STACK'
        print 'Seeding.label Seeding'
        print 'Seeding.draw STACK'
        print 'Up.label Up and Down'
        print 'Up.draw STACK'
        print 'Downloading.label Downloading'
        print 'Downloading.draw STACK'
        print 'Will.label Will Verify'
        print 'Will.draw STACK'
        print 'Verifying.label Verifying'
        print 'Verifying.draw STACK'

else:
 # OK we cheat - initialise ALL values to 0, then overwrite with 'real' ones, then spit out the whole dictionary.
 data = {'Stopped': 0, 'Seeding': 0, 'Idle': 0, 'Up': 0, 'Downloading':0, 'Will':0,'Verifying':0 }
 for state in os.popen("transmission-remote -l |sed 's/^\s\+//g' | sed 's/\s\{2,\}/~/g' | cut -d '~' -f8 | sort | uniq -c | grep -v '1 $' | grep -v '1 Status' | awk '{print $2, $1}'"):
  (s,c) = state.split()
  data[s] = c
 for k,v in data.iteritems():
  print k + '.value' ,v

EDIT: Suppose I should really find a nice template that copes with embedding code, but in the meantime thats what 'view source' is for...

Kerberos & SSH at CERN

Quick blog posting to save me having to explain to people what's needed:

1) RTFM: http://linux.web.cern.ch/linux/docs/kerberos-access.shtml -- thats where most of the good debugging tips are

2) to get it working between say your ubuntu laptop and cern hosts you'll need to append

allow_weak_crypto = true

to /etc/krb5.conf [libdefaults] section. (see bug)

3) make life easy and put a few things in your ~/.ssh/config

host *
        Protocol 2
        VerifyHostKeyDNS yes
        VisualHostKey yes
        GSSAPIAuthentication yes
        PreferredAuthentications gssapi-with-mic,publickey

host lxplus
        hostname lxplus.cern.ch
        ForwardAgent yes
        GSSAPIDelegateCredentials yes
        GSSAPITrustDNS yes

(despite what the man page says there are NO SPACES between the options in PreferredAuthentications (see SSH bug 1702)

killed

I rarely have a need for bluetooth on my laptop, so mostly have it disabled. however when I was running Fedora 12. I had a nice option somewhere to leave bluetooth disabled and use the kill switch on my HP Compaq 8510p to purely turn off the wifi (advantage is that it also means the LED indicates wifi activity, not just always on)

However since migrating to Ubuntu Lucid 10.04 I can't reenable the bluetooth. All I have in the gnome widget is 'Your computer does not have any Bluetooth adaptors plugged in.' Grrr. Time to boot off a CD methinks and fix this.

yes, Yes, YES!

*something* at home on my linux box is doing alot of DNS lookups

grep 'yes.elwell.org.uk from 127.0.0.1' /var/log/daemon.log | wc -l
13284

and I have no idea what's responsible. (googling for yes and DNS is too vague)
Anyone any ideas?

Update Sussed it
smb.conf had "wins server = yes" not wins support....

concerto + twitter

During the trials of concerto at CERN, I wanted to make the text fields a bit more dynamic like say following a specific twitter feed.

This would (ideally) simply consist of a DB entry with type=twitter, value=name of person. In the Real World however I got round this by hacking up a simple python script that connects to the DB and does a simple UPDATE of an existing text/plain entry via cron.

posted below incase others find it useful:

#!/usr/bin/python

# Script to pull twitter feeds for cern / lhcstatus
import twitter
import MySQLdb


conn = MySQLdb.connect (host = "...", user = "...", passwd = "...", db = "...")
api = twitter.Api()
feeds = { 'cern': 25, 'lhcstatus': 28 }
for f in feeds:
 tweets = api.GetUserTimeline(f,count=1)
 for tweet in tweets:
  sql = 'UPDATE content SET CONTENT = "%s" WHERE id = %s' % (tweet.GetText(),feeds[f])
  cursor = conn.cursor ()
  try:
   cursor.execute(sql)
   conn.commit()
  except:
   conn.rollback()
  cursor.close ()
conn.close ()

yes its hacky, but hey, called from cron regularly it works :-)

They're following me!

As you know, I've been 'tweeting' the Status of the LHC from the 'Page1 comments' field. Decided to knock up a quick script to plot the number of followers over time, and lo:

Design vs Reality

Decided to check that savannah was behaving as it should be. As with any good process we have a nice flowchart that depicts the allowed transition states.

ie:

however, thanks to a quick 'copy n paste' into graphviz it appears the reality is a little more complicated. Ho hum, cleanup time....




update: Just tidied up savannah and lo:

transmission

Snippet to see how many chunks of a bittorrent file are still to be downloaded (ie, what does '84% complete *really* mean):

transmission-remote -t 2 -i | egrep "^ (0|1)"| grep -o 0 | wc -l

where -t 2 is the number of the torrent.