Radio Scrobbling via MQTT

Since I often listen to the most excellent Soma FM during the day (go support them, it's worth it!) I often wish that I could go and stash some similar tunes to listen to when I'm not online. To do this, I sell my soul to CBS and use last.fm to generate similar / listened to by others recommendations.

Sadly Rhythmnbox doesn't scrobble radio stations (even though the scrobble API has a flag for this
track.scrobble(chosenByUser=0) to indicate that its been chosen by a radio service), despite it updating my 'now listening' status on IM. So, in a geeky overkill solution I decided to save time in the long run and use a pub/sub system with MQTT. Oh, and use some python and twitter along the way...

The design is that I have an MQTT publisher that uses the tritter streaming api (via tweepy) and filters for all the somafm twitter streams that Rusty pushes with BigUrl. This gives me near realtime announcements of the start of the track being played (after some trivial split / strip manipulation)

ie

justcovers       Pat Boone - Enter Sandman  (2012-06-25 07:20:30)
digitalis        Zephyrs - Stand Round Hold Hands (Ulrich  (2012-06-25 07:20:37)
secretagentsoma  Eat Static - Love Truncheon  (2012-06-25 07:21:02)
beatblender      Kerry Chandler (Lazy Dog Ben W - Isis  (2012-06-25 07:21:29)
sonicuniverse    In The Country - Only The Birds Can See Us  (2012-06-25 07:21:29)

..which I can then publish to an internal MQTT broker using mosquitto python bindings.

In a separate process I have a subscriber that I can point to the broker and 'subscribe' to the radio station when I'm listening and it scrobbles that information to last.fm. 

All in, an interesting sunday evening work. 

apcupsd via Python to Pachube

In python, the 'default' URL accessing toolkit (urllib2) doesn't support PUT, however I've discovered requests which a) does and b) is in Fedora. So one apt-get install python-requests.noarch and you're off.

So - one trivial script (reusing much of the code for my MQTT variant) you get https://gist.github.com/1903259. The only minor niggle was working out how to prepend the minimum extra json content (version and datastreams) as I'd not used the json libs before. (Hmm yet another github toy - embeddable gists with syntax highlighting. Nice) Update now I have 24h worth of data - plot attached

QSL Cards

At the shack there are thousands of QSL cards stuffed into boxes. As an experiment I decided to see if there was a quick way of scanning these - so using a webcam to capture a still image, followed by a python script to stitch the 2 adjacent images together (as the front and back will be on 2 separate images) I managed 400 or so in a session.

stuffed up on flickr for now till I get round to finding a better place for em.

oh the script? it's my 1st adventure into the Python Imaging Library -- and its rather nice.

#!/usr/bin/python

# Script to generate a single image of a QSL card from 2 adjacent
# scanned images
#
# Andrew Elwell for F6KAR
# This script may be copied or modified under the terms of the GPL
#
# We assume that the cards are arranged with the following seq:
# [- 1F] [1R 2F] [2R 3F] .... where FR are Front and Rear of cards

from PIL import Image
import os

indir = "/home/aelwell/Downloads/qsl/batch2"
outdir = "out/"

#leftbox = (100,100,900,1200)
leftbox = (0,0,750,1000)
rightbox = (850,0,1600,1000)


#todo = os.listdir(indir)
#todo = list(todo)
#todo.sort()
#for i in todo:

for n in range(1,403):
i = "Image-" + str(n) + ".jpg"
image = Image.open(indir + "/" +i)
print i

merge = Image.new('RGB',(1000,1500),)

try:
merge.paste(front,(0,0,1000,750))
except:
print "should skip on 1st iteration"

rear = image.crop(leftbox)
rear = rear.rotate(90)

merge.paste(rear,(0,750,1000,1500))

front = image.crop(rightbox)
front = front.rotate(270)

merge.save(outdir + i,"JPEG")

transmission + munin

OK, so things like the Rocks Avalanche Installer (pdf) show how useful bittorrent can be for providing a scalable install service. How do you know the status of bittorrent installs though? transmission-remote -l is all well and good if you're on the console but what about long term graphing? Cue Munin and a quick bit of python:

#!/usr/bin/python
# Munin plugin for monitorring transmission bittorrent status
# Andrew Elwell 
# $Id: $
import sys, os

if len(sys.argv) == 2 and sys.argv[1] == "autoconf":
        print "yes"
elif len(sys.argv) == 2 and sys.argv[1] == "config":
        print 'graph_args --base 1000'
        print 'graph_title Transmission torrent status'
        print 'graph_vlabel Count'
        print 'graph_category Transmission'

        print 'Stopped.label Stopped'
        print 'Stopped.draw AREA'
        print 'Idle.label Idle'
        print 'Idle.draw STACK'
        print 'Seeding.label Seeding'
        print 'Seeding.draw STACK'
        print 'Up.label Up and Down'
        print 'Up.draw STACK'
        print 'Downloading.label Downloading'
        print 'Downloading.draw STACK'
        print 'Will.label Will Verify'
        print 'Will.draw STACK'
        print 'Verifying.label Verifying'
        print 'Verifying.draw STACK'

else:
 # OK we cheat - initialise ALL values to 0, then overwrite with 'real' ones, then spit out the whole dictionary.
 data = {'Stopped': 0, 'Seeding': 0, 'Idle': 0, 'Up': 0, 'Downloading':0, 'Will':0,'Verifying':0 }
 for state in os.popen("transmission-remote -l |sed 's/^\s\+//g' | sed 's/\s\{2,\}/~/g' | cut -d '~' -f8 | sort | uniq -c | grep -v '1 $' | grep -v '1 Status' | awk '{print $2, $1}'"):
  (s,c) = state.split()
  data[s] = c
 for k,v in data.iteritems():
  print k + '.value' ,v

EDIT: Suppose I should really find a nice template that copes with embedding code, but in the meantime thats what 'view source' is for...

concerto + twitter

During the trials of concerto at CERN, I wanted to make the text fields a bit more dynamic like say following a specific twitter feed.

This would (ideally) simply consist of a DB entry with type=twitter, value=name of person. In the Real World however I got round this by hacking up a simple python script that connects to the DB and does a simple UPDATE of an existing text/plain entry via cron.

posted below incase others find it useful:

#!/usr/bin/python

# Script to pull twitter feeds for cern / lhcstatus
import twitter
import MySQLdb


conn = MySQLdb.connect (host = "...", user = "...", passwd = "...", db = "...")
api = twitter.Api()
feeds = { 'cern': 25, 'lhcstatus': 28 }
for f in feeds:
 tweets = api.GetUserTimeline(f,count=1)
 for tweet in tweets:
  sql = 'UPDATE content SET CONTENT = "%s" WHERE id = %s' % (tweet.GetText(),feeds[f])
  cursor = conn.cursor ()
  try:
   cursor.execute(sql)
   conn.commit()
  except:
   conn.rollback()
  cursor.close ()
conn.close ()

yes its hacky, but hey, called from cron regularly it works :-)

BeautifulSoup + savannah

At work, we use savanne to power http://savannah.cern.ch where we keep track of all the glite-middleware (amongst other things)

However, one of the problems with savanne/savannah is that it doesn't have a nice machine readable API, but requires the user to click with a browser.

Cue Beautiful Soup and lo, the python SavannahCLI was born.

Well, in its early years it was unloved and handed round from developer to developer, until it arrived as a pedantic teenager to yours truly who decided to get rid of the horrible 'operation, item, name, value" syntax and replace with something more, um, fluid instead. (--patch 1234 --set foo=bar). It also had a huge set of hard coded lookup tables for converting select items to names and vice versa -- WHY?

thanks to some googling, I came across this which, with a little refinement gave me:

def enum_forms():
  # we search within the table tag so that we lose the LHS searchbox
  selects = soup.find('table').findAll('select')
  for sel in selects:
     sel_name = sel['name']
     print "SELECT NAME=%s" % sel_name 
     for o in sel.findAll('option'):
         print "  %s, %s" % (o['value'],o.renderContents())

hey, it's not finished yet, but sure saves a whole pile of hard-coded badness.