Monthly Archives: July 2006

Karma in Launchpad

For some time, I have been curious about the karma system in Launchpad – the massive infrastructure being erected not only for Ubuntu, but for the F/OSS community in general.

The karma system gives “karma points” for actions which the system appreciates, like bug reporting, specification writing, bug fixing and so on. But so far, it does little with this information apart from showing who the top 5 contributors in all of Launchpad are (lower left hand corner), and show how the karma is divided up between various tasks.

Now, with so much else around Ubuntu and Launchpad, it didn’t appear out of nowhere. The karma system is obviously inspired by systems in other F/OSS communities. And I thought it would be interesting to examine how these compare to Launchpad. The two that I’ve noted (though there are undoubtedly more) is Slashdot and the GNOME Bugzilla.

In both Slashdot and GNOME, karma or bugzilla points play an important social function. It is a value of trust.

Slashdot karma is gained by writing insightful or relevant comments or posting interesting new stories which are moderated as such by the other readers. In turn, you can use (expend) this karma to moderate other posts as insightful or not, thus strengthening the system. If you’ve been modded insightful with +5 modifier, chances are that you will also apply that good judgement when moderating other posts, so the system gives you 5 points to mod posts where you see fit. Thus the system trusts you the more you contribute. On Slashdot, karma doesn’t have an easily accessible point value but rather vague textual identifiers such as “Terrible”, “Neutral” and “Excellent” – this is to avoid some people gathering huge amounts of karma and thus setting themselves above moderation. The Slashdot FAQ has this gentle reminder on the topic:

Karma is used to remove risky users from the moderator pool, and to assign a bonus point to users who have contributed positively to Slashdot in the past. It is not your IQ, dick length/cup size, value as a human being, or a score in a video game. It does not determine your worth as a Slashdot reader. It does not cure cancer or grant you a seat on the secret spaceship that will be traveling to Mars when the Krulls return to destroy the planet in 2012. Karma fluctuates dramatically as users post, moderate, and meta-moderate. Don’t let it bother you. It’s just a number in the database.

The GNOME Bugzilla
The GNOME Bugzilla on other hand, works somewhat differently. The stated goal of the point system is to make make each contributor’s level of experience with Bugzilla and GNOME clear in an easily accessible manner. On a typical bug report, you will see the reporter’s point score prominently displayed, and when people comment, their point score will be displayed as well.

At the GUADEC, I talked to Olav, the GNOME bugmaster and maintainer of the GNOME Bugzilla, about how the system works and what they use it for. I asked whether the point system wouldn’t encourage long-time contributors to ignore bug reports from newbies, but he explained, that quite to the contrary, these long-time contributors could spend more time on newbie bugs, because they could rely on the bug information from other people with high point-scores and thus didn’t have to triage these as thorughly. It also helps new users to see which comments are from experienced people whose advice they would do well to heed.

The Bugzilla point score is calculated from the following formula:

log_10(1 + #comments) + log_2(1 + #bugs_closed) + log_2(1 + #bugs_reported)

The point scale is logarithmic with different bases for comments (log10) and bugs closed and bugs reported (log2). That means that you need to make 10 comments to get a point, but only close 2 bugs to get another. But for the next point after that, you’ll need to reach 100 comments, or close 4 bugs. Then 8, 16, 32 and so on. This means that it gets progressively harder to gain more points as you go along.

This means that going from 0 to 1 is very easy, but already at 5 points you can be considered a proficient bug hunter. Reaching the mid-20s range like some of the core developers will require a much greater effort. So far, only one bug hunter has reached 27 points. A list of how the points are distributed among the registered Bugzilla users is quite telling.

This logarithmic increase in points almost entirely cancels out the “computer game” tendency addressed on Slashdot, but the points themselves do not easily indicate a level of expertise to newbies. 27 points doesn’t sound like much compared to Launchpad’s several hundreds of thousands of points, but when you look at the statistics behind it, it is somewhat more impressive:

Since all bug management actions are counted towards the same score, you can access the bug management information, and can thus directly see what you have done to make you point score increase by looking at the list of bugs reported/closed/commented on the contributor page.

The GNOME Bugzilla aptly uses these statistics to generate weekly reports which give an idea of the amount of work being done within the last week. There are also annual statistics list which is announced and examined with much interest among the GNOME bugsquad.

So in this way, there a two distinct set of “point values”. There’s an overall contribution to GNOME value, and there is a contribution within a set period of time (last 7 days, last calendar year). This makes it possible to represent and acknowledge the experience and contributions from each individual both in total and over a shorter period of time.

In order to solve the problem of the point score giving a level of trust that is not easily interpretable for new users, the GNOME developers have introduced that developers can register the packages they maintain so that this information will also appear on the bug report, like this:

There has also been a discussion about turning the point values into some easy-to-understand text alternative. I would imagine that they could well end up with something silly and similar to the titles of the individual class levels in Dungeons & Dragons (“congratulations, you have reached level 23 – you are now a Grandmaster of Flowers!”). One GNOME developer has even compared developing GNOME to playing online role playing games!


Since Launchpad is much more than just a news forum or a bug tracker, it’s Karma system needs to span over several different elements that are not easily alignable. Launchpad consists of:

Malone – the bugtracker
Soyuz – the distribution management software
Blueprint – the Specification Tracker
Rosetta – the translation software
Bazaar – the version control system
– the so-far unnamed support tracker

Each of these distinctive parts have different uses and thus offer different ways of gaining karma. Different actions add different points to your karma total. For instance, reporting a new bug gives you 10 points, while providing an (unaccepted answer) on a support request gives you 1 point.

The developers admit that they’re just guessing when it comes to what right point values for each action should be:

The score associated with each action type is currently just a best guess. We have yet to do any serious analysis on the real data to determine which actions should be rewarded more and which actions rewarded less.

The way the system is built makes it possible to award karma points for actions which aren’t related to any of the above parts of Launchpad, such as registering an email address or a GPG key. This may further dilute the relevant data from the different parts of Launchpad.

All of a user’s karma points from all over Launchpad are then calculated with a degradation algorithm, where contributions today are worth 100%, contributions 182 days ago are worth 50% and contributions made 365 are worth 0% – which means that if you don’t contribute for a year, you karma will drop to zero. The developers provide the following rationale:

We feel it important that Karma is reduced by time, as otherwise new users may feel that they have no hope of catching up with a long time user, even if that long time user is no longer active.

This means that the Launchpad karma cannot be used to measure longtime experience or social trust within Launchpad, as trust or ability does not degrade like this. Instead, Launchpad karma is solely a measure of activity which is recalculated on a daily basis. This means that the karma values can fluctuate wildly from day to day – especially depending on how certain tasks are rated. For a long period of time, the Rosetta developers had way too much karma compared to their work because Rosetta gave out karma points more easily than eg. Malone.

The use of karma in Launchpad has been discussed on the Launchpad-users mailing-list, and some bug reports have been filed on it to base the karma on logarithmic factors like the GNOME Bugzilla and displaying the karma next to the contributor’s name on a bug report. Also, there has been bug reports asking for an easy way to use the list of karma gained as a log through which you can access the comments and bug reports that you wrote to get the karma in the first place.

And yet another bug report has been filed in the hope that more statistics on karma distribution will be available than the current top 5.

Being part of a immensely complicated (and unfinished) system, the Launchpad karma system has a lot unresolved issues. First of all, it doesn’t even seem like the Karma system has a solid purpose yet. From the specs it seems like it was mostly to log user activity. But since the user can’t use the log to backtrack to the actions they performed, it is not that helpful in that regard. In the Malone, there is talk of using Karma as an incentive for people do good work. But if they have trouble figuring out how the karma is counted, and if the karma drops back to zero after a year, then that is not a very good incentive.

The central focus of the Karma system should not simply be activity, but trust. Launchpad will used by a lot of people, and their interactions are based on very few factors. Having social guide line such as Karma can help a lot. Both Slashdot and the GNOME Bugzilla provide examples of how that can be done successfully.

I’m afraid that the Launchpad developers have been too caught up in the technicalities to think of the actual use cases of the Karma system. I hope this will help a bit (I have also sent a link to this text to the Launchpad-users mailing-list).

Long live afrobeat!

Last night I took some time off from the rather intense note-taking and -organizing I’ve been doing and went out to have a look about on at the ongoing Copenhagen Jazz Festival. Specifically, I went to a Tony Allen concert. Allen is a legendary Nigerian drummer, co-originator of the entire afrobeat genre, and described by Brian Eno as “perhaps the greatest drummer who has ever lived.”

Last year I had been to a great Femi Kuti concert at the Roskilde Festival, and now I had a chance to listen to Femi’s father Fela Kuti’s main collaborator live, playing a wonderfully jazzy set in Copenhagen.

Tony Allen is an old man now, at 66. But he plays so very tightly and effortlessly it is a joy to hear. The whole band plays around the drum beat to create the most organic sound I’ve heard. It is difficult to distinguish the individual instruments (apart from the horns obviously) from that swinging whole which sounds seems like one big pulsating instrument. Then, when one of the musicians play a solo, you realise how that one part of the rhythm has been supporting the rest in the same way that they are supporting its creative experimentation now.

I got a mental image of fireworks, slowly onfolding, reenveloping upon itself and swaying softly against the night sky. Or perhaps like the flowers of a coral reef opening and closing but swaying in unison.

And at the centre of it all, behind his drumkit, would be this little man, sitting there, smiling at it all. And once the song ended, he would simply say, “Yeah”. Not smugly, but knowingly. His meltingly rusty warm voice displaying an affection for the music that said it all.

Finishing up the second set, Allen looked out at the audience and said “We’ finishing now, can’t play all night.”

The crowd applauded in attempt to make him play more.

He replied “I ain’t young no mo’, y’know! What do you want me to do? Live endlessly?”

The crowd caught it and shouted “Endlessly! Endlessly”

He smiled sagely back at them. “Heh, endlessly. One more song for tonight…”

But in the end, they ended up coming back out for a two-song encore. With this kind of music, experiencing it live is so much more enriching than merely listening to the music on your home stereo, but if you’re curious you can find both Tony Allen’s music and a lot of other afrobeat music around the Intarweb.

Mail, Mail and more Mail

Since I came from Læsø [note the Wikipedia entry name in the address bar – not easy to recognize] late Tuesday evening, I have been catching up on my mail. And lots of it.

Doing fieldwork in an on-line community means that you’re reading lots and lots and lots of mail. From the various Ubuntu mailing-lists I get an easy 200 emails a day, So when I got back, after a holiday trip plus 2 weeks of intensive fieldwork, I had a couple of thousands of mails to go through for exciting ethnographic content.

Sorting my mail, I find there are the following main categories:

Bug mail – updates on various bugs that I subscribe or have been subscribed to, through the Malone bugtracking system.

Spec mail – updates on the various specs that I subscribe to through the Blueprint Specification tracking system.

Change mail – Every time a new version of software package is uploaded in the new Edgy Eft version of Ubuntu, a mail is sent to this list. A typical change mail.(I hope to examine this further as I find this to be a prime example of communication directly through technological production rather than actual conventional social interaction).

Ubuntu Mailing lists – This is the bulk of the mail I get. These are divided into Ubuntu-Devel for Ubuntu development discussion, Ubuntu-desktop for Desktop and Usability discussions (very little happens there…), Ubuntu-Doc for Documentation discussions, the Sounder (apparently named after a pack warthogs, because the first Ubuntu mailing-list was called Warthogs) for random community chit-chat and lots of other, specialized ones that I pay less attention to.

Obviously, I find the Sounder most readable, but it often degenerates into long, awkward discussions about very little. It seems that discussions are either specifically on the technical issues or it can explode in any other direction at any whim. As Biella Coleman observed somewhere, geeks often self-reflexively comment upon this, as in this email signature:

Arguing with an engineer is like wrestling with a pig in mud.
After a while, you realise the pig is enjoying it.

On top of all that, there’s my usual mail, and then there are the Blogs. Following GUADEC, I’ve begun to read not only the Planet Ubuntu feed (which I’ve been reading for a long time now) but also the the Planet GNOME and Planet Debian feeds in order to get an idea of what the upstream is up to. I did this when I found out how central the Planet GNOME is to that community’s communication.

Basically it is just one big blog combining blog posts from all the different individuals who have had their blog signed up, but it gives a much better feeling of community than the mailing-lists. Maybe it’s because of the little heads next to each blog post. These are called Hackergotchis by the way.

So, I find myself spending a good part of my day reading and writing notes just like if I was doing a literature project. The difference is that here I can ping the authors on IRC at ask them what they mean, exactly.

And the Survey Results are in..

For three weeks through May and June I ran a Web Survey directed at the active members of the Ubuntu community. I got more than 290 valid responses, and I’ve set up a page to share the results with the Ubuntu community.

Not having done any major statistical surveys before, this was quite a change of pace for me – and I’ve certainly learned a lot about the finer points of quantitative analysis. Among other things, how to phrase your questions to get the kind of answers you’re looking for.

I started out with lots of comment boxes, soft interpretation of questions and giving the respondees the opportunity to write in the margin, make small comments and generally give feedback at every step. It was a good way to find out what worked in the survey, but it also made it very difficult to process the bits that didn’t work as well. As the quote goes:

??There are three kinds of lies: lies, damned lies, and statistics?

And once you start “massaging” data to get nice pie diagrams and meaningful percentages, it is very easy to just modify the data as well. Naturally, I didn’t do so, I just noted how easy it is to do – even unknowingly. Quantitative data seems to be just as liable to suffer from over- interpretation as qualitative data.

Now it’s out there, and hopefully it’ll be of some use. In any case I’ll be able to use for my thesis.

Now I’m off to Læsø for a few days, and will be completely off the grid until Sunday – something that has become increasingly difficult to do as I get more involved in this brave new on-line world.

Back from the field

I arrived back in Copenhagen late last night on delayed flight from Barcelona and the 2006 version of the GUADEC conference. Actually, the conference took place in Vilanova i la Geltrú, 45 km south of Barcelona.

The GUADEC (GNOME Users and Developers European Conference) and GNOME, for the uninitiated is the acronym for “GNU Network Object Model Environment” the full name of which people rarely use anymore. Actually, GNOME is a Free Software Desktop Environment – which means that it is a wide collection of different elements that seek to make the user experience of the computer as smooth as possible. Just like the Linux kernel or the X-Window system, GNOME is an central part of many Linux distributions such as Red Hat, SuSe and, of course, Ubuntu.

It is worth noting that GNOME is not the only Linux desktop environment available. There is also KDE (for the K Desktop Environment – very creative name, yes?) along with many others. For KDE users, there is a now a separate version of Ubuntu awkwardly named Kubuntu which is based on KDE rather than GNOME.

Anyway! Before this all but drowns in strange geeky acronymics (and to be sure, there’s a lot of that going on), I’ll tell you about the actual conference. I went there to get an impression of how one of the older Free Software projects (the GNOME project was begun in 1997) works socially and organizationally compared to Ubuntu.

The conference was a very different event compared to the Ubuntu Developers’ Summit. It is much more of a community-driven event, organized by volunteers and dedicated members of the GNOME community. Sure, there are lots of corporate sponsors and keynotes and workshop discussions, but everything seemed much more relaxed and unhurried. There were no results that had to be reached. No specifications that had to be written. It was just a good, social occasion for people to meet and have fun. It had a unmistakable feel of geek summercamp about it, with people going to the beach, sitting and hacking in good humour, hanging out. There was even a football tournament (“the FreeFA World Cup”), sponsored parties and live music with the GNOME band!

But a more typical image from the conference would be this:

I didn’t actually get to take any pictures at the GUADEC, but luckily almost everybody else did. It seems that almost every geek present had a digital camera, and quite a few had really big expensive ones as well. It seems that photography is popular hobby among geeks. At the time of writing, more than 1700 pictures on Flickr has been tagged GUADEC2006, and just by flicking through a few of them, you’ll get a good idea of the smiley, relaxed atmosphere there.
(also, the two pictures presented here are linked from Flickr).

At the same time, there was an intense amount of posting on the aggregated Planet GNOME blog feed that collects blog posts from lots and lots of GNOME developers. It is definitely another good way to get a feel of the atmosphere at the conference.

Compare to the Ubuntu Summit was called, arranged and sponsored solely by Canonical, the company employing the Ubuntu core developers, with an intense working schedule with workshops from 9 am to 6 pm every day and having to sneak in the social time after that and cut down on sleep instead. Actually, at GUADEC people also cut down on sleep, in a way that left most people absolutely exhausted at the end of the full week of conference.

Oh, and photos from the Ubuntu Summit has been gathered here. Including a very nice group photo. I unintentionally managed to end straight in the middle of the photo. A key to who the rest of the participants are can be found here.

Also, compare the playful banter of the planet GNOME with the more sober tone among the posters at the Planet Ubuntu blog feed. Also, note the small overlap of posts. I especially recommend the Ubuntu Quality Assurance guy (“bug hunter”) Simon Law’s blog posts which does well to capture the duality of the conference: Trying to both do hacking stuff and have time for social interactions and adventures in Paris.

Flying on the cheap

Having tried different budget airlines over the past few weeks, I’m appalled at how tacky they have become. On Ryanair flights, they will blare advertisements over the loudspeakers before take-off, before attempting to sell you all kinds of crap during the flight. On Sterling flights, they not only try to sell you the usual fare of tax-free crap, but also raffle tickets.

Both companies have a strict policy against passengers consuming their own foods and drinks on-board the plane. Instead, we should buy their over-priced foil-wrapped sandwiches and canned beverages.

Added to the usual tourist trap hooks that you stumble across all over both Paris, Barcelona and Vilanova it is easy to get more than usually tired of air travel. Sure it may be cheap, but they sure do their best to make your aware of the fact for just a little extra money, your travel experience could be so much more comfortable..