Debian as the research library of Free Software

I’ve had a quite interesting discussion with Lars Risan in the comments to my recent blog post on Launchpad about Lars’ paper on the role of technical infrastructure in Free Software development, and I think Lars does well to describe the central tension within these:

I think Debian is a tremendously interesting case when it comes to understand human cooperation. Because, not unlike what Eric Raymond pointed out long ago, there is the â??Bazaarâ? of Debian, the heterogeneity of the network. And there is the â??Cathedralâ? of it: The ability to structure the large amount of work to the degree that you can slip a Debian DVD into a Windows-computer, and turn it into a Debian computer in less than an hour. How far can you take this mix of â??cathedralâ? and â??bazaarâ?? How much bazaar can you have before it forks, and how much central control can or must you enforce upon the network? How can you build a system that enables the â??beauty of (hacking) mankindâ? to simply do good, and/or how much must you invoke something like the (partly fantasy figures) of the Ubuntu-Launchpad and the Debian-Cabal?

This much-discussed question of centralization connects well with the stated goal of World Domination that many Free Software communities state again and again in a typical “ha-ha, only serious” fashion.

Global domination is a centralistic quest to compete with the cathedrals of Windows and Mac OS, and to do this while maintaining the freedom of learning and technical exploration that is the essence of Free Software will inevitably be a balance act. It is one that all Linux distributions are performing with various degrees of success.

One way is, as Canonical proposes, to let companies define the supported Free Software that the end user might need, and guarantee that it is available and that it “just works”. Thus creating small pools of centralization – the specific traits of Ubuntu – merging with the larger decentralized pool of shared knowledge within the bazaar – Debian Unstable every six months.

Mark Shuttleworth describes this relationship as Debian as “the Tibetan Plateau of the free software landscapeâ? upon which Ubuntu is built:

By contrast with Debianâ??s Plateau, Ubuntu is a cluster of peaks. By narrowing the focus and allowing the KDE, Gnome and server communities to leverage the base of Debian without treading on one anotherâ??s toes, we can create a K2, and a Kangchenjunga and a Lhotse. Ubuntuâ??s peaks depend on the plateau for their initial start, and their strong base. Ubuntu needs to be humble about its achievements, because much of its elevation comes from Debian.
[…]
Many people have asked why I decided to build Ubuntu alongside, or on top of, Debian, rather than trying to get Debian to turn into a peak in its own right. The reason is simple – I believe that Debianâ??s breadth is too precious to compromise just because one person with resources cares a lot about a few specific use cases. We should not narrow the scope of Debian.

At last year’s DebConf, a talk compared Ubuntu’s use of Debian to shopping at a supermarket, a one-stop shopping for your free software needs. This innocent analogy sparked a lot of discussion, with many of the Debianistas arguing that this reduced Debian to a giant collection of un-integrated software – which wouldn’t be very interesting at all. As Debian Developer Joey Hess put it:

My main motive for contributing to Debian is to make Debian the best distro I can; I don’t mind if others use that work, especially if stuff gets contributed back. But it’s long been clear to me that the most important added value to Debian is not adding another package to the shelf, but finding new ways to integrate our software together. When you’re working mostly above the level of individual software packages, to have your work mostly appreciated on the basis of “component contained in Ubuntu” is not very motivating.

Integrating and creating a universal distribution is still a central motivation for many Debian developers, who may find that Ubuntu is not only stealing their thunder but also compromising their ideals, again with no small amount of distrust towards the dealings of the opaque corporate entity that is Canonical.

But as Shuttleworth also notes, that when you seek narrow goals, it will come at the cost of openness and democracy. The Debian community is obviously unwilling to make such a trade, and as such, must look elsewhere for inspiration on how to organize themselves towards their goal.

Taking a cue from an earlier comparison of the Free Software communities to the scientific communities, I think it would be worthwhile for Debian to leverage their solid technical infrastructure towards becoming commonly agreed upon open standards to allow the exchange of knowledge and code will be possible on equal terms. In that way, I think that Debian is quite well poised to become a sort of â??research library of Free Softwareâ? – collecting the â??monographsâ? and â??articlesâ? of code, cataloguing and organizing them for easy and open access.

Almost all Debian developers, and apparently 76 % of all Debian users,use Debian Unstable or Testing.

Debian Unstable, codenamed Sid, is the latest, most volatile version of Debian, where new packages are uploaded to, and changes to old packages are added on a daily basis. It contains all of the latest releases of the upstream communities, packaged and categorised. Debian Testing contains all of the packages that haven’t had a critical bug filed against them after spending 10 days in Unstable.

Only 24% of the Debian users use Debian Stable, since the releases are so rare as to only come out every 2 or 3 years – much too rarely to keep up with all the latest hardware and desktop applications, making Debian Stable relatively unattractive in the long term.

Since Debian Unstable (or Testing, for the less adventurous) fulfills the needs for both the users and the Debian developers, there is little incentive to make an Upstream Version Freeze and actually get a new release out, and since Debian is solely based on volunteer work (with suggestions to pay release managers meeting fierce opposition), bug fixing and release deadlines aren’t enforced that well, further slowing the actually release of the Debian Stable.

As Shuttleworth points out, Debian Unstable is “not subject to the same political tradeoffs that are inevitable when dealing with releases, architectures, dates, deliverables, translation, documentation and so on.” – it is the core strength of Debian.

As the developers I interviewed said again and again when asked what they get out of contributing to Free Software projects like Debian: “I get a system that works for me.” To some extent I think that is what Joey Hess’ means when he says that his main motivation for contributing is “to make Debian the best distro I can.”

Debian Unstable is not merely about keeping software up to date, but also about improving it, further integrating it within the Debian infrastructure. It is a research library in the way it is not only focused on gathering knowledge, but also on using it. It is also a laboratory where Free Software is shared and combined in new and interesting ways.

Maybe that would the most fitting analogy to what Debian really is: A research library containing all of the Free Software that lives up to set strict criteria. Offering all the development tools, libraries and applications up-to-date for new experimentation.

This would fit well with the Lars’ counter-argument:

Science is (also) extremely disunified (there are 22.000 different medical journals in the world. â??Medicineâ? (in singular) is known by no-one). And â??scienceâ? can live with this relative disunity. Can Debian? There is at least one difference: â??Debianâ? as a whole must occasionally come together to produce a release. A release which is unified enough for a computer to work. Fuzziness is simulated by computers, because they work only by strictly separated ones and zeroes. The knowledge of Debian, packed together as a release is pretty much a unity. A unity of which a large degree of coherence is required. The knowledge of science may be unified in an encyclopaedia, but, metaphorically, the discrepancies and glitches of that body of knowledge is so enormous, so â??buggyâ?, that it would make any computer halt very early in the boot.

Much like the knowledge of science only can gathered for release in an encyclopaedia that is outdated before it is even published, all of the software within Debian will also be outdated by the time it is released. Perhaps Debian should take inspiration from Wikipedia rather than other Linux distributions.

Wikipedia is a reservoir of such diverse knowledge that it would never be in a position where it could be published in paper form. It would require an insane amount of work, flamewars and unhappiness for all involved. It would require a centralization that simply is not present in the Wikipedia community.

But in spite of this, by far the biggest part of Wikipedia is eminently usable and accessible at any given time. As is Debian Unstable.

Perhaps the Debian community should take some comfort in this.

7 comments

  1. tosky

    Almost all the debianistas in my GLUG use testing for their personal/development systems. I know that it’s not statistically relevant, but…

  2. Andreas

    @anonim & tosky,

    thanks for pointing this out. I didn’t actually check the statistics that thoroughly, and noticed that the 76% included Debian Testing – which may now receive a fair bit of use since it is getting security updates as well.

    I’ve now changed the post to reflect this.

  3. effie_jayx

    Nobody could have laid it out better… Gotta say.. It was a well documented read and it can really raise some awarness.. and more openness… I am an Ubuntuntero and share with lot’s of debianitas…

  4. Andreas

    As Kevin points out, there was a similar discussion on the debian-devel mailing list following Mark Shuttleworth’s blog post in September.

    Firstly, most replies cast doubt about the completely unscientific statistics that Shuttleworth’s comments were based on. They argued that most of the servers running Debian Stable weren’t represented in the survey due to security reasons.

    Following that, there were several arguments against giving up work on Debian Stable releases.

    The most recurring one was that Debian Testing would never be good enough for servers:

    For desktops I kind of agree with you. On a desktop I always want to have the “new and fancy” and I run unstable on my desktops (the other reason is to help find bugs). On the other hand I also maintain a bunch of servers and I would not want to run testing on them. I really like stable for servers, as I have an extremely solid system which doesn’t give me much work, except installing security upgrades.

    Others argued that the long lifespan of Debian releases make them much more reliable than Ubuntu. Taking a good deal of pride in the quality and stability of the stable releases, they did not like the insinuation that :

    Most Ubuntu stable releases are only supported for a relatively short time, which is not long enough for me; and most Ubuntu packages are in “universe” where (AFAIK) they only get support in a stable release on a “best-effort” basis by a small and overworked MOTU team.

    Others looked upon Shuttleworth’s suggestions with downright hostility:

    Mark Shuttleworth wants Ubuntu to replace Debian on all market segments, turning Debian into a provider for unstable packages. But we have no reason to help him.

    While others sought a more pragmatic solution, effectively dodging the bullet:

    If you’re interested in Debian supporting unstable and testing and not stable, then there are several things you can do to help bring those distributions, especially testing, up to par as a really usable distribution for a larger percentage of users […]

    Since all of these activities also have side effects that make stable releases better and/or more frequent, and most of the work done targeted at making stable releases better makes testing better in between, I’ve not worried much about trying to realign the project’s focus away from stable.

    But most remarkably, one commenter simply remarked:

    Then people would need to accept that Debian cannot be the solution to every problem, and this would be politically bad.

    This is quite an acute observation, since as I was reading this, I had begun to consider whether it would be possible to turn Debian Stable into a server release, thus focusing the release process to only the core packages.

    But in a way, this is what Ubuntu has done by enforcing a centralized reorganization of their copy of the Debian software repositories into a small supported ‘Main’ repository and a huge unsupported ‘Universe’ repository.

    Could the Debian community make a similar decision about which packages are core server packages and which ones aren’t?

    Most likely, it would be an impossible compromise – similar to if the wikipedians were to make an abridged version of their work available in paper form. It would require a degree of centralization that wouldn’t be politically feasible across such a diverse community.

Post a comment

You may use the following HTML:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>