From the Canyon Edge -- :-Dustin

Monday, August 30, 2010

How My Work Benefits Free Software

I'm personally offended when Canonical and Ubuntu are dogged about a perceived lack of contribution to the Free Software ecosystem.

So I'd like to talk about the work I have been doing at Canonical on Ubuntu, and how I believe it benefits Free Software in 7 important ways:
  1. Packaging
  2. Documentation
  3. Communication
  4. Conferences
  5. Upstream Contributions
  6. Being an Upstream
  7. Bringing Free Software to the Masses

1. Packaging

Perhaps the highest volume, least glorious work I do is around a process called "packaging". It's not very sexy, but it is tremendously important. Packaging is what allows you to go to the Ubuntu Software Center, choose a program, click 'install' and then run that program. Packaging also allows command line (server) users to simply type "sudo apt-get install " to install a program. Millions of iPhone and Android users are familiar with this concept, with portals like the App Store and the Android Market. Packaging makes Linux accessible to millions of people who are not willing or able to run "./configure && make && make install", or chase down a complex, cascading set of library dependencies. We, the packagers, do that hard work, and ensure that program installation (and uninstallation!) is an simple, painless experience.

I've packaged many dozens of applications for Ubuntu (and some of those have been uploaded to Debian as well). In many cases, this is work that the developers of this software is unwilling or unable to do.

In practice...

About a year ago, I had solar panels installed on my roof. The power inverter attached to those panels actually has a USB input and some Windows-only software that can read statistics about my photo-voltaic output. I first checked the Ubuntu archive for a package that could communicate with my inverter, to no avail. Searching the web, I found a guy named Curt Blank who wrote just such a GPL Free Software program called 'Aurora'. After a few dozen emails back and forth, and sending patches his way, we finally got it working like a charm against my hardware. I wanted to make sure the next Ubuntu user to buy an Aurora inverter could simply "sudo apt-get install aurora", so I packaged Curt's project and uploaded it to Ubuntu. It's available now, as of 10.04 (Lucid). Curt was pleasantly surprised and appreciative of having his software included in a distribution, and available to all Ubuntu users!

2. Documentation

No one should *ever* play down the importance of documentation! Shrink-wrap software always comes with a manual. Many Windows/Mac users buying shrink wrap software consult that manual when something goes awry. Free Software is delivered almost exclusively electronically -- but this is no excuse for omitting a manual! Documentation for Free Software comes in many forms. Web delivered content, such as Wiki's are quite popular. The http://help.ubuntu.com/community documentation wiki is *outstanding*!

As an Ubuntu Community Member, I am a frequent contributor to that documentation, particularly in the UEC and Virtualization sections. This information helps many first time Free Software users clear their first hurdle, and gives them confidence that there is a tremendous community of fellow users who have solve many of the problems they encounter. Ubuntu Community documentation is some of the best around. Google for almost any given Linux help topic (printing, dvd, kvm, etc), and the word "Ubuntu", and you are sure to find some pertinent, accurate, informative documentation.
In practice...

When I first joined the Ubuntu Server Team in 2008, I found myself needing to read hundreds of manpages (text manuals for the command line). In some cases, I did not have the needed manpages on my local system. In other cases, I needed a generic way of search for multiple terms in all manpages. And in general, I prefer reading documentation in a web browser rather than a terminal. Thus, I created http://manpages.ubuntu.com/, which is a web application updated nightly, generating HTML versions of all manpages in the Ubuntu distribution. This site contains hundreds of thousands of pages, all indexed by Google (and thus searchable), marked up using HTML for readability, cross-linked for easy traversing through manpage-to-manpage references, and able to render a printer-friendly PDF of any given page. I continue to maintain the source code that generates manpages.ubuntu.com (which is also Free Software), and the hosting of the content is courtesy of Canonical.

3. Communication

On a daily basis, there are several hundred of Ubuntu developers and community members distributed across hundreds of IRC channels, mailing lists, and web forums. These are open communication channels whereby we develop Free Software and support its millions of users. We respond to messages in real-time, helping individual users with their issues. All of these media are archived and published, and the Internet has a very long memory. Quite often the most important service performed is not the assistance of the single user asking the question. Rather, it's the scores of others who search the web for that same question weeks, months, or years later, find the archive, and solve their own problem without needing to register for or learn the nuances of IRC/lists/forums.

Blogs and Launchpad bug reports can serve these same purposes. Any time a person contacts me privately to discuss a bug or a problem, I always kindly request that we move the conversation to a public forum (usually a launchpad.net bug report) so that others might learn from our discussion. For frequently-asked-questions, I tend to use my blog, so that I can update the response at a later date (should things change), and always refer future questions to the same URL.
In practice...

I developed and maintain a feature in eCryptfs which allows users to encrypt their home directories. Recovery of such encrypted data can be slightly complicated, if you know nothing about data encryption. Many new users have struggled with this, and many have been saved by: http://blog.dustinkirkland.com/2009/03/mounting-your-encrypted-home-from.html. The latest comment, AUGUST 23, 2010 8:46 PM: "scratchr said...THANK YOU THIS SAVES MY BUTT!!!!!!". This is but one of a couple hundred posts that I have syndicated to planet.ubuntu.com in about 2 years.

4. Conferences

Twice-a-year we hold an open developer summit, where we brainstorm our next release. These sessions are broadcast over the Internet, and anyone in the world is welcome to attend, and have their opinions heard. The open communications and friendly nature of our developer summit encourages collaboration and participation across many facets of the Free Software sphere.

External to Ubuntu, we also travel to countless tradeshows, summits, festivals, conferences, and gatherings of software technical and business people. Quite often, we are introducing people to the concept and practice of Free Software. Other times, we are collaborating with other Free Software developers, helping further the over all cause.
In practice...

Over the past 3 years, Canonical has sponsored (all or in part) my attendance to, and I have taken time out of my personal and work schedule to actively participate in multiple Ubuntu Developer summits, LinuxCons, Linux Plumbers, Linux Conf AU, CloudCamps, Texas Linux Fest, et al. Many of these were sponsored by Canonical, helping the conference pay for itself.

5. Upstream Contributions

Ubuntu is a "distribution" -- a collection of the best of open source software. You may think of Ubuntu as a grocery store. We seek out the best fruits and vegetables, and the best canned, bottled, and boxed products we can find, and make them available to you. In some cases, we do the canning/bottling/boxing (aka, packaging) for you. In other cases, we make available the products of our distributors (Debian, typically). You, the shopper at the Ubuntu grocery mart, have hopefully come to trust the quality of goods we provide, and trust your family and friends to use our packages too.

Sometimes the items we receive from our upstream have flaws that we can detect and fix. It is quite easy for us to fix those issues, and make the improved packages available to our users. It takes a bit more effort to send those fixes upstream, to the people we received the subject code from. And although it does take effort (tremendous effort, in some cases), it's a very important part of being a good citizen in our community. Communicating with upstream is a critically important part of my job, and one that I take very seriously, and in which I take much pride.
In practice...

Update-motd is a package that I initially created to allow for Ubuntu Server administrators to dynamically create the Message-of-the-Day, /etc/motd, by placing scripts in a specific directory, /etc/update-motd.d. These scripts would run at login (or at a specified interval), and command-line users would see the latest and greatest information at each login. This program stood alone for a while during its experimental phases, and I eventually ported it to C, generating a patch against PAM itself. Now that the functionality has stabilized, it is included in Ubuntu's upstream, Debian's, PAM package.

6. Being an Upstream

Returning to that grocery store analogy, sometimes the Ubuntu Supermarket lacks a particular "product" that either our customers ask for, or that we think our customers might want. In such cases, we may "become" an upstream and write some new software to serve our purposes and provide new features to Ubuntu users. Some of these projects are essential to improving the Free Software end user experience. A few that come to mind immediately: command-not-found, apport, and upstart.

Ubuntu as a whole, though, is an Upstream to various derivative distributions, such as Kubuntu, Xubuntu, Mint, and TurnKey Linux cloud appliances. We are furthering Free Software by producing a rock solid base that other people can use to build innovative derivatives that serve unique purposes and reach all sorts of interesting people.

"Being an upstream" maintainer is perhaps my favorite part of my job at Canonical, working on Ubuntu. We have an amazing suite of utilities (Launchpad, Bazaar, PPAs, and Ubuntu itself) for making the upstream experience simply outstanding. We have both a corporate and a community culture that strongly encourages the development of GPLv3 (and AGPL) Free Software. The software we develop and maintain as upstream is tightly integrated into Ubuntu, but it is freely available for inclusion in any other distribution as well.
In practice...

I have initiated several upstream projects, to fill gaps that we perceived in our Ubuntu Server distribution. These are all 100% Free Software, developed and maintained in Launchpad/Bazaar, included in Ubuntu, but sometimes included in other distributions as well. Byobu is an elegant configuration and usability layer on GNU Screen, which makes it quite functional as a text-based window manager. TestDrive is a handy utility that synchronizes daily ISOs (from Ubuntu or other distributions) and launches them in a virtual machine (KVM, VirtualBox, or Parallels) for your perusal -- without having to know anything about virtualization. PowerNap operates much like a desktop screen saver for command-line-only servers, bringing systems down to a lower power state when underutilized. eCryptfs is a cryptographic filesystem in the Linux kernel; I maintain the user space tools, which have been enhanced to support per-user encrypted home directories. This is a feature that was pioneered and funded by Canonical, and contributed directly to the upstream project. For fun, and in my spare time, I spend many hours working as an upstream developer of Musica, Pictor, Screenbin, BogoSec, Lynx-web-app, as well as Byobu, PowerNap, and TestDrive.

7. Bringing Free Software to the Masses

By far, the most important contribution I make to the Free Software world is doing my little part within Canonical and Ubuntu that makes Free Software available to the entire world. From Canonical's partnerships with OEMs such as Dell for desktops/laptops/netbooks, to our Cloud server images available in Amazon's EC2, we are making Free Software available to people who have never used Free Software before. Call us a "match maker" or "middle man" if you will, but we are undoubtedly helping connect the dots, and plug many new people into Free Software.
In practice...

Believe me, I had tried for nearly 10 years (1998-2007) to convert my wife, mom, dad, sisters, friends, and extended family to join me in replacing Microsoft Windows with Linux and Free Software. In 2005, I managed to convert my lovely, non-technical wife, Kim, to Fedora -- but I failed on every other account. I'm ever so proud to say that all of the above (wife, mom, dad, siblings, inlaws, and friends) are proud users of Linux and Free Software today, August 24, 2010. And Ubuntu is the reason why. Ubuntu is Linux for Human Beings. Their wireless just works. Their printing just works. They can play music and watch videos and browse the web. They can work on documents, spreadsheets, and presentations. When they need a new program, they can find it and install it. They can upgrade across releases all by themselves. They can plug in peripherals and video conference with their kids and grandkids. All of this was possible with Linux and Free Software in 2004, for an subject matter expert like me (or perhaps you). None of this was possible for my non-technical relatives before Ubuntu. THIS is what we do, and THIS is why Ubuntu is important to the Free Software ecosystem. We are mainstreaming Linux and Free Software. This is a dream thousands of us have had for 20+ years, and it is something that we have been promising (promised?) for that same length of time. It has been co-joined with "Open Source", it has experienced some individual successes on Servers, and with particular projects (Firefox, OpenOffice, Apache), but never before has Free Software opened Desktops by the millions -- yes MILLIONS -- as it has with Ubuntu as a distribution.

I am helping that happen in very tiny ways, but I'm very, very, VERY proud of it.

:-Dustin

10 comments:

  1. I've been reading your blog for a few years or so, and I knew you were productive, but this is a really impressive post. As an Ubuntu user, let me say, "Thanks!"

    ReplyDelete
  2. Great post Dustin. You've brought in a personal aspect to the work that you've done which is very inspiring!

    ReplyDelete
  3. Please push your upstream and other work in to Debian, Gentoo and Fedora.

    ReplyDelete
  4. I don't think it really is the lack of contribution people complain about (well, maybe a few from your direct upstreams)
    I think it is a collection of things like working in your own LP world without interaction with upstreams or copyright assignment (which is only OK if the FSF does it, companies are a whole different ball game) or the perceived lack of attribution.
    But anyways, as long as you deal with people there will be drama. Get used to it.

    ReplyDelete
  5. Keep on doing what you do. Ubuntu is an incredible value and a wonderful product. The LTS rocks. I use it daily and full-time--at work and at home. I recommend it to family and friends, because of its ease of use and stability. I ask, "Tell me what do you like to do?" Then I'm happy to demonstrate, using Ubuntu, a relevant open source solution that often surprises, delivers, and rewards. Now that's a cool feeling.

    ReplyDelete
  6. Tom,

    Of course I didn't censor your post...I was merely sleeping when it landed in my inbox :-)

    To your point, thanks for sharing.

    Everyone is welcome to participate in our "LP world". And yes, there is always drama...and FUD. Hopefully we respond to both appropriately.

    :-Dustin

    ReplyDelete
  7. No worries ;) Sorry, I just assumed it was in waiting mode the whole day. You should sleep :)

    I didn't say that the points I mentioned are valid, they just exist. I personally think that the LP will get bigger and more important. I especially like the LP/quickly/etc stuff. Those can really bring awesome stuff.
    There is no upside to copyright assignment to companies though.

    ReplyDelete
  8. Regarding No. 2, Documentation, I have a saying. There is nothing more useless than a piece of software or hardware with no documentation.

    ReplyDelete

Please do not use blog comments for support requests! Blog comments do not scale well to this effect.

Instead, please use Launchpad for Bugs and StackExchange for Questions.
* bugs.launchpad.net
* stackexchange.com

Thanks,
:-Dustin

Printfriendly