KVM Ubuntu Developer Week Session

My apologies for not announcing this in advance, but I just today gave a 1-hour session on KVM in #ubuntu-classroom in IRC for the Ubuntu Developer Week.

You can read the log here.

From this session, you should learn:

• a bit about what KVM is, how it works, what you need to have to be able to use it
• launching of a basic virtual machine (using testdrive)
• dissection of the most useful KVM command line parameters
• creating a backing disk image (even arbitrarily large, like 1 Petabyte!)
• live migration of a VM

Enjoy,
:-Dustin

LCA2010: Day 5, Wellington, New Zealand

The day opened with a series of Lightning Talks. Unfortunately, I missed most of these as I had some issues with checkout at the hotel. Paul Fenwick's was a hilarious satire about unfriending people on Facebook.

Next, I attended Andrew Tridgell talk on Patent Defense for FOSS Developers. Having worked at IBM previously, I (unfortunately) have read many, many patents. Tridge's talk was about how to read a patent. The most interesting point he made was about triple damages. Many open source projects (or companies) have a "don't read patents so that we're not one day liable for triple damages" policy. Tridge says this is dumb for open source projects/companies. If they're sued for damages for infringing a patent, then they're probably dead bankrupt. If they're sued for triple damages, well, they're still probably dead bankrupt ;-)

He talked about the three types of patent defense:

• non-infringement - we don't do that - best defense
• prior-art - someone did that before - very tricky
• invalidity - you can't claim that - almost impossible
  

The most important part is independent versus dependent claims. When working against a patent, focus on the independent claims. Prove that you don't offend those, and you won't offend the dependent claims either. And when looking at the independent claims, focus on the elements. Try to find elements that you omit.

I've read more patents that I care to admit, and I pray that I can avoid being dragged back into that world again. That said, God bless the people who still are doing this today. They're doing this behind the scenes, and they're getting no credit whatsoever for it. Thank you so much. What you're doing for us is more valuable than anyone knows.

Next, I attended Arnd Bergmann's talk on Virtual Network Switching across Hypervisors. Current virtual networking in KVM looks like a basic switch/router (showing a basic home Linksys router on a slide). If that's all you want, that works pretty much fine. Arnd is working on all the other complex networks, bridging, VLANs, etc. I'm following his work quite closely on the upstream mailing lists. His slides are available at: http://www.lca2010.org.nz/wiki/Talks/virtualnetworks.

I spent the next several hours with David Howells from Red Hat (author of keyutils and the linux kernel keyring). He was helping me with a small, but invasive change to the way eCryptfs Encrypted Home Directories loads keys into the keyring. He helped me move the keys from the user keyring to the session keyring. This will improve the clearing and expiration of keys once I've tested and released it. Many, many thanks to David for his help and patience. Cheers!

Rusty Russell gave his usual excellent presentation, this time about Hacking a Wiiremote for his 2 year old daughter. Funny, cute presentation. Basically, he attached a series of LEDs to a scrunchy that his daughter would wear on her wrist. The Wiiremote detects movent of the LEDs, and it's attached to a Linux system where his code runs, using libcwiid (pronounced lib-seaweed, heh). He wrote a couple of Python programs, one that lets her smear paint on the TV screen, and another that lets her smack a baseball bouncing around the screen. Rusty is the quintessential hacker ;-)

And, alas, the week is over... Finally, the Closing Session. Lots of thank you's all around. These guys and gals put on an amazing conference. I hope I'm so lucky as to attend again some time.

From here, I'm taking a couple of days off, and hiking the Routeburn Track, one of New Zealands 5 Great Walks. I'm really looking forward to a couple of days unplugged, and in the Kiwi back country. Hopefully I'll get some beautiful pictures, and have some stories to tell in another post.

Kia ora,
:-Dustin

LCA2010: Day 4, Wellington, New Zealand

Back at the conference for Day 4!

The opening keynote by Glynn Moody was entitled Hackers at the End of the World. He started out by noting how amazed he was when researching Rebel Code (Sep 1999 - Sep 2000). He conducted 50+ interviews and was struck by how nice people were. Dan Lyons said much the same thing at his keynote at Canonical's All Hands meeting last year.

Glynn talked about "free and openness" leaking out into areas beyond software. There was actually an open physics project that started 2 weeks before Linus' famous 8/23/91 post. The Open genome project was saved by a lone hacker (Jim Kent) and a cluster of 100 Linux PCs. Open Science, open access to results, data... In many ways, the hacker culture is paving the way for science.

Documentation licensing has evolved. GPL, FDL, Copyright Commons, GNU GFDL. Wikipedia is probably the easiest way to explain to other people what Free Software is. I think I'm actually going to start using that to explain my job to friends and family. "So, Dustin, what do you do?" "Well, you know how Wikipedia works? Well, I'm a computer programmer that works on software in a similar manner!"

He says that the new millennium is thus far all about sharing. Web pages mostly have visible source code. Flash/Silverlight being the nasty exception. Blogs, delicious, Flickr, Youtube, Facebook, Myspace. People want to share.

Twitter/identi.ca is the "release early, release often" principle, applied to thinking :-)

For 25 years, hackers have been showing others how to create and nurture sustainable commons through openness and sharing. Great talk!

Next, I attended a session on Flapjack Monitoring for the Cloud. It talks Nagios, and uses beanstalkd. The examples were mostly shown on Ubuntu and packaged in Launchpad PPAs. The goal of Flapjack is to be easy to install, configure, maintain, and scale. The project looks interesting, but I really couldn't quite see the application for "the Cloud" yet.

As always, Jeremy Allison's talk was excellent and entertaining. He spoke about Microsoft - The Elephant in the Room. Great talk about Microsoft, and all the dirty things they do to Open Source. He talked about a few famous cases (Samba, OOXML, TomTom). The good news is that we have made Microsoft blink in the staring contest. This is good, but we haven't had a knockout punch yet. Microsoft may be winning in court, but they're rapidly losing the mindshare battle. OOXML showed this in a nutshell -- they pushed the standard through, but everyone lined up against them. Patents are their nuclear option. TomTom was the first time Microsoft really tried this tactic; we all should be very afraid! Jeremy reminds us that IBM was once as feared and hated as Microsoft is today. Elephants can learn new tricks.

Neil Brown gave an interesting presentation on Design Patterns in the Linux Kernel. He gave lots of great examples, grepping through the kernel for some interesting recurring patterns. Jono Lange asked (what I considered to be a really funny) question. Neil had shown a few dozen binary searches in the kernel. Jono says that he's a python developer, rather than a kernel developer, but in Python, he'd just write a binary search function, and use that in all of these places :-) Things are done differently in the Linux kernel.

Finally, I attended Launchpad Code Reviews by Tim Penhey. He started out by referencing the very funny website, http://thedailywtf.com. The talk was an excellent tour of Launchpad's merge proposal and code review features.

After the conference we had a couple of beers on the waterfront at Mac's Brewery, and then headed over to the Professional Networking event held at the Opera House.

Today is the final day of the photo competition. These entries focus on the Lambton quadrant.






Cheers,
:-Dustin

LCA2010: Day 3, Wellington, New Zealand

The day started with Benjamin Mako Hill's talk on Antifeatures, about technologies that are artificially hindered copyrights and market segmentation. The lyrics to Happy Birthday are actually still under active copyright. Mako has a satirical website that shows the ridiculousness of this: http://unhappybirthday.com.

The first example of an antifeature is paying to not list your phone number in the phone book. Who here has paid to have their phone number not listed? Spammers pay the phone company to get lists of phone numbers, and you pay the phone company to not give it to them. Absolutely ridiculous. He talked about Gator, the "most successful spyware in history". You could pay OEMs to not install this crap on your computer. All in the name of market segmentation--different people pay different amounts for basically the same software. The Gator episode was 10 years ago ... surely things are different now... Slide shows: Windows Vista (Starter, Home Basic, Home Premium, Vusiness, Enterprise, Ultimate). Vista Basic can only run 3 graphical applications simultaneously. The goal was to make a barely functional, crippled Windows and offer it at almost no charge, but it was so bad, you really had to buy the upgrade.

Some cameras charge more to support the raw image format. Surely compressing the image using jpeg compression is more work (all images start out raw). Most of us here have bought a printer for $100, and then subsequent ink cartridges for $75... Modern print cartridges actually have some silly, copyrighted code in the cartridge that makes it basically impossible (illegal) to buy cheap refills. The XBox is a simple PC engineered not to run another OS (specifically, Linux). TiVo, same thing. Computers are ultimately perfect copy machines -- much effort has been spent in the last 20 years trying to make PCs less perfect copiers. Who here hates the DVD "unskippable track"?

Phones are the most pervasive computers in the world, and almost every one is locked by a SIM card. Who here had an OpenMoko? "It didn't do a lot of things, but some of the things it didn't do were pretty great", quote-of-the-day by Benjamin Mako Hill. DRM is basically the mother of all antifeatures. There are currently over 10,000 people employed in the DRM industry. You can submit your antifeatures at: http://wiki.mako.cc/Antifeatures.

I experienced a pretty crappy antifeature on my way over here... I had a total of 4 flights to get from Austin to Wellington. Austin -> Houston -> Los Angeles -> Auckland -> Wellington. Although I didn't know it when I booked my travel, I happened to be in Houston the day before my flight started. For this reason, I called Continental to ask if I could just "skip" my Austin -> Houston leg. I wasn't asking for a refund. Just to print my boarding passes as normal, tear up the AUS-IAH boarding card, show up in Houston with my ID and my other 3 boarding passes, and commence my journey. The Continental agent told me that this would cost $700 in addition to the $2500 that had already been paid for my fare. I would have had to pay Continental $700 to not take a flight. Effing retarded.

Next, I attended The Kernel Report by Jon Corbet (which actually hit Slashdot today). He list 7 challenges facing the Linux kernel:

1. Vitality - Andrew Morton famously said that he expected the number Linux kernel patches to drop off at some point, implying that one day we would "finish" this kernel project (Sep 2005, 2.6.14 release). Changesets are still increasing, in fact, at a rate of 144 changesets per day, 7330 lines of code added per day. In the last year, the kernel has added: GEM, ext4, staging tree, wireless USB, KMS, btrfs, squashfs, wimax, 4096 cpu, tomoyo, integrity measurement, nilfs, radeon r6xx/r7xx, perf counter, userspace char devices, kmemleak, radeon kms, kernel shared memory, dynamic ftrace, drbd, tcp cookie transactions, nouveau.
2. Scalability - in June 1996, Linux 2.0 added support for 2 CPUs. Today, Linux supports 4096 CPUs! Problem areas: 10G ethernet for large packets okay, not for small packets though; SSDs with 100K ops/sec are on the horizon. Scalability is a 2-way street; scaling down is important too, with so many of us here with root on our phone. The amount of resources required to run our kernel is increasing, fortunately behind Moore's Law (at least for now).
3. Storage - Ext4 - better perf, many limits fixed, ext3 compat, still stabilizing. Btrfs - is a totally new filesystem, performance, full checksumming, snapshots, is likely the Linux FS of the future. SSDs - poor performance over time.
4. Visibility - What do we want to know about what our system is doing?
5. (????) - Real time is important for gadgets (sound, music), but also financial services.
6. Containment - Virtualization. Containers, more efficient, trickier to implement, in progress, still a long ways to go.
7. Hardware - Nearly universal now. Support more hardware than almost any other OS now. Few remaining problems are in graphics and network adapters, -staging tree is helping.

Corbet suggested that LKML is now friendlier than the Ubuntu mailing lists.

Note: At the time, I took offense to Jon's comment, calling it a "potshot" against Ubuntu. Per a private conversation with Jon, he was referring to a really nasty ubuntu-users@ thread. I consider the ubuntu-devel@ and ubuntu-devel-discuss@ lists to more akin to LKML, as developer/developer or user/developer communication. The latter two lists are tightly governed by the Ubuntu Code of Conduct, and in my experience, are much friendlier than LKML, historically. Jon assures me that he meant no offense to Ubuntu with his referenced comment.

Thankfully two other speakers today (mako and mjg) mentioned the Ubuntu Code of Conduct as being both important and effective in the Ubuntu community.

Next, I attended a Survey of Open Source Databases by Selena Deckelmann. In 2005, there were basically 5 open source databases -- BerkelyDB, MySQL, PostgreSQL, SQLite. Today, there's 50+. She conducted a vast survey, asking a dozen or so questions of ~50 different open source database projects. See the results at: http://ossdbsurvey.org.

Ted Tso gave a talk on Ext4 -- the 5th such talk by Ted in the last year and a half. It's quite interesting to see how the talk has evolved from design level to the implementation retrospective that this talk was. This talk was absolutely standing room only. Since I was standing, I didn't take notes. Basically, Ted cased a couple of the most critical bugs that were found, how they were found and fixed.

The next talk was on The Linux Community by Matthew James Garrett. It wasn't a technical talk at all -- it was all about the soft skills and relationships in the open source community. He notes that there are lots of different communities in the Free and Open Source community, each with their own definitions, and different rules for inclusion and exclusion. He showed a picture of the 2009 Linux kernel summit, followed by the UDS Barcelona group picture.

Matthew notes how much hostility there (still) is in the Linux community. He quotes himself from 2004, having said "**** on my ****" on a public mailing list. He also quotes a recent saga harassing girls that in the Linux community.

Matthew notes that that Ubuntu introduced the Ubuntu Code of Conduct to avoid some of the failings of the Debian Project, which can be summarized with, "Be excellent to one another" -- Bill and Ted. The Ubuntu Code of Conduct has made Ubuntu users part of the community in a way that's unique among distros. He asks, "What do we want our community to be?" He seems to suggest that we need to be far more open and welcoming. Question from Martin Krafft, "What changed in you, Matt, between 2004 and now for you to speak on this?" He responded that he's had a change of heart, realizing he work with people without screaming at them. Applause from the audience, and Matthew says that the applause is bizarrely touching. :-)

I really enjoyed the next talk on Teaching FOSS in Universities, by Andrew Tridgell and Bob Edwards. It's not just about programming, or how to use Linux. It's about how to format and submit a patch, join a project, start a project, run a project. They taught a CS Masters class, with a combination of practical and theoretical training, trying to get students involved in real projects, and to understand the FOSS community. The audience asked if they used Jono Bacon's book Art of Community. Tridge said "No, not really", but did use Karl Fogel's book, Producing Open Source Software, (and mentioned it several times in the talk).

Their lectures included:

1. Intro to FOSS, getting started in a project, using source code management
2. History of FOSS, FOSS licensing, FOSS and the law
3. FOSS project governance, FOSS and business, FOSS motivation
4. Samba case study, FOSS distros and platforms, FOSS culture
5. Starting a new project, FOSS tales, release early/often
   

The Lab work included:

1. Install a FOSS project (a2ps), contribute to one of 5 suggested projects
2. Finding a FOSS project, getting in touch
3. Project work
4. Project work
5. Video presentation on the experience
   

I asked if this really needed to be a grad level course, and if it could be taught as an undergraduate course to teach some FOSS skills a bit earlier in the student's academic career. Tridge answered that he thinks this could be a 2nd or 3rd year course, easily. After they taught the course, they found Greg DeKoenigsberg's Teaching Open Source course.

What can you do? Approach your local uni! Drop a line to a FOSS friendly lecturer. Help with the TOS textbook. See:

• http://teachingopensource.org
• http://cs.anu.edu.au/students/comp8440
  

The most entertaining, hilarious talk of the day was about the World's Worst Inventions, by Paul Fenwick (in a mad scientist costume). This talk is brought to you from the L.O.O.N.I.E.S. the League Of Overambitious New Inventors, Engineers and Scientists. The presentation was a series of funny old ads things like cocaine pills for headaches, asthma cigarettes (not for children under 6 years old). There were some great slides on really bad toys. The feed-me cabbage patch, from 1996, had one-way metal rollers in the mouth of these dolls. They came with some plastic, fake food. But when the food ran out, refills were expensive. So kids fed this dolls all sorts of things. Particularly, hair, fingers, and siblings. These were the ultimate zombie dolls :-)

Anyone remember melty beads? My sisters had these things that could be melted into keychains. Rather than using heat, Bindeez used water to fuse together. Unfortunately, they replaced one chemical with another that was 10x cheaper. The problem was that the cheaper chemical was actually GHB (yup, Liquid Ecstasy). Kids nibbled on the beads and were entertained for hours before being pulled from the market.

In the '50s, good parents could buy the Atomic Energy Lab for their enterprising young scientists, complete with a Geiger counter and, with real uranium samples!

The image that brought the house down was US Patent #4320756, Fresh-Air Breathing Method and Device. Yikes!!!


And finally, a modern one you can buy straight off of Amazon.com today, the Laptop Steering Wheel Desk.

What a great talk, and a nice way to end a highly technical day.

There was a showing of Code Rush there at the conference--the story of Mozilla opening their code. The movie itself is a free download under the Creative Commons, so I downloaded it, and I'll watch it later, perhaps on my flight home.

Instead, I attended a screening of Peter Jackson's The Lovely Bones at the Embassy Theatre. This was poignant for a few reasons. Wellington, being the home town of Peter Jackson, and also being the Hollywood of New Zealand, a good bit of the movie was produced here. My new friends at Weta Digital did the digital effects rendering for the film (again, on a 5,000 node Ubuntu server farm). And the Embassy Theatre is phenomenal! Huge seats (with brass name placards on each), a perfect screen, perfect sound, very fancy.

Finally, I submitted a few more images for today's Photo Competition, of the Waterfront quadrant. For your enjoyment ...






:-Dustin

LCA2010: Day 2, Wellington, New Zealand

Day 2, Tuesday, January 19, 2010...

I already covered Tuesday's highlight in another post, noting that Weta Digital used Ubuntu on their 35,000 core to render the digital effects and graphics for the recent blockbuster, Avatar.

I had lunch with Paul Gunn, the gentleman who gave the morning presentation, and he introduced me to several of his colleagues at the end of the day, retiring to The Malt House for a few local ales, and hearing much more about their impressive data center and heavy using of Ubuntu throughout their infrastructure.

The day started with the keynote from Gabiella Coleman, an anthropology researcher who studies the free and open source culture. She started by quoting Dickens, "These are the Best of Times and these are the Worst of Times..." She talked about her introduction to Linux, a college classmate raving about a Slackware Linux CD. As she started her university research, that same classmate introduced her to the Copy Left movement. Her speech was interesting, very academic. She read a lot of quotes, and provided a fairly comprehensive history of Free and Open Source software.

Next I attended Paul Gunn's talk in the SysAdmin Track on Weta Digital's effects rendering data center, here in Wellington. Very early in the talk, he noted that 90% of their artists run the Ubuntu Desktop, with the remaining 10% using Windows. Adobe Photoshop is one of the few remaining apps that still need Windows for. He went on to talk more about their server render farm, noting that all of their 35,000 cores are also running Ubuntu. In the talk, he called these "Ubuntu Servers", which to me meant that they were installed from the Ubuntu Server seed. Actually, after having talked extensively with Paul and his colleagues, these nodes are actually imaged with the same Ubuntu Desktop image that the artists used. This consistency is critical for their operation -- a single image across thousands of systems. The size of the data center is just incredible. Somewhere around 5,000 HP (and some IBM) blades, mostly 4x or 8x Intel processors, for a total of about 35,000 cores (or render slots, as they call them). They have 2 petabytes of network storage, carved into hundreds of multi-terabyte NFS mounts. 10 Gigabit networking through. 10 years ago (October 2000), their render farm consisted of 13 machines, each were dual core 733MHz systems, with 100mbps networking. As you can tell, they have grown quite a bit. The rest of the talk was about heat and energy efficiency -- something that is quite important

Next, I attended a session in the Data Storage track on Sheepdog, a shared network storage solution for KVM. I've followed the discussions about Sheepdog on the qemu-devel@ and kvm@ mailing lists, so I'm fairly familiar with its promises. Admittedly, I haven't used it yet. This talk was a technical discussion of the design, which I found quite interesting. It's not a general filesystem, but instead designed specifically for QEMU. Volumes are divided into 4MB objects, each replicated to multiple nodes, for high availability. Their goal is to become the de facto standard for cloud storage. Ambitious. I'll keep my eye on it.

Back in the SysAdmin track, I attended a session on RAID. It was a lengthy narrative about this system administrator's quest to solve some strange behavior with his Linux software RAID. His talk walked through all the successive testing and debugging he did (perhaps interesting to those less expert in RAID administration). As it turned out, he used two Western Digital disks, one supporting some obscure error correction scheme and the other not. This minor difference was causing some annoying behavior.

I popped into the Launchpad track for James Westby's talk on Ubuntu distributed development. The audience (and in particular, a few Debian developers) were most interested in bzr's "send" command. Other audience members seemed to be impressed with bzr being able to handle branching from cvs, svn, and git repositories (with the appropriate plugin).

Next, I attended a short session on Linux Containers, by Sam Vilain. He noted the difference and situations where you might choose Xen/KVM virtualization (need full hardware emulation, different kernels), and where you might choose containers (need maximum performance). In truth, he noted, many situations would benefit from a combination of both. I tend to agree with this, and would like to see Ubuntu fully support both KVM and LXC in appropriate situations. He used "sudo apt-get install lxc" on Ubuntu in his examples and demonstration.

I attended a session in the Storage Track by Arjen Lentz on Multi-Master Replication with MMM that was completely over my head.

And I finished my day in the Storage Track with an excellent talk (and demo!) by Ben Balbo, on his roll-your-own-dropbox implementation. He gave a bit about the design of Dropbox, reasons why you might use Dropbox, and reasons why you might not (cost, privacy, bandwidth). Particularly here in New Zealand and Australia, latency to the US or Amazon can really be a problem. So he demonstrated his solution, which is built on top of git. I must say, it looked very impressive. There was no mention of Ubuntu One, though it was certainly in my mind.

On a second note, two of my photographs (1 and 2) were selected as finalists in the Courtenay Quadrant contest!

Today's contest is from the Cuba Quadrant. Here are my submissions:




:-Dustin

35,000-Core Ubuntu Farm Renders Avatar

I just attended Paul Gunn's talk at LCA2010, entitled:

• Challenges in Data Centre Growth (or, "You need how many processors to finish the movie???")

Paul is a Systems Administrator at Weta Digital, a Wellywood digital effects studio here in Wellington, New Zealand. Check out some of the feature films that Weta Digital has worked on, and I think you'll recognize a few. District9, Day the Earth Stood Still, Jumper, King Kong, Lord of the Rings, Fantastic Four, Eragon, X-Men, i-Robot. Wow!

It was a great talk, about the type of data center needed to render special effects in today's blockbuster movies. They have a 2 Petabyte disk array, 10gbps networking, and 35,000 cores (4,000+ HP blades) in their data center, and still it takes 48 hours to render some of their graphic sequences.

According to Paul, Ubuntu is at the core of all of this, running on all of the rendering nodes, and 90% of the desktops at Weta Digital. He notes that his farm (he calls it a "render wall") is in fact an Ubuntu Server farm, and not RHEL as he has seen reported in the media.

Here's a couple of articles on Weta Digital's data center and their work on Avatar:

• http://www.information-management.com/newsletters/avatar_data_processing-10016774-1.html
• http://www.itworld.com/hardware/93127/data-center-plays-supporting-role-avatar

:-Dustin

LCA2010: Day 1, Wellington, New Zealand

I have the privilege of attending linux.conf.au, LCA2010 in Wellington, New Zealand, representing Canonical and Ubuntu at one of the most reputable Linux and Free Software conferences in the world. While LCA has been around since 1999, this is my first attendance.

Here are a few of my notes and highlights so far.

Day 0, Sunday, January 17, 2010...

The event actually open the previous afternoon, with registration and a Newbie Welcome talk by Rusty Russell. Rusty discussed some of the history and traditions of the conference. We adjourned to a nearby watering hole, the Green Man Pub, where we newbies were treated to a round of beer. I talked a bit about LXC containers with Rusty. He expressed a lack of confidence in the security of containers, claiming that if he has root in a container, he can do very malicious things to the host with iptables in the guest. As a side note, the schwag bag is actually pretty impressive ;-)

Day 1, Monday, January 18, 2010...

The day started with a welcome plenary in the auditorium. The general organization of conference was discussed. The highlight was this hilarious video about Wellington:

• http://www.youtube.com/watch?v=AWVoaf3OuN8

And there's a charity auction this week, benefiting a local life-flight charity. The winner(s) of the auction get to participate in a training mission aboard the helicopter, dangling on a cable below the helicopter. Sounds like fun!

The first two days of LCA consist of a series of mini-summits. I attended all of the Distro Mini-Summit sessions on Monday, organized by Martin Krafft.

Aside... This was the first time I met Martin in person, however, I certainly recognized his name. When I interviewed for a job with Canonical, the feedback I got at the end of the interview was that I had tremendous Linux experience, but I was light on Debian specific expertise. Later that I week, I bought Martin Krafft's book, The Debian System, and read it cover to cover. I eventually got the job. I think I owe Martin a beer :-)

The first session I attended was led by Lucas Nussbaum about Debian Quality Assurance. He noted that Debian is well known and respected for long/bumpy release cycles, and in general, quality. Debian has a famous "Release when ready" policy, that is unique among all Linux distributions. Each package is "owned" by an individual developer. Debian also has the concept of "Release-Critical bugs", which block release. There are also "Non-maintainer uploads". Lucas notes a real problem in Debian with "data aggregation". There is no "Launchpad" for Debian. There are a bunch of disparate sites and databases (PTS, BTS, UDD, etc). Archive rebuilds identify FTBFS problems and toolchain bugs. Lintian provides extensive static analysis of source and binary packages.

The next talk was about the Linux Standards Base, by Bdale Garbee. I've been around the LSB in one way or another for several years. I certainly understand why it's important, and I think it would be really advantageous to the Ubuntu Server. Bdale's talk was interesting, in particular coming from a Debian person. It's been my experience in the Debian/Ubuntu world that there's quite a bit of hostility toward the LSB due to it's dependence on RPMs. So it was an eye opener to hear Bdale speak favorably of it.

Martin Krafft then talked about vcs-pkg.org. All distros are trending toward DCVS packaging. He gave a lot of credit to Ubuntu for leading the way, with Launchpad and bzr, noting bzr's impressive merge conflict resolution handling. There was a bit of dialog with a Fedora developer present, noting that Fedora is also moving all packaging to git (from cvs).

Michael Homer of GoboLinux then talked about domain specific package systems. Most in the room agreed that this presents an interesting, as yet unsolved problem for most distros. The key idea is there are other non-distro sources of software, such as CPAN Perl modules. Similar repositories exist for Python, Ruby Gems, Java. Matt Zimmerman also mentioned Firefox add-ons, noting that these have actually been pretty successful, and not really causing havoc within Ubuntu. Members of the audience (including myself) noted that some big software vendors are just packing up their entire toolchain and libraries with their software in a tarball, or even virtual machine image.

Scott James Remnant had a session scheduled to talk about Upstart and Boot-times, but he had some travel issues, and thus we just continued the previous discussion.

The next talk was on OpenSolaris packaging. It's basically RPM. I didn't have a lot invested in this talk.

I gave the next talk, about Launchpad, from the perspective of an upstream maintainer and developer. Here are my slides, shared under the CC-by-SA 2.0 license. I'm proud to say that the room was packed, standing room only. The Launchpad development team are holding their own Launchpad Mini-Summit at LCA today. I expect they will talk in much greater detail about all things Launchpad.

Finally, Lucas Nussbaum closed the day talking about Debian/Ubuntu collaboration. It was quite an interesting talk, similar to some things Lucas talked to Ubuntu developers about at UDS-Dallas. I asked if he ever thought Debian would adopt Launchpad for at least some of its features (the bug tracker, for starters). Lucas and Martin agreed, "no", Debian would never want to depend on Canonical hosting a Debian critical service. Perhaps, and it's a long shot, but perhaps they might do this if they could start their own Launchpad instance. But they suspect most Debian developers would be against this.

I had a couple of beers with some of the Launchpad Team (Tim, Jono, Michael), and Lucas at a nearby sports pub. I swear, cricket is a complicated sport.

Finally, I finished my evening wandering around Wellington, taking pictures, a few of which I'm entering into the LCA2010 Photographic Competition.

My entries for the the Courtenay quadrant of Wellington are the following images. Cheers.







Until tomorrow!

:-Dustin

Streaming Live, Down-sampled Audio from MythTV over SSH

I'm hanging out here in Wellington, New Zealand this weekend ahead of linux.conf.au.

Unfortunately, I'm missing the New Orleans Saints vs. Arizona Cardinals NFL playoff game. I'm not a huge NFL fan, but having grown up in Louisiana, I am a Saints fan. And they have had a pretty incredible season.

I scoured the Internet trying to find a live stream, but for various reasons, I couldn't get any of them to work. My mom has a Slingbox setup, and I have gotten that to work under Wine, but I try avoid it whenever possible.

Hence, I hacked this up, which allows me to stream audio over SSH from my MythTV backend at home. I thought I'd document it, in case it's useful to anyone else. Or more likely, to myself some time in the future.

First, I used MythWeb to record the program. While normally I would record this from my HD tuner, for this usage, I want the low-def version, as my bandwidth here in NZ leaves a bit to be desired.

Next, I located the actual path to the file actively being recorded. It was the most recently touched file in my recordings directory. I found it like this:

ls -t /var/lib/mythtv/recordings/*mpg | head -n1 
/var/lib/mythtv/recordings/1002_20100116163100.mpg

Now I just want to extract the audio of this mpg. And my bandwidth is fairly constrained, so I'm going to downsample it from 44KHz to 11KHz too, using ffmpeg:

tail -f/var/lib/mythtv/recordings/1002_20100116163100.mpg \
  | ffmpeg -i - -ar 11025 -f wav /tmp/out.wav
...
Input #0, mpeg, from 'pipe:':
Duration: N/A, start: 3069.045267, bitrate: 6384 kb/s
Stream #0.0[0x1e0]: Video: mpeg2video, yuv420p, 480x480 [PAR 4:3 DAR 4:3], 6
000 kb/s, 29.97 tbr, 90k tbn, 59.94 tbc
Stream #0.1[0x1c0]: Audio: mp2, 48000 Hz, stereo, s16, 384 kb/s
Output #0, wav, to 'out.wav':
Stream #0.0: Audio: pcm_s16le, 11025 Hz, stereo, s16, 352 kb/s
Stream mapping:
Stream #0.1 -> #0.0
size=   55680kB time=1293.60 bitrate= 352.6kbits/s

Here, I can see the new bitrate is ~350kb/s -> 44KB/s, which is about what I'm getting here in the hotel. So the quality will be a bit lower (more like AM radio), but it shouldn't skip, or pause to buffer.

Now, I just need to get a stream that I can pipe directly into my player. I used sshfs to mount the remote output directory locally

mkdir /tmp/mnt
sshfs remote.example.com:/tmp /tmp/mnt

Finally, I just need to pipe this output to my player, vlc, into its standard in.

cat /tmp/mnt/out.wav | vlc -

And there we have it! Working like a champ. Streaming down-sampled audio from my MythTV back home over SSH, some 15,000 miles away. Who dat!

:-Dustin

Book Review: Freedom (The Daemon Sequel)

Background

In September of 2008, I blogged a book review about a little known techno-thriller called The Daemon, which was penned under the pseudonym Leinad Zeraus. I called it the "most thought provoking cyberpunk novel I think I've ever read".

My blog post found it's way in Daniel Suarez's Google Alerts reader, and a few days later, he introduced himself as the real Leinad Zeraus and we exchanged a few emails. I noted my profession as a developer in the open source community, and he responded:

"I sincerely appreciate the kind words. Daemon (and the upcoming sequel) are labors of love to me, and knowing that they strike a chord with folks in the open-source community makes my day."

Eventually, his privately published small printing of the book was picked up by a major publisher and it was re-printed in hardback. Someone from the publishing house contacted me and offered me 3 hardback copies to give away on my blog, however I saw fit.

And so in January 2009, I blogged a set of "challenges", promoting the use of eCryptfs and Encrypted Home Directories in Ubuntu, and offering a copy of the book to the winner of each of 3 increasingly difficult challenges. I enjoyed designing and running the contests, and it was really a booming success, promoting Daniel's book (which itself deals with security, privacy, and encryption), as well as my technical work on Ubuntu and eCryptfs.

So in December 2009, I returned from a holiday and found one of my biggest surprises of the Christmas season -- an advance copy of Freedom, the sequel to Daemon! (Along with a hand-written note from Daniel.) I tore through the book in 3 sittings, thoroughly enjoying it.

As of January 7, 2009, it's now available for purchase.

Here's my review. No spoilers, if you've already read Daemon. If you haven't, then you might want to do so before going further.

'Freedom' Review

Freedom picks up where Daemon left off, a few months later. Most of the major characters are back, in one form or another. And the lines between good and evil are even more fuzzy.

The Daemon itself is still fully operational. The Darknet online community of the Daemon-savvy has grown exponentially beyond it's early rollcall of Gearheads and Slashdotters to include people of all walks of life. Most importantly, the Darknet is evolving at the speed of light. It's at that point when your Grandma joins Facebook, and your Grandpa starts blogging -- you never know who you're going to find on the Darknet. D-space is ever growing, with hundreds of thousands of people wearing D-space sunglasses, able to see objects and meta-data about the world around them. Again, it's at that point when everyone has a smart phone, and even Old Uncle Fester is using GPS and Google Maps.

In the real world, the economic times are all too familiar. Fuel prices are somehow now obeying Moore's Law, doubling every few months. And the US Dollar is falling precipitously, threatening to take the rest of the world down with it. At one point, Natalie (our heroine from the NSA) rifles through her mail, discarding her financial statement in despair. Wow, I've been there...out of school, in the tech industry for 10 years now, maxing out my 401(k) the entire time, and barely have what I've put in. Yeesh.

All financial power is consolidated in the hands of a few large, private corporations, who have contracted private mercenary armies to keep it that way. Governments are becoming stool pigeons of the uber-wealthy. All in the name of control.

I particularly enjoyed more than a handful of "open source" mentions. Heck, the title of the book is Freedom. In any case, the concept of democracy (or lack thereof) is certainly a central theme. I dog-eared page 265, which has (I think) the most important quote of the book. Jon Ross says:

"Democracy requires active participation, and sooner or later someone 'offers' to take all the difficult decision-making away from you and your hectic life. But the darknet throws those decision back onto you. It hard-codes democracy into the DNA of civilization. You upvote and downvote many times a day on things that directly affect your life and the lives of people around you--not just once every few years on things you haven't got a chance in hell of affecting."

Wow. Ain't that the truth! I could spend all day on the phone with Palm's tech support trying to explain to they that their TouchStone charger doesn't work worth a damn, and the Palm Pre's charge-handling software is oh-so-buggy... Or I could spend 1 minute downvoting the item on Amazon.com, and 5 minutes writing a review stating that the $69 accessory doesn't actually work as advertised. Which of those methods are the most effective? (Look for that blog post soon...)

Personally, I can't think of a more robust online community where democracy, up/down voting, and reputations are as important and successful as the Ubuntu community. We're producing an operating system and a vast set of applications (30K+ packages at my last check) in a super-democratic manner. Anyone can open bugs, offer fixes, commentary, review, etc. People gain reputations, and in our online community (Launchpad.net), everyone has a "Karma" score. While fuzzy, it does subtlety help "reward" some people for their contribution and establish/verify reputations. There's a key parallel to Launchpad's Karma in the Darknet.) And we have our own mission, against a nefarious, well-funded, ruthless enemy who restricts our software freedom, our Bug #1, filed by our own fearless leader, who also happens to be a wealthy computer programmer ;-)

Forgive me, Daniel for perhaps reading too much into it, but there's a very interesting parallel between Ubuntu's struggles and successes and those in Freedom. I would be really interested to hear from other readers of the book on their opinions on the idea.

I enjoyed the ending to Freedom much more than the ending to Daemon. Somewhat rare in the techno-thriller genre, we have a sequel that delivers as much of a punch as the original, introduces new concepts, and truly compliments its predecessor. If you're hoping to read more thought provoking literature in 2010, I'm pleased to recommend both Daemon and Freedom. And I believe there are many people in the Ubuntu (and other free software) communities that will identify with our hero's plight in Freedom. Enjoy!

Cheers,
:-Dustin

Byobu Custom Status Notifications in 3 Easy Steps!

Richard Johnson blogged a really cool hack for Byobu a couple of days ago. He created a custom script to retrieve his next appointment from his calendar and display it in Byobu's status bar. This is really cool ;-)

Many Byobu users have asked me for this sort of functionality over the last year -- the ability to add custom status items to the lower bar. I'm pleased to announce that this is now possible as of Byobu 2.40+!

This version is already in Lucid, but if you're not yet running Lucid, you can grab the latest backported package from the Byobu PPA:

• https://launchpad.net/~byobu/+archive/ppa

After upgrading Byobu, you can create your own custom script, in 3 easy steps!

I run a lot of KVM virtual machines on my Ubuntu servers, usually through the Ubuntu Enterprise Cloud and Eucalyptus. At any given time, when I log onto a Eucalyptus Node and automatically launch Byobu, I'd like to know how many virtual machines are running on that system. So I create a simple shell script that does something like this -- grep the process table for kvm processes and echo a count of VMs running.

Now, let's add this to my Byobu status bar!

1. Create your Byobu bin directory:

mkdir -p $HOME/.byobu/bin

2. Create your script, naming it NN_foo, where NN is the number in seconds that you want to run your script, and foo is some description of your status item:

cat > $HOME/.byobu/bin/10_vms <<EOF
#!/bin/sh
echo "VMs:\$(pgrep -c kvm)"
EOF
cat $HOME/.byobu/bin/10_vms

Note that I called this script "10_vms". The number "10" is the frequency in seconds you want Byobu to refresh this status item. You need to be careful here, making sure that your status program doesn't gobble too much CPU! Choose this number carefully, and consider how often the information displayed is likely to change. I recommend running "time" on your script a few times, seeing how long it actually takes to run. My example is trivial, and executes within 0.003 seconds on my system, so running it every 10 seconds is still relatively inexpensive. Your script might be much more complex, and may not need to be run as frequently!

3. Make your script executable:

chmod +x $HOME/.byobu/bin/10_vms

And within a couple of seconds, it should show up in your status bar!


Related hacks:

• You can put as many of these as you want in your $HOME/.byobu/bin.
• You can enable/disable each by changing the permissions, adding or removing the executable bit (chmod +x or chmod -x).
• You can disable all of your custom scripts in the Byobu Menu with:
  F9 -> Toggle status notifications -> Disable "custom" -> Apply
  
• You can write your custom script or program in any language, as long as it's executable and emits something on stdout.
• You can even add support for the "Byobu Status Detail" feature (see ctrl-a-$), if you add a handler for a "--detail" command line option. e.g. for my VM script, I could add the following line:

  [ "$1" = "--detail" ] && ps -ef | grep kvm

If you find this useful, and create some cool, custom scripts, please post a comment! -- Ideally with a link to your script's source code, and a GPLv3 copyright header ;-)

Happy New Year!

:-Dustin