Still Running Ubuntu in 2012

Winter time is road race season in Austin, Texas.  The weather is cool and sunny, and it's actually quite a nice time to venture out for some long runs.  I've run three races in Austin so far this year.

Austin Gorilla Run 5k

Former Gazzanger Unji Udeshi was one of the organizers and race chairs for the Austin Gorilla Run, which was truly one of the most unique races I've ever experienced.

Proceeds from the race directly benefit Ugandans, Rwandans, and citizens of the Democratic Republic of Congo in veterinary educational training, with the end goal of protecting the highly endangered Mountain Gorillas in Africa. In 1987 there were only 248 mountain gorillas alive in the world, but through the veterinary and conservation efforts of the MGCF, the population has nearly tripled to 720.

Over 1200 of us dressed up in full Gorilla suits to "run" a 5K (for a very loose definition of "run").  Mostly we just goofed off and had a really good time.

Here's a quick pose in the Gazzang parking lot before the race....

Being the good gorillas were were, Larry (Gazzang CEO) and I took a break from the race to a little climbing.

I brought my dogs, Tiger and Aggie, who were a little skittish of all the goofball gorillas at first!  They're really well trained dogs, and are running up to 15 miles with me at a time now.  Kim took a ton of awesome pictures.

And our crew, unmasked and post race enjoying the Silverback Pale Ale from Austin's own Thirsty Planet Brewery...  Unji, Larry Warnock, Liz Britain, Catelin Warnock, Dustin, Tiger, and Aggie!

Austin 3M Half Marathon

A few weeks later, I shucked the gorilla suit and ran Austin's 3M Half Marathon.  This is quite simply my favorite race!  This was my 4th running of the race.  I think I actually first fell in love with running at the finish line of this race in 2004, which I finished in a blazing (for me, anyway) time of 1:48:14!

Well, I was a little shy of my personal best time, but I did finish in 1:59:07, which beat my goal of 2 hours.

I was proud of my time, but I was far more proud of my splits, actually.  Almost perfectly negative splits!  Negative splits mean that you accelerate your pace throughout the race.  It takes quite a bit of discipline to perfect, and this is about as close as I've ever come:

Mile 1     9:56 Min/Mi
Mile 2     9:30 Min/Mi
Mile 3     9:38 Min/Mi
Mile 4     9:29 Min/Mi
Mile 5     9:21 Min/Mi
Mile 6     9:07 Min/Mi
Mile 7     8:58 Min/Mi
Mile 8     8:46 Min/Mi
Mile 9     8:37 Min/Mi
Mile 10    8:44 Min/Mi
Mile 11    8:35 Min/Mi
Mile 12    8:27 Min/Mi
Mile 13    8:18 Min/Mi
Mile 13.1  7:53 Min/Mi

Austin Livestrong Half Marathon

This past Sunday, I ran the Half Marathon portion of Austin's Livestrong race, benefiting Austin native Lance Armstrong's awesome cancer foundation.  Another beautiful, perfect morning for a run!

Now, originally, I registered for the full marathon, and had every intention on running it (would have been my 5th full marathon).  I trained myself up to 15 miles, but I had a minor surgery in November and missed 8 weeks of training, and couldn't quite get myself up over the 20+ mile barrier in time.  No matter, I'm planning on running the Marine Corps Marathon (again) in Washington DC in October as my penance.

As usual, I wore my I'm Running Ubuntu shirt.  As usual, it evokes a response, Austin having such a healthy tech community.  Interestingly, though, this time I heard a lot more "Ubuntu...Linux...yeah!" catcalls than ever before.  I liked that a lot, as it really showed how many actually recognized Ubuntu as a Linux distribution, rather than just a fun word to yell at someone running past them :-)

I beat my 3M time by 28 seconds, finishing in 1:58:39.  Interestingly, though, this time I did not run negative splits.  My splits were actually pretty flat, with my fastest mile (8:47) less than 30 seconds different from my slowest mile (9:16), which is fairly tight, compared to the 1m40s difference between my fastest and slowest mile in the 3M.  It's a different style of running, trying to bang out the same pace mile after mile, than the negative split approach.

Mile 1     8:59 Min/Mi
Mile 2     8:56 Min/Mi
Mile 3     9:10 Min/Mi
Mile 4     9:07 Min/Mi
Mile 5     9:02 Min/Mi
Mile 6     9:08 Min/Mi
Mile 7     8:54 Min/Mi
Mile 8     8:39 Min/Mi
Mile 9     8:48 Min/Mi
Mile 10    8:50 Min/Mi
Mile 11    9:04 Min/Mi
Mile 12    8:47 Min/Mi
Mile 13    9:16 Min/Mi
Mile 13.1  7:40 Min/Mi

So what's the point of this post?  Get out there and get some exercise!  Support a charity and run a race or two!  Support your own cause (Ubuntu/Linux for me) by wearing a shirt and showing some pride for something you believe in!

:-Dustin

Thoughts on Hiring Linux Hackers (in 2012)

I have interviewed hundreds of candidates and had the delight of hiring dozens of Linux and open source developers, engineers, and interns over the last 10 years -- at IBM, Canonical, and now Gazzang.  The most recent one signed his contract this morning, in fact!  It's quite a rush to bring new talent into a small team.

Linux jobs are actually hotter now than ever before!  The Wall Street Journal picked this up recently.  And while HostGator has been running giant billboards throughout Austin for at least 2 years now, which plainly asks, "Do you know Linux?  We're hiring!" -- I was impressed to see that they had the same billboard scaled up to 3-stories in height right in Times Square, New York.

Given that my own well being is so deeply invested in being an open source hacker, I selfishly love seeing the Linux and open source job market expanding so vibrantly.

From the interviewer's chair, however, my poking and prodding of a given candidate's Linux skills have changed a bit over those 10 years.  I'm often looking for the candidate's inquisitive nature.  I want to know how interested they really are in going down the rabbit hole.

• 10 years ago, you had to know how to deploy and run a LAMP stack, and hack your way around Apache, MySQL, PostgreSQL, PHP, Perl, and Python.  You would shriek in horror at bad HTML and CSS and could really make a website sing with a little Javascript.
• 9 years ago, I wanted to see someone who regularly compiled their own upstream kernel, maybe tweaked a few configuration options on or off just for fun.  Bonus points for each additional software package you compiled from source.  Gentoo users were shoe-ins.
• 8 years ago, I wanted to talk to people who were sending and receiving PGP or GPG signed, encrypted email.  I was delighted by those who had at least 1024D keys!
• 7 years ago, I found users who were willing and able to tweak their SELinux policies and AppArmor profiles absolutely intriguing.  If you were running SELinux in enforcing mode on a production system, well, damn, you probably got the job!
• 6 years ago, I wanted someone who had built their own Beowulf cluster, for fun, over the weekend.  If not Beowulf, then some sort of cluster computing.  Maybe Condor, or MPICH.
• 5 years ago, I'd structure some conversation around reinstalling dd-wrt or openwrt firmware on routers.  What serious hackers would run stock router firmware?!?
• 4 years ago, I needed you to have experience with open source virtualization, such as KVM, Xen, and QEMU.  Oh, and surely you're running MythTV on a few computers around the house, right?
• 3 years ago, it was all about developers who had Launchpad or Github accounts, had written some open source software and packaged it for Ubuntu or Fedora.  While your friends update one other over Facebook, you're pushing updates over git and bzr.
• 2 years  ago, I was interested in people who had built or deployed their own cloud infrastructure using Eucalyptus or OpenStack.
• And last year, it was all about the move from traditional configuration management to cloud-ready service orchestration; experience with Puppet/Chef/Juju were golden.

Nowadays?  Well, it's additive, to an extent.  Hopefully you have the LAMP stack and kernel compilations in your pocket, can send and receive signed/encrypted email.  No real hacker ever runs stock firmware on their router, surely you're using virtual machines and cloud computing on a daily basis, and hopefully you spend as much time on Launchpad/Github/StackExchange as Facebook/Twitter :-)

But you need to be on the cusp of what's next.  I'm hoping you've rooted your phone, jacked your bootloader, and installed a CyanogenMod of your choosing -- at least on your phone at least if not your tablet and e-Reader too!  Hopefully you've tried out this big data business and threw together a map-reduce Hadoop job or two, just for grins.  Clearly you'll have a strong, informed opinions on Unity vs. Gnome3, upstart vs. systemd, and the UEFI secure boot mess.

Oh, and big bonus points if you read my blog.  But you knew that already.  If you read my blog, you've seen this.  And this is what we'll talk about in our interview :-)

:-Dustin

ecryptfs-utils-96 released

-utils-96

ecryptfs-utils-96 has been released, with upstream tarballs (and signatures) available on Launchpad at:

• https://launchpad.net/ecryptfs/trunk/96

And now in the Ubuntu precise development release.

Special thanks to first time contributors Colin King and Eddie Garcia!


[ Dustin Kirkland ]
  * CONTRIBUTING:
    - added a new file to describe how to contribute to ecryptfs
  * === added directory img/old, img/old/ecryptfs_14.png,
    img/old/ecryptfs_192.png, img/old/ecryptfs_64.png:
    - saving the old logos/branding for posterity
  * debian/copyright, img/COPYING:
    - added CC-by-SA 3.0 license
    - use the text version
  * img/ecryptfs_14.png, img/ecryptfs_192.png, img/ecryptfs_64.png:
    - added scaled copies of images used for Launchpad.net branding
  * src/utils/ecryptfs-recover-private: LP: #847505
    - add an option to allow user to enter the mount passphrase,
      in case they've recorded that, but forgotten their login
      passphrase
  * src/libecryptfs/sysfs.c: LP: #802197
    - default sysfs to /sys, if not found in /etc/mtab
    - it seems that reading /etc/mtab for this is outdated
    - ensure that ecryptfs works even if there is no sysfs entry
      in /etc/mtab
  * src/key_mod/ecryptfs_key_mod_tspi.c: LP: #462225
    - fix TPM and string_to_uuid 64bits issue
    - thanks to Janos for the patch

  [ Tyler Hicks ]
  * CONTRIBUTING:
    - clarified how to contribute to the ecryptfs kernel module
  * tests/lib/etl_funcs.sh:
    - created eCryptfs test library of bash functions for use in test
      cases and test harnesses
  * test/etl_add_passphrase_key_to_keyring.c:
    - created a C helper program to allow bash scripts to interface to
      the libecryptfs function that adds passphrase-based keys to the
      kernel keyring
  * tests/kernel/tests.rc, tests/userspace/tests.rc:
    - created a test case category files for test harnesses to source
      when running testcases of a certain category (destructive, safe,
      etc.)
  * tests/run_tests.sh:
    - created a test harness to run eCryptfs test cases
  * tests/kernel/miscdev-bad-count.sh,
    tests/kernel/miscdev-bad-count/test.c:
    - created test case for miscdev issue reported to mailing list
  * tests/kernel/lp-885744.sh:
    - created test case for pathconf bug
  * tests/kernel/lp-926292.sh:
    - created test case for checking stale inode attrs after setxattr
  * tests/new.sh:
    - created new test case template to copy from
  * tests/userspace/verify-passphrase-sig.sh,
    tests/userspace/verify-passphrase-sig/test.c:
    - created test case, for make check, to test the creation of
      passphrase-based fekeks and signatures
  * configure.ac, Makefile.am, tests/Makefile.am, tests/lib/Makefile.am,
    tests/kernel/Makefile.am, tests/userspace/Makefile.am:
    - updated and created autoconf/automake files to build the new tests
      directory
    - added make check target

  [ Eddie Garcia ]
  * img/*: LP: #907131
    - contributing a new set of logos and branding under the CC-by-SA3.0
      license

  [ Colin King ]
  * tests/kernel/extend-file-random.sh,
    tests/kernel/extend-file-random/test.c:
    - Test to randomly extend file size, read/write + unlink
  * tests/kernel/trunc-file.sh, tests/kernel/trunc-file/test.c:
    - Test to exercise file truncation
  * tests/kernel/directory-concurrent.sh,
    tests/kernel/directory-concurrent/test.c:
    - test for directory creation/deletion races with multiple processes
  * tests/kernel/file-concurrent.sh,
    tests/kernel/file-concurrent/test.c:
    - test for file creation/truncation/unlink races with multiple
      processes
  * tests/kernel/inotify.sh, tests/kernel/inotify/test.c:
    - test for proper inotify support
  * tests/kernel/mmap-dir.sh, tests/kernel/mmap-dir/test.c:
    - test that directory files cannot be mmap'ed
  * tests/kernel/read-dir.sh, tests/kernel/read-dir/test.c:
    - test that read() on directory files returns the right error
  * tests/kernel/setattr-flush-dirty.sh:
    - test that the modified timestamp isn't clobbered in writeback
  * tests/kernel/inode-race-stat.sh, tests/kernel/inode-race-stat/test.c:
    - test for inode initialization race condition


 -- Dustin Kirkland  Thu, 16 Feb 2012 14:23:18 -0600                                                                                                                                                                                                        


:-Dustin

Gazzang Bang and the SXSW Startup Pub Crawl

The Gazzang office at 502 Baylor Street in Austin, Texas is one of the destinations of the 2012 SXSW Startup Pub Crawl, on Thursday, March 8th.

Join us between 4 and 10 pm for an open house, drum circle, and some awesome live music from the Lost Pines bluegrass band!  Please RSVP here.  Come talk to us over free beer and food about Cloud security, data privacy, encryption, eCryptfs, key management, Linux, and Ubuntu.  Meet the entire cast of the Sh*t IT Security Guys Say short film.  And tap into the vibrant tech start-up culture that's rocking downtown Austin by day, juxtaposed against the awesome live music culture that rocks downtown Austin by night.



View Larger Map

Come get your bang on!

:-Dustin

Gazzang Presents: Sh*t IT Security Guys Say

We had a blast at the Gazzang offices last week shooting this fun video, Sh*t IT Security Guys Say.  What a great way to kick back and have a little fun on a Friday afternoon ;-)






We worked with Austin filmmaker Brandon Stephens who took some time away from work on his feature film, Enemy of the Mind, to hack on this little project.  Our CEO Larry Warnock (Mr. Backdoor) called the shots and our new Marketing Director, David Tishgart (Mr. Redbull) handled the script.  Also featured in the short: Ben First (Marketing, aka Mr. Ruby), Liz Britain (Marketing, aka Ms. Slashdot), Rob Balena (Sales, aka Mr. Millennium Falcon), Sergio Pena (Mr. $*&%!#), Eddie Garcia (Engineering, aka Mr. IT), and I guess I'm Mr. Wingdings ;-)

As many of my fellow hackers, I predictably cringe when I watch a movie or a tv show and the hapless IT characters attempt to interface with a computer or discuss technology.  The Matrix, The Net, Swordfish, whatever, it's all painful to hear.  And funny enough, our little video is no different, and this time I actually share the blame :-)  Most of our one-liners make no IT sense whatsoever.  And while some of the one-liners I proposed made perfect IT/Security sense, but they just didn't play well on the screen.

In any case, for my hacker/dev/IT peeps, here's my full list of one-liners I proposed for our project:

 - Right, RSA 4096 is definitely the way to go - Ubuntu or Fedora? - Did you read Bruce Schneier's post today? - Wow, check Slashdot! - Open a new terminal - Emacs or Vi? - Grab my public key - apt-get dist-upgrade - Sure, I encrypt my home directory - Hang on, I'm recompiling my kernel - PC Load letter????  The f*ck does that mean? - Yeah, I need to merge those changes - We're moving from MD5 to SHA512 hashes - Of course I've rooted my Android! - Chef or Puppet? - There's an XKCD about that :-) - Users, I swear...add it to the FAQ - Buffer overflow, uh oh... - Python or Perl?  Ruby!?! -- you gotta be kidding me :-( - You don't have to forward me that email.  I've already seen it.  You don't use email encryption :-) - Would you sign my public key? - Fire up an instance in EC2 - My kernel oops'd - TCP or UDP? - There's not enough entropy on this friggin machine! - You haven't rooted your phone? - No open access points?  I see 12 running WEP.  Give me a minute...  Okay, I'm in. - Where's your public key? - Drop that in a pastebin - Okay, I have it.  What's your fingerprint? - Java or C++? - What do you think of Unity? - OpenStack or Eucalyptus? - Check StackExchange - Shit, not another core dump...

I hope you enjoy watching it as much as we enjoyed making it!

Cheers!
:-Dustin

bootmail encryption and shutdown messages now supported

I've made two pretty cool changes to the bootmail utility...

1. Bootmail now sends a message on both boot, and shutdown, using an upstart job.  Big thanks to Clint Byrum for a bit of help on that one!
2. Bootmail has always sent GPG-signed email.  But now, it will actually send GPG-encrypted email too!  All you need to do is set the RECIPIENT_KEYID variable in /etc/bootmail/gpg.conf to your GPG key id, and bootmail will send you GPG encrypted AND signed boot and shutdown messages!

Now, perhaps you wondering why, or how one would use this...

Actually, I have all of my EC2 instances set to install and use bootmail.  With this, I get an email when I start, reboot, and shutdown an instance.  I find it helps me remember what instances I have have running at any one time, by keeping the email in my Inbox (I practice Inbox Zero).

Moreover, I use cr-gpg with Gmail, so that I can read GPG encrypted email and verify GPG signatures within my Gmail web interface.  Check out this post for more information on how to set that up!

:-Dustin

ssh-import-id gaining some steam

My Google Alerts and IRC highlights have been firing almost daily with references to ssh-import-id, a handy utility I co-authored with my buddy Scott Moser a couple of years ago.

That's quite exciting to me actually, as I find the tool really, really useful, and I wish more people knew about it.  I tried in vain to contribute it to the OpenSSH project, as a complement to ssh-copy-id, but it never landed there.  Oh well.  There's rarely a day that goes by that I don't use it, actually.  I frequently use virtual machines in public clouds;  usually EC2 but not exclusively.  I often want to share that machine with a colleague.  Rather than sharing a password, I simply:

$ ssh-import-id edygarcia sergio-pena
INFO: Successfully authorized [edygarcia] 
INFO: Successfully authorized [sergio-pena]

And now, I just share the hostname or IP with Eddie and Sergio and they can SSH into this machine and authenticate using their SSH keypair.

Reviewing what actually happened...

1. ssh-import-id looped over each of the arguments on the command line, which are typically Launchpad user IDs
2. Fetched each user's public keys from https://launchpad.net/~/+sshkeys
3. Validated each key's syntax
4. And concatenated the results to the local ~/.ssh/authorized_keys file

The methodology is secure in that:

• I know what each of my colleague's Launchpad IDs are, and that's easier to remember than their SSH fingerprints
• I know that they had to authenticate with Launchpad to upload their SSH public keys
• I know that the communication between my system and Launchpad was authenticated and private as it used https with a valid SSL certificate

Note that I've uploaded a couple of minor fixes to ssh-import-id in the last 2 weeks that more accurately validates the contents of the public keys retrieved from Launchpad (thanks, Soren for one of those).

You can always grab the latest version from ppa:launchpad/ssh-import-id, though perhaps I should SRU some of these changes to Lucid/Natty/Oneiric.  Anyone willing to test and validate those SRUs, if I propose and upload them?

Cheers,
:-Dustin