How Encrypted-Home + eCryptfs Works

One comment to my last post, Jaunty Encrypted Home Directories, deserves a blog post of its own to answer..

gotgenes said...

Very cool. So does this mean there's now a separation of a user's password from the user's decryption password? This was an issue brought up in your previous post--oftentimes we want our encrypted data passphrase to be significantly longer than our user passphrase.

Yes. And we provided that same separation in Intrepid as well. Let me explain in better detail how this feature works...

When you login, the pam_ecryptfs module takes your login passphrase (which is usually pretty short, and easy to remember), and symmetrically decrypts the file ~/.ecryptfs/wrapped-passphrase, which will yield your mount passphrase.

Your mount passphrase is a randomly generated 128-bit key. This is very hard to remember, and rather hard to guess. 2^128 is a big number!

340,282,366,920,938,463,463,374,607,431,768,211,456
unique passphrases

This mount passphrase is salted, hashed 65536 times using sha512 (i.e. key strengthened), and loaded into your kernel keyring. In eCryptfs terms, the salted, hashed passphrase is your "file encryption key, encryption key", or fekek.

New for Jaunty (and the upstream 2.6.29 kernel), we have encrypted filenames too. In this case, the mount passphrase is salted with a different value, hashed 65536 times using sha512, and also loaded into your kernel keyring. This second key is your "filename encryption key", or fnek.

Now, every time the kernel encounters an encrypted filename in your encrypted home directory, it is decrypted using your fnek and presented to you in plaintext.

To read/write encrypted data from/to a file, your fekek is used to decrypt the "file encryption key", or fek, which is embedded in the header of the file. This embedded fek is unique per file! This means that two files, with the exact same contents, will produce two completely different encrypted files, which helps thwart known-plaintext attacks.

Does that explanation help?

:-Dustin

Jaunty Encrypted Home Directories

So this post isn't exactly "hot off the press". It's about a month late. But better late than never... Two big announcements on the Ubuntu eCryptfs front:

• Ubuntu now supports per-user encrypted home directories
• Filenames are now encrypted too
  

I have been trusting eCryptfs with my entire home directory since December, and things have been working well.

Here are some simple instructions...

Server/Alternate Installer

It's easy to setup from the server/alternate installer:


LiveCD Desktop Installer

The desktop installation is only slightly more complex. Boot the LiveCD installer, and preseed a special value:

• Select your language
• Press F6
• Then ESC
• Add "user-setup/encrypt-home=true" just before the "--".

You will see a new option on the user-details page of the installer:


Post-installation, on a Running System

If you have a running Jaunty system, and you want to add another user, you can easily add a new user and have their home directory encrypted, with:

$ sudo adduser --encrypt-home foo_user

Important Caveats!

1. You really must record your randomly generated mount passphrase after the installation. This is easy to do with:
   $ ecryptfs-unwrap-passphrase ~/.ecryptfs/wrapped-passphrase
2. Swap space. Decrypted copies of your files could easy leak to your swap space. I strongly recommended that either:
   
   • You do not use swap (I have 4GB memory and don't really need it)
   • Or your encrypt your swap with:
     $ sudo ecryptfs-setup-swap
   In either case, however, you will not be able to hibernate your system (but suspend will continue to work just fine). It is for this reason that the option is hidden in the default installation. We're trying to fix the swap issues for Karmic.
3. Auto-login and encrypted-home are simply incompatible. You must enter a password to decrypt your home directory, so automatic login is not possible. However, if you want to automatically login to your desktop, you can actually use the encrypted-private feature, and store a subset of your data in ~/Private. After installation, you can configure this with:
   $ ecryptfs-setup-private

Migration of Existing Data to an Encrypted Home Directory

We won't be able to provide an automated mechanism for live migration of data into your encrypted home directory in time for Jaunty. (Sorry, more pressing Ubuntu Server work took precedence...) I will provide some step-by-step instructions (and maybe a script?) here in my blog--stay tuned!

:-Dustin

Improving Manpage Content

For those looking to contribute to Ubuntu (and in general, to Linux)
documentation, I'm pleased to announce a new vector.

I have previously posted about the Ubuntu Manpage Repository:

• http://manpages.ubuntu.com/

Earlier this week, I rolled out a new feature, whereby each page has a tiny "bug" icon near the top. Such as:

• http://manpages.ubuntu.com/manpages/jaunty/en/man7/ecryptfs.7.html

Clicking on the bug icon will bring you to a Launchpad page where you can report a bug against the package providing incorrect manpage content. Please file these bugs with:

• Importance: Wishlist
• Tags: manpage
  

For example:

• https://bugs.launchpad.net/ubuntu/+source/ecryptfs-utils/+bug/329491

Note also that you can download the source of any given manpage directly from the manpage-repository webpage. See the link to ecryptfs.7.gz:

• http://manpages.ubuntu.com/manpages.gz/jaunty/man7/ecryptfs.7.gz

For information about generating a patch, see:

• https://wiki.ubuntu.com/UbuntuDevelopment/Patches

From the manpage source, you can correct the manpage documentation, generate a patch, and attach it to the Launchpad bug. If you do this, make sure you:

1. flag the attachment as a patch
2. mark the bug as 'In Progress'
3. assign the bug to yourself
4. subscribe the appropriate team for uploading
   
   • ubuntu-main-sponsors
   • ubuntu-universe-sponsors

And in the interest of being a good Open Source Citizen, please also forward your patch to the appropriate upstream project (and/or Debian)!

:-Dustin

Ubuntu Server: Suspend/Hibernate/Resume Call for Testing!

Late last week, Mark announced the next Ubuntu release, Karmic Koala. Mark noted that:

the best way to conserve energy is to go to sleep,
and these days even servers can suspend and resume,
so imagine if we could make it possible to build
a cloud computing facility that drops its energy use
virtually to zero

In December of 2008, I asked what you thought about suspending and hibernating Ubuntu Servers. While the "poll" wasn't unanimous, the overwhelming majority of you thought it might actually be useful in some situations.

It's not too early to start looking at this feature for Karmic. So I spent some time last week testing this, and I must say that I'm impressed with the very preliminary results. I was able to install the Ubuntu Jaunty Server on my hardware, both suspend and hibernate the system from the command line, and then remotely resume the system using wake-on-lan.

Impressively, my test system's power consumption is reduced by ~80% while in the suspended state, and can wake in ~5 seconds!

My test hardware is actually a desktop kit system (Asus P1-AH2, AMD x2 5800, 4GB, nVidia), running the Jaunty amd64 server.

At this point, I'm quite interested in gathering any feedback from the Ubuntu Server community on their experience suspending/hibernating/resuming the Jaunty Server on whatever hardware they might have. Here are some detailed instructions...

Server BIOS Setup

Check your BIOS for any relevant settings:

• Make sure that wake-on-lan is enabled. This setting takes a variety of names in different BIOS implementations.
  
• Make sure that low-power states are enabled (my bios had a setting for "S1 & S3" power states). Again, this may be labeled differently, but should be something along the lines of "Advanced Power Management".
  

Server OS Setup

Install Ubuntu Jaunty Server (i386 or amd64).

Install pm-utils and ethtool, both of which are now on the Jaunty Server CD.

• sudo apt-get install pm-utils ethtool

Configure your network interface to wake-on-lan.

• sudo ethtool -s eth0 wol g

I want this setting persistent across reboots, so I put this in /etc/rc.local.

Obtain the MAC address of your server, such that you can send the wake-on-lan magic packet from another system.

• ifconfig eth0 | head -n1 | sed 's/^.*HWaddr //'

First, let ensure that wake-on-lan is working. Power this system down.

• sudo poweroff

From another system, install the wakeonlan package.

• sudo apt-get install wakeonlan

And send the magic packet to the MAC address you obtained earlier.

• wakeonlan 00:11:22:33:44:55

If your server powers itself on, congratulations, wake-on-lan is working. If not, you need to get this working before you proceed further.

Once the server is back up, test suspend. Launch a program to run in the background, to ensure that the resume restored this state. And then run the pm-suspend command

• screen top
• (new window in screen-profiles)
  
• sudo pm-suspend

Make sure that the system enters a lower power state. On my machine, the hard drives stop spinning, as well as the cdrom drive, cpu fans, case fans, etc. I tested resume twice, once by hitting the power button, and once by sending the wakeonlan signal.

• wakeonlan 00:11:22:33:44:55

Then, test hibernate in a similar manner.

• pm-hibernate

And

• wakeonlan 00:11:22:33:44:55

Make sure that your context was saved, and the system is still running 'screen top' or whatever you chose to run.

Automated Testing

Next, I ran the automated test-suspend script developed by Andy Whitcroft on the Ubuntu Kernel team. I sent him a minor patch that tailored it a bit better for server testing. For details, see:
https://wiki.ubuntu.com/KernelTeam/SuspendResumeTesting

• wget http://people.ubuntu.com/~apw/suspend-resume/test-suspend

• sudo bash test-suspend --full --server

And I recorded my results at:
https://wiki.ubuntu.com/KernelTeam/SuspendResumeTesting/Feedback/Server
noting that this machine (Asus P1-AH2) was running the Ubuntu Server.

Do you have a server running Jaunty? Would you care to share your results?


:-Dustin

Server Migration From Ubuntu 8.04 To Debian 5.0?

I read planet.ubuntu.com every day, through my RSS reader. And I follow Aaron Toponce's blog posts regularly.

But his post today, Server Migration From Ubuntu 8.04 To Debian 5.0, makes me wonder..."wait a minute, huh?"

I should state up front that I'm a biased reader. I am employed by Canonical, on the Ubuntu Server Team. My daily job is to make the Ubuntu Server a rock-solid, secure, feature-ful, performant, enterprise-class operating system. So I can't even pretend that I'm not biased.

I do feel it's necessary, though, to challenge Aaron's opinion as stated in this blog entry. I very much believe that Ubuntu 8.04.2 LTS (aka Hardy Heron) is every bit as reliable and secure as the Debian Lenny release.

Aaron writes:

Another nice thing with Debian stable, is it releases when it’s ready. The Debian community has taken some flack for this, with 2-3 years at times between releases. However, Debian stable is the operating system that is high production quality. While most end users tend to run testing or unstable on their desktops or laptops, many prefer stable for their production server.

This is what the Ubuntu LTS release is. These are only released every 2 years, as opposed to our non-LTS releases which happen every 6 months. The LTS development cycle is handled differently than other development cycles. Many features are bypassed in the interest of additional testing and bug fixing. Packages contained in an LTS release are supported for security updates for 5+ years; thus, we are more conservative about what is acceptable in an LTS release.

Ubuntu LTS is absolutely a stable and secure server operating system, ideally suited for production purposes. My production website (www.divitup.com) is running Ubuntu LTS 8.04.2 Server and will continue to do so until Ubuntu LTS 10.04 is released.

Aaron goes on to write:

Now, with that said, I personally have never had any problems with my LTS server, either Dapper 6.06 or Hardy 8.04. But do I want to risk it? Should I chance it? While nothing may ever happen that causes critical concern for me with an LTS release, I feel more comfortable putting my trust in Debian stable than I do Ubuntu LTS.

Multiple years running Ubuntu LTS releases, and no problems. That's great! Me too.

So why spread Fear, Uncertainty and Doubt (tm) about the Ubuntu LTS Server on the planet.ubuntu.com aggregator, without concrete examples? That seems counterproductive. I don't get it...

I do not mean to attack Aaron personally. He's not the first person to tell me that they choose to run Debian on their server rather than Ubuntu. That's certainly their (your) right, and I'm all for choice.

If you know that there are concrete examples of things that we could do better on the Ubuntu Server, specific areas of improvement, particular cases where the Ubuntu Server did not meet your needs, by all means, we want this information! We're honestly trying to build the best enterprise class Linux server operating system on the market. We need to know from you where there is room for improvement.

:-Dustin

A Very Good Year

A year ago today, I joined Canonical to work on the Ubuntu Server.

I have been a professional software engineer since May of 2000, and this has easily been the best year of my working, professional career.

And a sincere thank you to Canonical, my team, my manager Rick, and to Mark for this opportunity. And here's to many more excellent years to come...

Although my team is dispersed across 9 time zones (UTC+1 .. UTC-8), we have spent more time together than some co-workers do in the same building. I have hosted colleagues at my home for meetings, weekends, holidays, and dinner. And we have traveled the world (Boston, Prague, London, Montreal, Paris, San Francisco, Berlin) for sprints, conferences, and meetings.

I joined a bit too late in the Hardy development cycle to drive any particular features, but I did learn many Ubuntu processes, Debian policies and packaging, and did fix a number of bugs. In my spare time, I managed to create the Ubuntu Developer Documentation Search, as well as the Ubuntu Manpage Repository. During the Intrepid cycle, I owned a couple of development items, including booting-degraded-RAID, encrypted-Private-directories. And thus far in Jaunty, I've managed to support encrypted-Home-directories (with encrypted filenames), screen-profiles, screenbin, and kvm.

I would like to express my appreciation to the Ubuntu Community for your acceptance of me as an Ubuntu Member, MOTU, and Core Dev. This is truly an amazing community of people, who's accomplishment (the ongoing release and support of the Ubuntu distribution and its derivatives) is nothing short of amazing. I have no doubt that we are changing the landscape of computing in the 21st century--from the smallest embedded devices, to netbooks, notebooks, desktops, servers, and the Cloud.

:-Dustin