From the Canyon Edge -- :-Dustin

Monday, May 21, 2012

Introducing zEscrow -- or, How to save your encrypted life!

I had the honor of introducing zEscrow about a week ago, at the Ubuntu Developer Summit during Friday's plenary of lightning talks.  You can also view my slides now!

zEscrow is a free service offered by my employer, Gazzang, to users of Ubuntu's Encrypted Home Directory, to aid them in safely backing up and retrieving the bit of configuration and key material necessary to recover that data later.  I can't state this emphatically enough...

This very well may
save your encrypted life at some point!

The Quick Start Guide

If you're running a version of prior to Ubuntu 12.04 LTS, first add the PPA:

  sudo apt-add-repository ppa:zescrow/ppa
  sudo apt-get install zescrow

And if you're on Ubuntu 12.04 LTS, just install.

  sudo apt-get install zescrow-client

Now, just run zescrow, and follow the three simple prompts:

  1. Choose your server
  2. Enter your login password
  3. Visit the one-time URL

How it Works

Some inquiring minds might want to know the nitty gritty details.  You're welcome to read the code, as Gazzang has released both the client and server as free and open source code in Launchpad under the AGPL.  Here's a narrative pseudocode of the algorithm though:
  1. Choose your zEscrow server.  I recommend that you use the default,
  2. The zescrow utility will download the public GPG key associated with your zEscrow server and load it into a temporary keyring stored entirely in memory.
  3. Enter your LOGIN password.  This will be used to decrypt your ~/.ecryptfs/wrapped-passphrase file.  Under NO circumstances will your LOGIN password will sent to the remote server!!!
  4. The utility will create a tar archive of your entire ~/.ecryptfs directory, but replacing your wrapped-passphrase file, with unwrapped-passphrase.  This protects your LOGIN passphrase from ever leaving your system, but ensures that your randomly generated MOUNT passphrase will be securely transferred to the remote server
  5. This ecryptfs.tar archive is securely transmitted to the zEscrow server over SSL.
  6. Upon a successful transmission to the zEscrow server, a cryptographically nonced URL link is sent back to the client utility, which embeds a checksum of the transmitted archive, verifying the integrity of the transmission.
  7. You MUST complete the transaction by opening the link IMMEDIATELY, to "claim" this upload as yours.  Upon doing so, you'll be required to login using Google OpenID.  
    • (Yes, you must have a Google OpenID to use this service.  Sorry.  Send a patch, if you want support for another OpenID provider).
  8. That's it!  You can now download your backups from at any time, and use ecryptfs-recover-private to get your data back, following these instructions!

The Motivation

This might help explain why I have personally received hundreds (probably climbing north of a thousand) emails, IRC messages, forum posts, StackExchange questions, Launchpad bugs, SMS messages and even phone calls to my cell phone (!?!) from users who have forgotten their login password, or did not record their randomly generated eCryptfs mount password at installation, and are now cryptographically locked out of their own data :-(

Unhappy Users Don't Back Up their eCryptfs Passphrase

A few random quotes from the last 2 months alone:
  • "Through idiocracy I have screwed up my encrypted home directory and if possible I need help getting it back."
  • "I was trying to mount my encrypted home directory from a livecd in order to back up my data (according to the instructions), when I accidentally deleted one of the .ecryptfs folders in my encrypted home."
  • "Mr Kirkland, my name is MB. I used an Ubuntu system with ecryptfs. Something happened and it all went up in smoke. I saved a backup and moved on. Chalked it up to bad backup practices and moved on. I found the encrypted backup a few days ago, and I've been trying to unscrew it. I *think* I found the old wrapped-passphrase file, and I tried to fix it. So far, I've been unable".
  • "Please help as I am stuck in Korea and will be totally shagged without my e-mail and data. I have 6 months un-backedup work on the disk, of course. And I saved the password for the disk on my home partition...great move eh?"
I can't even respond to most of these emails, if it's clear that the user hasn't backed up their random, mount passphrase.  These are usually 16 or 32 characters of hexadecimal [0-9a-f], representing 128-bits or 256-bits of entropy.  You're doing battle with a mathematical Highlander at this point...  There can be only one, and the chances are absolutely astronomical that it won't be you :-(

But Happy Users Do Back Up their eCryptfs Passphrase!

On the other hand, I have helped hundreds upon hundreds of users recover their data, when its clear that they HAVE backed up their randomly generated MOUNT passphrase.  These two blog post of mine, about the ecryptfs-recover-private utility and how to mount your encrypted home from a live CD, are my two all-time most viewed posts.  A few quotes from happy users:
  • "you saved my life, thank you!"
  • "Where do I send hugs? It's great, thanks so much! I just want to add my note"
  • "Worked like a charm - thanks."
  • "Thanks $deity and Dustin, this method works for recover my encrypted private directory and backup it to external drive. Thanks again for this tutorial."
  • "Thanks Man!! it worked for me!!"
  • "Today, making a liveCD and following your instructions above put a massive smile on my face. I can't believe I've now got access to everything again and nothing is lost. Thank you so much for sharing your knowledge - I shall sleep well tonight!"
  • "Thank you for this addition to Natty! I was having a hard time mounting my files on a system I wrecked ;)"
  • "thank's a lot, u'r save my life"
  • "My god. Thank you so much! I tried to upgrade to 11.04, and it wrecked my OS. This is a lifesaver."
  • "This is cake my friend nice job! I remember when this was stuff was hard. I've been trying to recover a drive for some time now."
  • "I just wanted to say thanks for building this. I used it to recover a ~/.Private directory on an external drive, and it worked flawlessly. It's folks like yourself building tools like this that makes open source projects such a pleasure to use. So kudos, and thanks."
If you use the free zEscrow service from Gazzang, in conjunction with Ubuntu's Encrypted Home Directory, and the ecryptfs-recover-private utility, you'll almost certainly be counted in the "Happy Users".  And if not...well, you're a bit on your own!  Please, please, please write down your passphrase and store it in a very safe, very private place!!!



  1. This comment has been removed by the author.

  2. I tried installing zescrow in 12.04 without success.
    sudo apt-get install zescrow-client
    does seem to work however.
    Nice idea, thank-you for implementing such a useful utility.

    1. Ah, thanks! Fixed the typo. It should be 'sudo apt-get install zescrow-client', as you say!


  3. Jef,

    Thanks for the information. I have removed that paragraph entirely because it, along with this thread, distracts from the real message of this post -- consider using zEscrow to backup your eCryptfs keys.


  4. I've been using encryptfs for several releases now. Recently I've found out that there is a several-years running problem with it that is not being looked at: . I don't think that my files should be getting randomly corrupted without any tools to monitor this or let me know - something a common user on Ubuntu without commercial support would've unlikely realized and found out (solution to this involved looking at the system logs, googling, finding one right comment and deleting the corrupted files in question).

    So at this point I'm a bit dubious about this technology. I've tried this escrow and it does not work:

    Your LOGIN passphrase:
    ERROR: Failed to upload to server
    ERROR: Beware, this escrow did NOT succeed!

    I don't know did it fail, it doesn't tell me. I effectively can't take any steps to resolve the problem...

  5. I know you've been planning on deploying a key escrow service for a while. It's good to see your efforts come to fruition.

    The Microsoft Windows BitLocker Full Volume Encryption wizard refuses to initiate conversion until the user backs up the recovery password by printing it out or saving it to USB. In enterprise environments managed via an Active Directory, administrators can configure clients to require seamless transfer of the recovery keys to the domain controller prior to conversion. The GUI tools that you can get to easily via CPL and context menus in the shell make it as difficult as possible for the user to lose his data.

    There's something to be said for a heavy-handed approach of that sort. If the user wants to use the "power tools" to do the encryption, then fine -- don't get in the way. But if the user is running the standard "point and click" tools, then don't budge until the user has been forced to take some kind of action to back up his recovery key.

    Ubuntu devs: Integrate zEscrow into the standard eCryptfs deployment scenario. Supply your own servers too, and give enterprise users an easy way to set up their own in-house servers. Force its use. It's tough love. Kinda like UAC. They'll thank you for it later.

  6. Can you give some insight about the privacy concerned people, what exactly is backed up and maybe for the extra cautious ones how we can do it ourselves?

  7. This looks great! Integrating zEscrow into the standard eCryptfs would be great!

    Are there plans to implementing login with a Launchpad OpenID?


Please do not use blog comments for support requests! Blog comments do not scale well to this effect.

Instead, please use Launchpad for Bugs and StackExchange for Questions.