OpenZFS Developer Summit Keynote: Everything Old is New Again...But Better!

On Monday this week, I was afforded the distinct privilege to deliver the opening keynote at the OpenZFS Developer Summit in San Francisco.  It was a beautiful little event, with a full day of informative presentations and lots of networking during lunch and breaks.

Below, you can view my slides, download the PDF, or watch the talk (starts at 31:10) and demo in its entirety.

Hopefully you'll enjoy the demo -- especially the most interesting raw tracing system new in the Ubuntu 16.04 LTS Linux 4.4 kernel, something called The Berkeley Packet Filter, or "BPF" for short.  I used a series of open source utilities from Brendan Gregg (from Netflix), called iovisor/bcc.  Quoting the README.md on Github:

BCC is a toolkit for creating efficient kernel tracing and manipulation programs, and includes several useful tools and examples. It makes use of extended BPF (Berkeley Packet Filters), formally known as eBPF, a new feature that was first added to Linux 3.15. Much of what BCC uses requires Linux 4.1 and above.

I'll follow up this post with another one, formally introducing BPF and how to install and use bcc in Ubuntu 16.04 LTS, if anyone is interested...

Open ZFS Keynote (public) from Dustin Kirkland

:-Dustin

Container Camp London: Streamlining HPC Workloads with Containers

A couple of weeks ago, I delivered a talk at the Container Camp UK 2016.  It was an brilliant event, on a beautiful stage at Picturehouse Central in Picadilly Circus in London.

You're welcome to view the slides or download them as a PDF, or watch my talk below.

And for the techies who want to skip the slide fluff and get their hands dirty, setup your OpenStack and LXD and start streamlining your HPC workloads using this guide.

Streamlining HPC Workloads with Containers from Dustin Kirkland

Enjoy,
:-Dustin

HOWTO: Launch an Ubuntu Cloud Image with KVM from the Command Line

I reinstalled my primary laptop (Lenovo x250) about 3 months ago (June 30, 2016), when I got a shiny new SSD, with a fresh Ubuntu 16.04 LTS image.

Just yesterday, I needed to test something in KVM.  Something that could only be tested in KVM.

kirkland@x250:~⟫ kvm
The program 'kvm' is currently not installed. You can install it by typing:
sudo apt install qemu-kvm
127 kirkland@x250:~⟫ 

I don't have KVM installed?  How is that even possible?  I used to be the maintainer of the virtualization stack in Ubuntu (kvm, qemu, libvirt, virt-manager, et al.)!  I lived and breathed virtualization on Ubuntu for years...

Alas, it seems that I've use LXD for everything these days!  It's built into every Ubuntu 16.04 LTS server, and one 'apt install lxd' away from having it on your desktop.  With ZFS, instances start in under 3 seconds.  Snapshots, live migration, an image store, a REST API, all built in.  Try it out, if you haven't, it's great!

kirkland@x250:~⟫ time lxc launch ubuntu:x
Creating supreme-parakeet
Starting supreme-parakeet
real    0m1.851s
user    0m0.008s
sys     0m0.000s
kirkland@x250:~⟫ lxc exec supreme-parakeet bash
root@supreme-parakeet:~# 

But that's enough of a LXD advertisement...back to the title of the blog post.

Here, I want to download an Ubuntu cloud image, and boot into it.  There's one extra step nowadays.  You need to create your "user data" and feed it into cloud-init.

First, create a simple text file, called "seed":

kirkland@x250:~⟫ cat seed
#cloud-config
password: passw0rd
chpasswd: { expire: False }
ssh_pwauth: True
ssh_import_id: kirkland

Now, generate a "seed.img" disk, like this:

kirkland@x250:~⟫ cloud-localds seed.img seed
kirkland@x250:~⟫ ls -halF seed.img 
-rw-rw-r-- 1 kirkland kirkland 366K Sep 20 17:12 seed.img

Next, download your image from cloud-images.ubuntu.com:

kirkland@x250:~⟫ wget http://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64-disk1.img                                                                                                                                                          
--2016-09-20 17:13:57--  http://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64-disk1.img
Resolving cloud-images.ubuntu.com (cloud-images.ubuntu.com)... 91.189.88.141, 2001:67c:1360:8001:ffff:ffff:ffff:fffe
Connecting to cloud-images.ubuntu.com (cloud-images.ubuntu.com)|91.189.88.141|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 312606720 (298M) [application/octet-stream]
Saving to: ‘xenial-server-cloudimg-amd64-disk1.img’
xenial-server-cloudimg-amd64-disk1.img 
100%[=================================] 298.12M  3.35MB/s    in 88s     
2016-09-20 17:15:25 (3.39 MB/s) - ‘xenial-server-cloudimg-amd64-disk1.img’ saved [312606720/312606720]

In the nominal case, you can now just launch KVM, and add your user data as a cdrom disk.  When it boots, you can login with "ubuntu" and "passw0rd", which we set in the seed:

kirkland@x250:~⟫ kvm -cdrom seed.img -hda xenial-server-cloudimg-amd64-disk1.img

Finally, let's enable more bells an whistles, and speed this VM up.  Let's give it all 4 CPUs, a healthy 8GB of memory, a virtio disk, and let's port forward ssh to 2222:

kirkland@x250:~⟫ kvm -m 8192 \
    -smp 4 \
    -cdrom seed.img \
    -device e1000,netdev=user.0 \
    -netdev user,id=user.0,hostfwd=tcp::5555-:22 \
    -drive file=xenial-server-cloudimg-amd64-disk1.img,if=virtio,cache=writeback,index=0

And with that, we can how ssh into the VM, with the public SSH key specified in our seed:

kirkland@x250:~⟫ ssh -p 5555 ubuntu@localhost
The authenticity of host '[localhost]:5555 ([127.0.0.1]:5555)' can't be established.
RSA key fingerprint is SHA256:w2FyU6TcZVj1WuaBA799pCE5MLShHzwio8tn8XwKSdg.
No matching host key fingerprint found in DNS.
Are you sure you want to continue connecting (yes/no)? yes

Welcome to Ubuntu 16.04.1 LTS (GNU/Linux 4.4.0-36-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

0 packages can be updated.
0 updates are security updates.

ubuntu@ubuntu:~⟫ 

Cheers,
:-Dustin