AWSnap! Snappy Ubuntu Now Available on AWS!

Awww snap!

That's right!  Snappy Ubuntu images are now on AWS, for your EC2 computing pleasure.

Enjoy this screencast as we start a Snappy Ubuntu instance in AWS, and install the xkcd-webserver package.


And a transcript of the commands follows below.

kirkland@x230:/tmp⟫ cat cloud.cfg
#cloud-config
    snappy:
       ssh_enabled: True
kirkland@x230:/tmp⟫ aws ec2 describe-images \
> --region us-east-1 \
> --image-ids ami-5c442634

{
    "Images": [
        {
            "ImageType": "machine",
            "Description": "ubuntu-core-devel-1418912739-141-amd64",
            "Hypervisor": "xen",
            "ImageLocation": "ucore-images/ubuntu-core-devel-1418912739-141-amd64.manifest.xml",
            "SriovNetSupport": "simple",
            "ImageId": "ami-5c442634",
            "RootDeviceType": "instance-store",
            "Architecture": "x86_64",
            "BlockDeviceMappings": [],
            "State": "available",
            "VirtualizationType": "hvm",
            "Name": "ubuntu-core-devel-1418912739-141-amd64",
            "OwnerId": "649108100275",
            "Public": false
        }
    ]
}
kirkland@x230:/tmp⟫
kirkland@x230:/tmp⟫ # NOTE: This AMI will almost certainly have changed by the time you're watching this ;-)
kirkland@x230:/tmp⟫ clear
kirkland@x230:/tmp⟫ aws ec2 run-instances \
> --region us-east-1 \
> --image-id ami-5c442634 \
> --key-name id_rsa \
> --instance-type m3.medium \
> --user-data "$(cat cloud.cfg)"
{
    "ReservationId": "r-c6811e28",
    "Groups": [
        {
            "GroupName": "default",
            "GroupId": "sg-d5d135bc"
        }
    ],
    "OwnerId": "357813986684",
    "Instances": [
        {
            "KeyName": "id_rsa",
            "PublicDnsName": null,
            "ProductCodes": [],
            "StateTransitionReason": null,
            "LaunchTime": "2014-12-18T17:29:07.000Z",
            "Monitoring": {
                "State": "disabled"
            },
            "ClientToken": null,
            "StateReason": {
                "Message": "pending",
                "Code": "pending"
            },
            "RootDeviceType": "instance-store",
            "Architecture": "x86_64",
            "PrivateDnsName": null,
            "ImageId": "ami-5c442634",
            "BlockDeviceMappings": [],
            "Placement": {
                "GroupName": null,
                "AvailabilityZone": "us-east-1e",
                "Tenancy": "default"
            },
            "AmiLaunchIndex": 0,
            "VirtualizationType": "hvm",
            "NetworkInterfaces": [],
            "SecurityGroups": [
                {
                    "GroupName": "default",
                    "GroupId": "sg-d5d135bc"
                }
            ],
            "State": {
                "Name": "pending",
                "Code": 0
            },
            "Hypervisor": "xen",
            "InstanceId": "i-af43de51",
            "InstanceType": "m3.medium",
            "EbsOptimized": false
        }
    ]
}
kirkland@x230:/tmp⟫
kirkland@x230:/tmp⟫ aws ec2 describe-instances --region us-east-1 | grep PublicIpAddress
                    "PublicIpAddress": "54.145.196.209",
kirkland@x230:/tmp⟫ ssh -i ~/.ssh/id_rsa ubuntu@54.145.196.209
ssh: connect to host 54.145.196.209 port 22: Connection refused
255 kirkland@x230:/tmp⟫ ssh -i ~/.ssh/id_rsa ubuntu@54.145.196.209
The authenticity of host '54.145.196.209 (54.145.196.209)' can't be established.
RSA key fingerprint is 91:91:6e:0a:54:a5:07:b9:79:30:5b:61:d4:a8:ce:6f.
No matching host key fingerprint found in DNS.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '54.145.196.209' (RSA) to the list of known hosts.
Welcome to Ubuntu Vivid Vervet (development branch) (GNU/Linux 3.16.0-25-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

Welcome to the Ubuntu Core rolling development release.

 * See https://ubuntu.com/snappy

It's a brave new world here in snappy Ubuntu Core! This machine
does not use apt-get or deb packages. Please see 'snappy --help'
for app installation and transactional updates.

To run a command as administrator (user "root"), use "sudo ".
See "man sudo_root" for details.

ubuntu@ip-10-153-149-47:~$ mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,relatime,size=1923976k,nr_inodes=480994,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=385432k,mode=755)
/dev/xvda1 on / type ext4 (ro,relatime,data=ordered)
/dev/xvda3 on /writable type ext4 (rw,relatime,discard,data=ordered)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,mode=755)
tmpfs on /etc/fstab type tmpfs (rw,nosuid,noexec,relatime,mode=755)
/dev/xvda3 on /etc/systemd/system type ext4 (rw,relatime,discard,data=ordered)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset,clone_children)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
tmpfs on /etc/machine-id type tmpfs (ro,relatime,size=385432k,mode=755)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=300,minproto=5,maxproto=5,direct)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
mqueue on /dev/mqueue type mqueue (rw,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
/dev/xvda3 on /etc/hosts type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /etc/sudoers.d type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /root type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /var/lib/click/frameworks type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /usr/share/click/frameworks type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /var/lib/systemd/snappy type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /var/lib/systemd/click type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /var/lib/initramfs-tools type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /etc/writable type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /etc/ssh type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /var/tmp type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /var/lib/apparmor type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /var/cache/apparmor type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /etc/apparmor.d/cache type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /etc/ufw type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /var/log type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /var/lib/system-image type ext4 (rw,relatime,discard,data=ordered)
tmpfs on /var/lib/sudo type tmpfs (rw,relatime,mode=700)
/dev/xvda3 on /var/lib/logrotate type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /var/lib/dhcp type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /var/lib/dbus type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /var/lib/cloud type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /var/lib/apps type ext4 (rw,relatime,discard,data=ordered)
tmpfs on /mnt type tmpfs (rw,relatime)
tmpfs on /tmp type tmpfs (rw,relatime)
/dev/xvda3 on /apps type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda3 on /home type ext4 (rw,relatime,discard,data=ordered)
/dev/xvdb on /mnt type ext3 (rw,relatime,data=ordered)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=385432k,mode=700,uid=1000,gid=1000)
ubuntu@ip-10-153-149-47:~$ mount | grep " / "
/dev/xvda1 on / type ext4 (ro,relatime,data=ordered)
ubuntu@ip-10-153-149-47:~$ sudo touch /foo
touch: cannot touch ‘/foo’: Read-only file system
ubuntu@ip-10-153-149-47:~$ sudo apt-get update
Ubuntu Core does not use apt-get, see 'snappy --help'!
ubuntu@ip-10-153-149-47:~$ sudo snappy --help
Usage:snappy [-h] [-v]
             {info,versions,search,update-versions,update,rollback,install,uninstall,tags,build,chroot,framework,fake-version,nap}
             ...

snappy command line interface

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         Print this version string and exit

Commands:
  {info,versions,search,update-versions,update,rollback,install,uninstall,tags,build,chroot,framework,fake-version,nap}
    info
    versions
    search
    update-versions
    update
    rollback            undo last system-image update.
    install
    uninstall
    tags
    build
    chroot
    framework
    fake-version        ==SUPPRESS==
    nap                 ==SUPPRESS==
ubuntu@ip-10-153-149-47:~$ sudo snappy info
release: ubuntu-core/devel
frameworks:
apps:
ubuntu@ip-10-153-149-47:~$ sudo snappy versions -a
Part         Tag   Installed  Available  Fingerprint     Active
ubuntu-core  edge  141        -          7f068cb4fa876c  *
ubuntu@ip-10-153-149-47:~$ sudo snappy search docker
Part    Version    Description
docker  1.3.2.007  The docker app deployment mechanism
ubuntu@ip-10-153-149-47:~$ sudo snappy install docker
docker      4 MB     [=============================================================================================================]    OK
Part    Tag   Installed  Available  Fingerprint     Active
docker  edge  1.3.2.007  -          b1f2f85e77adab  *
ubuntu@ip-10-153-149-47:~$ sudo snappy versions -a
Part         Tag   Installed  Available  Fingerprint     Active
ubuntu-core  edge  141        -          7f068cb4fa876c  *
docker       edge  1.3.2.007  -          b1f2f85e77adab  *
ubuntu@ip-10-153-149-47:~$ sudo snappy search webserver
Part                  Version  Description
go-example-webserver  1.0.1    Minimal Golang webserver for snappy
xkcd-webserver        0.3.1    Show random XKCD compic via a build-in webserver
ubuntu@ip-10-153-149-47:~$ sudo snappy install xkcd-webserver
xkcd-webserver     21 kB     [=====================================================================================================]    OK
Part            Tag   Installed  Available  Fingerprint     Active
xkcd-webserver  edge  0.3.1      -          3a9152b8bff494  *
ubuntu@ip-10-153-149-47:~$ exit
logout
Connection to 54.145.196.209 closed.
kirkland@x230:/tmp⟫ ec2-instances
i-af43de51 ec2-54-145-196-209.compute-1.amazonaws.com
kirkland@x230:/tmp⟫ ec2-terminate-instances i-af43de51
INSTANCE        i-af43de51      running shutting-down
kirkland@x230:/tmp⟫

Cheers!
Dustin

Hollywood Technodrama -- There's an App for that!

Wargames.  Hackers.  Swordfish.  Superman 3.  Jurassic Park.  GoldenEye.  The Matrix.

You've all seen the high stakes hacking scene, packed with techno-babble and dripping in drama.  And the command and control center with dozens of over-sized monitors, overloaded with scrolling text...

I was stuck on a plane a few weeks back, traveling home from Las Vegas, and the in flight WiFi was down.  I know, I know.  Real world problems.  Suddenly, I had 2 hours on my hands, without access to email, IRC, or any other distractions.

It's at this point I turned to my folder of unfinished ideas, and cherry-picked one that would take just a couple of fun hours to hack.  And I'm pleased to introduce the fruits of that, um, labor -- the hollywood package for Ubuntu :-)  Call it an early Christmas present!  All code is on both Launchpad and Github.


If you're already running Vivid (Ubuntu 15.04) -- I salute you! -- and you can simply:

sudo apt-get install hollywood

If you're on any other version of Ubuntu, you'll need to:

sudo apt-add-repository ppa:hollywood/ppa
sudo apt-get update
sudo apt-get install hollywood

Fire up a terminal, maximize it, open byobu, and run the hollywood command.  Then sit back and soak into the trance...

I recently jumped on the vertical monitor bandwagon, for my secondary display.  It's fantastic for reading and writing code.  It's also hollywood-worthy ;-)


How does all of this work?

For starters, it's all running in a Byobu (tmux) session, which enables us to split a single shell console into a bunch of "panes" or "splits".

The hollywood package depends on a handful of utilities that I found (mostly apt-cache searching the Ubuntu archives for monitors and utilities).  You can find a handful of scripts in /usr/lib/hollywood/.  Each of these is a "driver" for a widget that might run in one of these splits.  And ccze is magical, accepting input on stdin and colorizing the text.

In fact, they're quite easy to write :-)  I'm happy to accept contributions of new driver widgets, as long as you follow a couple of simple rules.  Each widget:

• Must run as a regular, non-root user
• Must not eat all available CPU, Disk, or Memory
• Must not write data
• Must run indefinitely, until receiving a Ctrl-C
• Must look hollywood cool!

So far, we have widgets that: generate passphrases encoded in NATO phonetic, monitor and render network bandwidth, emulate The Matrix, find and display, with syntax highlighting, source code on the system, show a bunch of error codes, hexdump a bunch of binaries, monitor some processes, render some images to ASCII art, colorize some log files, open random manpages, generate SSH keys and show their random art, stat a bunch of inodes in /proc and /sys and /dev, and show the tree output of some directories.

I also grabbed a copy of the Mission Impossible theme song, licensed under the Creative Commons.  I played it in the Totem music player in Ubuntu, with the Monoscope visual effect, and recorded a screencast with gtk-recordmydesktop.  I then mixed the output .ogv file, with the original .mp3 file, and transcoded it to mp4/h264/aac, reducing the audio bitrate to 64k and frame size to 128x96, using this command:

avconv -i missionimpossible.ogv -i MissionImpossibleTheme.mp3 -s 128x96 -b 64k -vcodec libx264 -acodec aac -f mpegts -strict experimental -y mi.mp4

Then, hollywood plays it in one of the splits with mplayer's ascii art video output on the console :-)

DISPLAY= mplayer -vo caca /usr/share/hollywood/mi.mp4

Sound totally cheesy?  Why, yes, it is :-)  That's the point :-)

Oh, and by the way...  If you ever sit down at someone else's Linux PC, and want to freak them out a little, just type:

ubuntu@x230:~⟫ PS1="root@$(hostname):~# "; clear 
root@x230:~# 



And then have fun!

That latter "hack", as well as the entire concept of hollywood is inspired in part by Kees Cook's awesome talk, in particular his "Useless Hollywood Drama Mode" in his exploit demo.

Happy hacking!

:-Dustin

A Snappy Ubuntu Walkthrough on Google Compute Engine

As promised last week, we're now proud to introduce Ubuntu Snappy images on another of our public cloud partners -- Google Compute Engine. In the video below, you can join us walking through the instructions we have published here. Snap it up! :-Dustin

It's a Snap!

A couple of months ago, I re-introduced an old friend -- Ubuntu JeOS (Just enough OS) -- the smallest, (merely 63MB compressed!) functional OS image that we can still call “Ubuntu”.  In fact, we call it Ubuntu Core.

That post was a prelude to something we’ve been actively developing at Canonical for most of 2014 -- Snappy Ubuntu Core!  Snappy Ubuntu combines the best of the ground-breaking image-based Ubuntu remix known as Ubuntu Touch for phones and tablets with the base Ubuntu server operating system trusted by millions of instances in the cloud.

Snappy introduces transactional updates and atomic, image based workflows -- old ideas implemented in databases for decades -- adapted to Ubuntu cloud and server ecosystems for the emerging cloud design patterns known as microservice architectures.

The underlying, base operating system is a very lean Ubuntu Core installation, running on a read-only system partition, much like your iOS, Android, or Ubuntu phone.  One or more “frameworks” can be installed through the snappy command, which is an adaptation of the click packaging system we developed for the Ubuntu Phone.  Perhaps the best sample framework is Docker.  Applications are also packaged and installed using snappy, but apps run within frameworks.  This means that any of the thousands of Docker images available in DockerHub are trivially installable as snap packages, running on the Docker framework in Snappy Ubuntu.

Take Snappy for a Drive

You can try Snappy for yourself in minutes!

You can download Snappy and launch it in a local virtual machine like this:

$ wget http://cdimage.ubuntu.com/ubuntu-core/preview/ubuntu-core-alpha-01.img
$ kvm -m 512 -redir :2222::22 -redir :4443::443 ubuntu-core-alpha-01.img

Then, SSH into it with password 'ubuntu':

$ ssh -p 2222 ubuntu@localhost

At this point, you might want to poke around the system.  Take a look at the mount points, and perhaps try to touch or modify some files.

$ sudo rm /sbin/init
rm: cannot remove ‘/sbin/init’: Permission denied
$ sudo touch /foo

touch: cannot touch ‘foo’: Permission denied
$ apt-get install docker
apt-get: command not found

Rather, let's have a look at the new snappy package manager:

$ sudo snappy --help

And now, let’s install the Docker framework:

$ sudo snappy install docker

At this point, we can do essentially anything available in the Docker ecosystem!

Now, we’ve created some sample Snappy apps using existing Docker containers.  For one example, let’s now install OwnCloud:

$ sudo snappy install owncloud

This will take a little while to install, but eventually, you can point a browser at your own private OwnCloud image, running within a Docker container, on your brand new Ubuntu Snappy system.

We can also update the entire system with a simple command and a reboot:

$ sudo snappy versions

$ sudo snappy update
$ sudo reboot

And we can rollback to the previous version!

$ sudo snappy rollback
$ sudo reboot

Here's a short screencast of all of the above...

While the downloadable image is available for your local testing today, you will very soon be able to launch Snappy Ubuntu instances in your favorite public (Azure, GCE, AWS) and private clouds (OpenStack).

Enjoy!
Dustin