A couple of months ago, I blogged a proposal for a tool I called
dotdee.
Based on the feedback I received here, in IRC, on the Debian dpkg mailing list, and at the Ubuntu Developer Summit in Budapest, I have vastly improved the implementation and cut a 1.x release series, which is now available in the
Ubuntu Oneiric archive, and in
ppa:dotdee/ppa for all supported Ubuntu releases!
Juan Negron has already begun using
dotdee in the Ubuntu Orchestra packaging (more about Orchestra very soon!), so I thought I should probably put together a small how-to, such that you could start using
dotdee too.
The full manpage is
here. This
how-to is more of a set of sample instructions for you to try.
INTRODUCTION
You can read the initial proposal
here. Basically,
dotdee is a utility that allows you to take any flat file in your filesystem, replace it with a symlink pointing to a file that is generated from a ".d" style directory. Using
inotify, the generated file is automatically and dynamically updated any time any file in the ".d" directory is added, deleted, or modified.
The files in the ".d" are processed in alphanumeric order (per POSIX shell ordering) and can take any combination of 3 different forms:
- Flat text files -- which are simply concatenated
- Executable programs/scripts/binaries -- the current state of the generated file is passed as STDIN, and the STDOUT of the executable replaces the current state of the generated file
- Patch/diff files -- which are applied by patch against the current state of the generated file
SETUP
To begin, you need to "setup" a file for management by
dotdee. Here, we use
dotdee --setup, we pass it the file to manage,
/etc/hosts, and we optionally tell
dotdee that the "#" symbol is the comment character in this file's format.
$ ll /etc/hosts
-rw-r--r-- 1 root root 219 2011-05-02 17:31 /etc/hosts
$ sudo dotdee --setup /etc/hosts "#"
update-alternatives: using /etc/dotdee//etc/hosts to provide /etc/hosts (etc:hosts) in auto mode.
Let's see what happened...
Note that
/etc/hosts is now a symbolic link...
$ ll /etc/hosts
lrwxrwxrwx 1 root root 27 2011-06-01 11:52 /etc/hosts -> /etc/alternatives/etc:hosts
$ ll /etc/alternatives/etc:hosts
lrwxrwxrwx 1 root root 22 2011-06-01 11:52 /etc/alternatives/etc:hosts -> /etc/dotdee//etc/hosts
That eventually points to
/etc/dotdee//etc/hosts. Note that this file is read-only! This is to try and prevent inadvertent writes to this dynamically generated file.
$ ll /etc/dotdee//etc/hosts
-r--r--r-- 1 root root 353 2011-06-01 11:52 /etc/dotdee//etc/hosts
Moreover, since we told
dotdee that the comment character is "#",
dotdee added a comment to the top of the file for us.
$ cat /etc/hosts
# DO NOT EDIT THIS FILE DIRECTLY!
# Rather, add, remove, or modify file(s) in [/etc/dotdee//etc/hosts.d]
# per the dotdee(8) manpage.
127.0.0.1 localhost
127.0.1.1 x201
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
Next, let's look at what's going on in the ".d" directory. Let's see where this directory actually is...
$ sudo dotdee --dir /etc/hosts
/etc/dotdee//etc/hosts.d
$ ll /etc/dotdee/etc/hosts.d/
total 16
drwxr-xr-x 2 root root 4096 2011-06-01 11:52 ./
drwxr-xr-x 3 root root 4096 2011-06-01 11:52 ../
-rw-r--r-- 1 root root 219 2011-05-02 17:31 50-original
-rw------- 1 root root 2 2011-06-01 11:52 .comment
50-original simply contains the original contents of the managed file. And
.comment contains the comment string (#).
But now, we can start adding information in this directory and dynamically update our
/etc/hosts!
FLAT FILES
Let's append the Google DNS IP address to our hosts, and see it immediately take effect. For this, we can create simple flat file at
/etc/dotdee/etc/hosts.d/70-googledns.
$ echo "8.8.8.8 googledns" | sudo tee /etc/dotdee/etc/hosts.d/70-googledns
8.8.8.8 googledns
$ cat /etc/hosts
# DO NOT EDIT THIS FILE DIRECTLY!
# Rather, add, remove, or modify file(s) in [/etc/dotdee//etc/hosts.d]
# per the dotdee(8) manpage.
127.0.0.1 localhost
127.0.1.1 x201
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
8.8.8.8 googledns
See our new entry at the end?
EXECUTABLES
sed is a great way to programmatically modify files or standard output. Let's use a sed script to remove blank lines from our generated
/etc/hosts.
$ printf '#!/bin/sh\n sed -e "/^$/d"\n' | sudo tee /etc/dotdee/etc/hosts.d/90-noblanklines
#!/bin/sh
sed -e "/^$/d"
$ sudo chmod +x /etc/dotdee/etc/hosts.d/90-noblanklines
$ cat /etc/hosts
# DO NOT EDIT THIS FILE DIRECTLY!
# Rather, add, remove, or modify file(s) in [/etc/dotdee//etc/hosts.d]
# per the dotdee(8) manpage.
127.0.0.1 localhost
127.0.1.1 x201
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
8.8.8.8 googledns
No blank lines!
PATCH FILES
Finally,
dotdee supports quilt-like patch files. Here's a simple patch, to our current
/etc/hosts file, which can insert some data into the middle of the file:
--- /etc/hosts 2011-06-01 12:39:45.277010248 -0500
+++ /tmp/hosts 2011-06-01 12:33:58.737010336 -0500
@@ -3,6 +3,7 @@
# per the dotdee(8) manpage.
127.0.0.1 localhost
127.0.1.1 x201
+1.2.3.4 foobar
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
Let's put this content in
/etc/dotdee/etc/hosts.d/91-foobar.patch. Note that this file must not be executable, and must end in either a ".patch" or ".diff" extension.
$ sudo vi /etc/dotdee/etc/hosts.d/91-foobar.patch
# Paste the above patch, write, and quit
$ cat /etc/hosts
# DO NOT EDIT THIS FILE DIRECTLY!
# Rather, add, remove, or modify file(s) in [/etc/dotdee//etc/hosts.d]
# per the dotdee(8) manpage.
127.0.0.1 localhost
127.0.1.1 x201
1.2.3.4 foobar
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
8.8.8.8 googledns
UNDO
While testing and debugging all of this, I found it quite useful to have an "undo" function at my disposal. So after running this demo, I can safely:
$ sudo dotdee --undo /etc/hosts
update-alternatives: using /etc/dotdee//etc/hosts.d/50-original to provide /etc/hosts (etc:hosts) in auto mode.
INFO: [/etc/hosts] has been restored
INFO: You may want to manually remove [/etc/dotdee//etc/hosts /etc/dotdee//etc/hosts.d]
$ sudo rm -rf /etc/dotdee//etc/hosts /etc/dotdee//etc/hosts.d
PACKAGING
Once you're comfortable with the above, you should be well set to use
dotdee as an administrator, or as a packager. As I said above, Juan is using
dotdee in the Ubuntu Orchestra packaging now, to generate and manage a file,
/etc/puppet/manifests/site.pp.
He uses
debhelper to install a header and footer for the file.
/etc/dotdee
/etc/dotdee/etc
/etc/dotdee/etc/puppet
/etc/dotdee/etc/puppet/manifests
/etc/dotdee/etc/puppet/manifests/site.pp.d
/etc/dotdee/etc/puppet/manifests/site.pp.d/10-header
/etc/dotdee/etc/puppet/manifests/site.pp.d/90-footer
/etc/dotdee/etc/puppet/manifests/node.pp.d
/etc/dotdee/etc/puppet/manifests/node.pp.d/10-header
/etc/dotdee/etc/puppet/manifests/node.pp.d/90-footer
His
10-header looks like this:
# Globals
Exec { path => "/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin" }
# Imports
And then his
90-footer looks like this:
# fixup permissions on sudo
class sudo {
file { "/etc/sudoers":
owner => root,
group => root,
mode => 440,
}
}
import "node"
In his postinst, he sets up the file for
dotdee management, if necessary:
dotdee --dir ${PUPPET_NODE_FILE} >/dev/null 2>&1 || \dotdee --setup ${PUPPET_NODE_FILE} "#"
And now, he can insert as many puppet snippets in between the header and footer of site.pp, as necessary, from other packages!
Pretty slick, huh!?!
COMMENTS?
I'm quite interested in hearing your questions and comments!
A number of people have asked about
Augeas, and how the two projects might be similar. They are similar in that they're both tools usable by system administrators to more programmatically interface with configuration files. I think they differ quite a bit after that.
dotdee is extremely small, fast, and simple. It's not specific to configuration files, and could actually work against any file on the filesystem. It's completely agnostic to the format of the file, whereas Augeas is a library/API that must understand the particular configuration file type. I'm hoping that
dotdee will be usable by Debian/Ubuntu packagers to improve some configuration file handling, in the long run!
Enjoy,
:-Dustin
