From the Canyon Edge -- :-Dustin

Friday, April 29, 2011

dotdee: A modern proposal for improving Debian Conffile Management


Debian's management of conffiles is a truly inspired approach to a very difficult topic in any Linux/UNIX distribution.

Conffiles are files in /etc that the Debian package management system (dpkg) won't automatically overwrite when you upgrade a package.  Local modifications by the system administrator are preserved.  This a critical policy expectation by Debian and Ubuntu system administrators, which ensures the proper in-place upgrade of packages on a running system.  Moreover, handling conffiles correctly, as a package maintainer according to Debian policy, requires considerable expertise.  Debian Developer RaphaĆ«l Hertzog has one of the best, most concise explanations of how dpkg handles conffiles in this blog post.


Having read in detail several times now the Debian policies on conffile management, I see some room for improvement, with respect to modern Debian and Ubuntu deployments.  I believe there are two key points that the current policy sufficient handle...
  1. Particularly in modern, massive Debian/Ubuntu deployments (think Cloud, Grid, and Cluster computing), it's no longer humans that are managing individual systems, but rather systems (puppet, chef, cfengine, et al.) managing systems.  (Insert your Skynet jokes here...)  As such, it's difficult (if not impossible) for a Debian package or a distribution to make configuration changes to another package's conffiles without violating Debian policy -- even when the change might objectively be the "right" or the "best" thing to do, in terms of end user experience and package operation (especially when the given system is only ever managed by a configuration management system).
  2. In other cases, one local package has a run-time dependency on a second package on the local system, but requires the package it depends on to be configured in a particular way.  Again, if that configuration lives in a conffile owned by the second package, the first package cannot automatically make that configuration change without violating said policy.
As an exception, rather than the rule, there are a couple of very mature packages that provide smart frameworks enabling system administrators, packagers, and distributions to configure them all within policy.

A good example would be apache2's /etc/apache2 directory, which allows for said admins, packagers, and distributions to drop their own configuration modifications as files (or symbolic links) into sourced directories such as /etc/apache2/conf.d, /etc/apache2/mods-available, and /etc/apache2/sites-available.


The concept of a ".d" directory in /etc is very well understood in most Linux/UNIX circles, actually.  Check your own /etc directory with the command: find /etc -type d -name "*.d" and you should see quite a number of ".d" style configuration directories.

Here, I'm proposing a tool that I think would greatly benefit Debian and Debian-derived distributions such as Ubuntu.  For the purposes of this discussion, let's call the tool "dotdee".  Its stated goal is to turn any given flat file on your system to a dynamically concatenated flat file generated from a unique ".d" directory dedicated to that file.  With such a dedicated and well-formed directory, system administrators, Debian packagers, and distributions could conveniently place additional configuration snippets in particular conffile's dedicated ".d" directory.

I have written a first prototype of the tool dotdee, which you can examine here.  It's a very simple, straightforward shell script, inspired a bit by Debian's incredibly useful update-alternatives tool.

The script runs in 3 different modes:
  1. sudo dotdee --setup /etc/path/to/some.conf
  2. sudo dotdee --update /etc/path/to/some.conf
  3. sudo dotdee --undo /etc/path/to/some.conf


First the setup mode takes a flat file as a target.  Assuming the file is not already managed by dotdee, a new directory structure is created under /etc/dotdee.  In the example above, that would be /etc/dotdee/etc/path/to/some.conf.d.  So "/etc/dotdee" is prepended, and ".d" is appended to the path which is to be managed.  It's trivial to get back to the name of the managed file by stripping /etc/dotdee from the head, and .d from the tail of the string.

Next, the actual managed flat file is moved from /etc/path/to/some.conf to /etc/dotdee/etc/path/to/some.conf.d/50-dpkg-original.  Again, this a well-formed path, with "/etc/dotee" prepended, a ".d" appended, and the file itself is renamed to "50-dpkg-original".  This is intended to clearly denote that this is the original, base file, as installed by dpkg itself.  The number "50" is precisely halfway between "00" and "99", leaving plenty of room for other file snippets to be placed in ordered positions before and/or after the original file.

After this, we run the update function, which will concatenate in alphanumeric order all of the files in /etc/dotdee/etc/path/to/some.conf.d/* and write the output into /etc/dotdee/etc/path/to/some.conf.

Finally, the update-alternatives system is used to place a symlink at the original location, /etc/path/to/some.conf, pointing to /etc/dotdee/etc/path/to/some.conf.  Additionally, a second, lower-priority alternative is also set, pointing to dpkg's original at /etc/dotdee/path/to/some.conf.d/50-dpkg-original.


As mentioned above, the update function performs the concatenation immediately, building the targeted path from its dotdee managed collection of snippets.  This should be run anytime a file is added, removed, or modified in the dotdee directory for a given managed file.  As a convenience, this could easily and automatically be performed by an inotify watch of the /etc/dotdee directory.  That, itself, would be a dotdee configuration option.


The undo function is something I added for my own development and debugging while working on the tool, however I quickly realized that it might be an important tool for other system administrators and packagers (think, postrm maintenance scripts!).


This would require some (minor?) integration with dpkg itself.  On package upgrade/installation, dpkg would need to need to detect when the target path of a file it wants to create/update is in fact a symbolic link referencing an /etc/dotdee path.  It would need to drill down into that path and place the file it wants to write on top of the 50-dpkg-original file instead.  I have not yet contacted the dpkg maintainers yet, so I don't know if this is a reasonable proposal or not.


So what would this look like in practice?

Once integrated with dpkg, I'd like dotdee to be a utility that human system administrators could run to manually turn a generic conffile into a ".d" style configuration directory, such that they could append their own changes to some numbered file in the dotdee directory, avoid the interactive dpkg-conffile-changed prompts.

More importantly, I would like one package's postinst maintainer script to be able take another package that it depends upon and turn its conffile into a dotdee managed file, such that it could append or prepend configuration information necessary for proper operation.


I plan to lead a session on this topic at the Ubuntu Developer Summit in May 2011 in Budapest, and I have also proposed this on the debian-dpkg@ list as well.

But in the mean time, what do you think?  Have you encountered this problem before?  How have you solved it?  What parts of this proposal do you think are reasonable?  Are any parts completely unreasonable to you?  Can you think of any extensions or changes you'd make?  Would you use something like this?