Deployment Systems - Configuration 19 Feb 2011

When you are putting together deployment automation for a non-trivial system, or a system you expect to become non-trivial, there seem to be some common patterns to the better ones I have seen or learned about, so I’ll try to brain-dump what I have observed. This first bit is about configuration.

Seperate Business, Environment, and Override Configuration

This is one of the most important as your system matures, but also one which I hav often done pretty poorly. The fundamental idea is that any configuration setting which varies based on the environment the system is running in should be managed seperately from configuration which only varies by the released version. For instance, if your system knows the domain it runs on, then the domain name is generally a configuration property, and varies by environment. In development it may be “ufo.example.com” as you run everything on your local machine, “test1.example.internal” in the first test environment, and and “example.com” in production.

Thread pool sizes, on the other hand, don’t usually vary by environment and are generally changed only when a new version is being pushed which needs to fiddle them. This configuration is the same between all environments, so is considered business configuration (or application configuration, whatever, pick the qualifier you prefer).

To the deployed application, the configuration space may be unified, but when you set up how you manage your configuration keep seperate config trees for environmental and business configuration, and do your darndest to minimize your environmental configuration. This drastically reduces the risk of “oh, farp, we forgot to change this configuration property between test and production” type incidents. It drastically reduces the “hey, Joe Dev, what is this new example.wiffle.age property supposed to be?” when Sam Hack pulls Joe’s changes as Joe just added it to the business config which is shared (yes, he should add docs as well, but we all know that even if he did add docs, which is unlikely, Sam is going to ask before searching the wiki for them).

Finally, provide a means to override or overlay the business configuration in a particular environment or deployment. This is pretty much needed when you want to flip your recurring job to a one minute interval instead of a one day interval in order to track down some nasty bug in it. Any overlaid property needs to be considered temporary – they should never survive the next deployment. If it needs to last past the next deployment, it is no longer an override, it is the configuration value and should be in the normal business configuration.

Implementing this can be as simple as maintaining three sets of properties files which are combined at deployment time. The first is bundled with the application at build time, and contains the business properties. The second is part of the environment, and is provided at deployment time. The last is the overrides, which is also provided at deployment time, and contains values to override the environment and business properties. You can get much more arcane, but really, this is all you need most of the time.