I’ve recently been involved in the transformation of the processes used to deliver change from being heavily manual, time consuming and error prone to fully automated, efficient and reliable.
To drive the new automated process without human interaction, other than maybe clicking a single button, we knew that we would need a way of managing application configuration that allowed the process to simply pick up the config values during its execution. Our automated deployment process was driven by bash scripts so compatibility with bash was desirable.
There were two types of config we had to deal with:
An additional complication to the management of config was due to the nature of our business with us hosting multiple deployments of the same product for different customers, up to around 20. This meant that the following dimensions had to be considered when deciding how to manage the config params:
We also decided that it was important to have an audit trail of changes to the config to help with the analysis and debug of any issues that may arise.
This would all be tied together through a Jenkins job where any preparation of the config could be performed and an artifact created to ease consumption by other Jenkins jobs.
So to summarise, our requirements were:
The only tools you’ll need for this solution are a shell environment (i.e. Bash), your favourite file editor and Git.
Starting with the first requirement, the need for the config to be consumable by shell scripts, we opted to define a set of key/value pairs in the form of Linux environment variables in a shell script called
params.sh, for example:
This provides a very simple way to inject parameter values into the process performing the application deployment. This could be via a Jenkins job or simply by sourcing the params file into another shell script e.g.
Addressing the second requirement, to support the provision of config files for multiple customers, products, environments and versions, we had to find a way of organising the config is such a way to make it easily accessible.
The options boiled down to preferring the config to be accessible first by customer then product or by product then customer. To illustrate this the directory structures for storing the config of each option would be:
To be honest it was six of one, half a dozen of the other as to which structure would be better. We went with the first option, customer then product, mainly due to the fact that then for a single customer we would have all of their config grouped together. As mentioned though there are arguments just as strong the other way.
So now we have the ability to source a parameter file into a Bash script that will be held in a directory structure such as
Finally, to meet the requirement of having an audit log of all config related changes we simply host the above file structure and parameter files in a Git repo.
Whenever we need create or update create a new Git commit and push the change to the central Git repo. A Jenkins job will then clone the repo and create an artifact (i.e. tar file) containing all config that can then be consumed by other processes (i.e. other Jenkins jobs).
As we are using Git we also include an audit log in the artifact to make it easier to reference the changes made to the config wherever they are being used. This is achieved with via the
git log command with some special formatting:
This produces output such as:
So that’s it. As in the title, this is a very simplistic approach that may not stand the test of the time but it works quite well for our basic needs right now.
Currently this solution doesn’t provide a way for app components to discover their config or receive config changes at runtime however without too much effort this solution could be reworked to play nicely with Spring Cloud Config or an equivalent tool.
The solution could also benefit from some kind inheritance model to reduce duplication. There are undoubtedly a common set of defaults values that rarely change across customers.
Arguably we could also do away with the version dimension as we will likely always be delivering the latest version for which the config must be compatible. We would therefore have a single set of config params for each customer / product / environment further reducing duplication.