If you’re running collectd chances are you’ve struggled or are struggling with the web frontend. The one shipped with collectd has some limitations and most people tend to use 3rd party solutions like Visage. Having looked at that and another few I wasn’t really happy with any of them due to dependencies or simple lack of functionalities. In the meantime I finally got some time to play with Graphite, which is a relatively new metrics collection system with some interesting properties. It has its own storage system called whisper and comes with a snazzy web frontend which is also compatible with rrdtools’s format. Being all written in python with the frontend based on django it was an easy choice for me.
Problems with RRD support and packaging
RRD support is buggy in the latest stable release (0.9.8), but a patch has been merged in trunk that fixes all known problems. The puppet module ships with it and applies it as part of the installation.
There currently aren’t any official rpm or debian packages so the installation is done from sources and the puppet module is for the most a bunch of execs. This is not very elegant, but it does the job for the time being and I’ve abstracted things enough that whenever packages become available it should be just a matter or swapping some resources out.
Some manual steps are still needed
A few steps still need to happen manually as automation would require more work than I have time for at the moment, but I hope to get to them later this month. The first problem is with creating the database and running django syncdb command. This expects user interaction to create an admin user. It is possible to sort this out and make the process unattended, but there’s a certain amount of work I didn’t have the time to do. Second you need to make the RRDs available to graphite yourself. In my case this means using lndir to create symlinks from graphite’s rrd directory to collectd’s one and setting followsymlink in apache for that dir.
Closing notes
This module only deals with the web frontend and whisper as a required dependency, there’s nothing in about carbon as I’m not running graphite for collection yet. Sooner or later I’m gonna do that so watch this space (or the git repo) if you’re interested in updates.
All the code is available from my repository on github
Credit to Youdevise for allowing me to release this code that has been developed while consulting for them.
If you enjoyed this article, please consider sharing it!
Hey, Thanks for the post! what did you make of the collectd to graphite pulgins? loggly/collectd-to-graphite was one. indygreg/collectd-carbon another?
Hey Andrew, thanks for your comment, hope the post was useful to you. I’ve only experimented with collectd-carbon because nodejs wasn’t viable where I’m doing this work. So far I’m pretty pleased with collectd-carbon and I’m hoping to roll it out in the next couple months. The biggest plus I’m looking forward to is whisper’s ability to define different storage schemas on a metric basis. I want this because some application metrics need very low resolution while other system metrics don’t and collectd’s RRD plugin does not allow you that level of control afaik. Have you tested either yourself? Have you thought about replacing collectd entirely with graphite?