Monitoring is a fairly important component of a production stack. I’ve heard really good arguments for it being part of the dev stack too - using production monitoring to test application response during continuous build/deployment supplements unit or integration tests really nicely. Puppet has OK support for Nagios, I thought it was time we had some better support for something else - a quick check showed nothing really compelling for Zabbix. Choice made.
Configuring Zabbix isn’t massively hard - it’s not incredibly easy, but not the end of the world. The thing that kills me, still does, is the screaming lack of built in support for automating the configuration of the application - by this I mean adding hosts, groups, templates etc. There are, as far as I can find, no command line tools that ship (I found zabcon, nuff said) with it that allow this. So, out of the box, if I want to change a configuration I need to start a web browser and drive it with a mouse. No! Wait! There’s a JSONRPC API that talks over HTTP - sweet. No. What? What a pain in the arse.
So, anyway, I’ve been hacking on some tools I found to get Puppet types up and running - the fastest way to get up to speed was to use https://github.com/vadv/zabbixapi. It still needed some monkey patching to get it working - which is leading me down the path of writing my own JSONRPC library so I don’t have to carry on down the road of forking someone else’s code.
The moral of this story is this: Your enterprise software needs to be automatable, using native tools that you yourself ship, thusly that other people can wrap your shit in their shit and get all the shit working together. I probably won’t ever use Zabbix, pretty much precisely because of this - it was hard to use, the API is hard to use and the internals are structured in a way that just doesn’t make sense to my brain.