/dev/oops

fiddyspence's blog

Puppet Auto Scaling

Wouldn’t it be a thing if Puppet just knew how many things of a type you wanted and when you built a server it just did the right thing. People in the community have been abusing Hiera as a classification tool for ages (and my feelings on this are still that if you’re using facts as a classification engine, you’re doing it wrong). Certificate CN regex matching is a thing, but it means you’ve got to worry about making the CN right.

Is there another way?

Imagine a world with a site.pp with this in it:

node default { include magic }

That world is here. Class magic will do all the things for you, assuming you’ve got a PuppetDB behind you, combined with some external data describing how many things of a certain type you want, Puppet can automagically make sure that you have the right number of webservers/load balancers/database servers. Hiera has a key magic::config which is a hash that describes what you want:

---
magic::config:
  balancers: 2
  webservers: 5
  otherthing: 2

The magic class compares that data to the current known state of your infrastructure as known by PuppetDB using the dalen/puppetdbquery and either reclassifies, newly classifies or does nothing to a node:

class magic ( $config ){

  $balancers  = query_nodes('Class[magic::balancers]')
  $webservers = query_nodes('Class[magic::webservers]')
  $otherthing = query_nodes('Class[magic::otherthing]')

  $balancercount = size($balancers)
  $webserverscount = size($webservers)
  $otherthingcount = size($otherthing)

  if $balancercount < $config['balancers'] or $clientcert in $balancers {
    include magic::balancers
    $done = true
  }

  if ( $webserverscount < $config['webservers'] or $clientcert in $webservers) and ! $done {
    include magic::webservers
    $done = true
  }

  if ( $otherthingcount < $config['otherthing'] or $clientcert in $otherthing) and ! $done {
    include magic::otherthing
    $done = true
  }

}

With this code, I can bring up an arbitrary number of nodes, and the first n (as described by $config) nodes will be classified by the class. The other nodes would effectively be superfluous and do nothing until either the desired config changes, or an existing node dies.

In the next post, I’ll show the contents of the magic:: classes to show how some other PuppetDB techniques can be used to make dynamic configurations really fly.