Puppet fileserving supports a REST api for file serving.  Individual mount points can be created which give access to specific nodes.  Alternatively you might use the special modules mount to serve data out of the files directory from <insert name of module here>.

There are security implications of fileserving from modules using the default REST API security configurations.  The issue is that the security landscape is a bit on the flat side - any authenticated node can gain access to any file resource from any module.  The upside is that it’s only open to authenticated nodes - no anonymous access allowed, but the downside is that any node can request any file.

Thus a file resource for my SSL private key,

file { ‘/etc/httpd/ssl/private_keys/puppet.spence.org.uk.local.pem’:
  ensure => file,
  owner  => ‘apache’,
  group  => ‘apache’,
  mode   => ‘0600’,
  source => ‘puppet:///modules/ssl/puppet.spence.org.uk.local.pem’,
}

is a little bit on the risky side if you ask me, because any node knowing the path can ask for the key.  Not.  Awesome.

The solution is to use auth.conf  to restrict access to the files path to just the nodes that need that particular set of resources either at the module or the individual file resource level:

path ~ \A/file_(metadata|content)/modules/ssl/puppet.spence.org.uk.local.pem
auth yes
allow puppet1.spence.org.uk.local
allow puppet2.spence.org.uk.local

You don’t necessarily need to restrict right down to the file level, like I’ve shown here.  In fact auth.conf gives enough flexibility to be able to serve *machine* specific files out of a module using regular expressions.

Consider:

file { ‘/etc/httpd/ssl/private_keys/puppet.spence.org.uk.local.pem’:
  ensure => file,
  owner  => ‘apache’,
  group  => ‘apache’,
  mode   => ‘0600’,
  source => ‘puppet:///modules/ssl/${::clientcert}/puppet.spence.org.uk.local.pem’,
}

Which is controlled by the following ACL:

path ~ \A/file_(metadata|content)s?/modules/[^/]+/([^/]+)
auth yes
allow $2

This means that you can have directory structures to securely serve node specific files from a single module where each node can only access it’s own files and not any other files from the same module:

├── files
│   ├── puppet1.spence.org.uk.local
│   │   └── puppet1.spence.org.uk.local.pem
│   └── spacewalk.spence.org.uk.local
│       └── spacewalk.spence.org.uk.local.pem

There’s some additional overhead to managing this - it’s a slightly custom auth.conf configuration, and it would need to go towards the top of the file so as to be the most specific match for the file_(content|metadata) endpoint.

Whether this is worthwhile depends entirely on whether the paranoia torrent is running strong or not.  I think it’s entirely necessary for multi-tenant infrastructure especially where one is hosting configurations for multiple business units or different organisations.

I also think it’s well worth doing for configurations that are security sensitive - private key data such as I’ve shown here, maybe passwords - that sort of stuff.

  modulepath = /data/puppet/core/modules:/data/puppet/webapps/modules

  manifest = /data/puppet/manifests/site.pp

   user { 'root': 
    password => 'myspecialpasswordhash',
  } 

class configurethewebserver inherits security { 
  User['root'] {
    password => 'wejustpwnedyourwebinfrastructure',
  }
}

DO NOT DO THIS ANY MORE - SINCE THE ADVENT OF TRUSTED FACTS, THIS TECHNIQUE IS DEPRECATED - LEAVING THIS POST HERE FOR THE SAKE OF HISTORY

This post was originally published at https://puppetlabs.com/blog/the-fact-is/ - I am republishing it here.

One of the major interfaces to extend functionality with Puppet is the use of Facter and custom facts to provide catalog compile time data to the Puppet master to customise the configurations for a specific node (using PuppetDB, this data becomes available even when you’re doing things elsewhere too).  A fact, in case you were wondering, is a key/value data pair that represents some aspect of node state, such as it’s IP address, uptime, operatingsystem or whether it’s a virtual machine.

As an example within Puppet, one might use this data in the context of a catalog compile to make a decision about the catalog that we sent back to the node.  A fact that tells us what Operating system a node has could cause some conditional logic in a class to tell the node the right set of packages to configure ntp on the system (because the package names differ between Linuxes, let alone what you do for a Windows).  Alternatively one might use the ipaddress fact to customise the contents the appropriate listen directive in Apache or SSH.

Custom facts might be written using either by extending Puppet using Ruby, or even more trivially by using the facter.d library shipped with the Puppet Labs stdlib module or even by simply setting an environment variable available during a Puppet agent run.  In this way, you can provide the catalog compilation infrastructure rich state information about your nodes.

We can see that it might become common practice in all one’s modules to alter behaviour in Puppet classes on the basis of node state data - in fact it’s one of the things that makes it so useable and flexible.  However, there are some security implications here.

To take an example, one of the more common facts that gets used is the ‘hostname’ fact.  Typically, unless for some reason you choose otherwise, this will match the certname of the node (i.e. the CN on the certificate generated out of the Puppet CA when you provisioned the node) though Puppet won’t use the same mechanism to arrive at that determination.

The value of $::hostname, depending on your platform, will probably be arrived at by running a command like hostname from the context of the Facter libraries used by Puppet.

A better solution is to extract the validated CN of the certificate used to authenticate a node on the Puppet master and use that data instead.  Previous versions of this post suggested using $clientcert - turns out that the shared popular misconception that there was any value in that bit of data was utterly mistaken.




From a security perspective there is a vast difference between the assertion backed by PKI that a node’s hostname is the fact versus the certname.  This is especially true given the variety of ways we might arrive at the value that has been sent to the Puppet master.  Assuming we trust the PKI, the state of being that the node’s hostname matches the certificate (if that’s the way we run our PKI) is significantly more trustworthy that the assertion that the hostname fact is the correct one.  This is the major reason Puppet uses SSL and PKI for node identification.

Admittedly, in order to get to the point where you’ve sent that data to a Puppet master, one needs sufficient rights over the private key material on a node to get as far as getting Puppet to think about sending you stuff.  The implication for secure module writing is that if you want to securely distribute the right configuration settings and associated data back down to nodes you must use trustworthy data when compiling a catalog.

For settings with a lesser security impact, OS package names for example, trusting facts is cheaper and probably secure enough (depending on your environment) that you might as well just use them.  What you probably don’t want to do however is have your module that distributes, for example, private SSL key data to nodes that asset via facts that they should have them.

For modules with security critical information in them, the determination as to how that class instantiates on a node should be made solely using data on the Puppet master - that might be data in the puppet code that comprises your class, in an ENC or in your hiera backend - it should in no way be influenced by an untrusted assertion from a node.  Even then, the root of the decision tree within your code ought to be your PKI - thus based on the SSL certificate the node used to authenticate.

If you completely trust data supplied via facts, then including data, settings or anything else in a catalog  where that information might be used to perform a privilege escalation where a node has spoofed fact data, you’ve been hacked.  The true danger here is that the escalation you achieve is to get root on something else.  If in your organisation different sets of users have root on different sets of boxes, your module design philosophy could lead to leakage across node sets.

As an aside, if users have root on multiple boxes, what’s to stop them moving SSL certs around the place and putting the right configurations on the wrong boxes.  Don’t give people root if you don’t have to by the way.  Sudo everything is probably a bad thing too.  Sudo don’t do that.

The alternative to using facts, as I mention above, is to use trustable data.  That’s a combination of $actualclientcert and the data you have control over on your Puppet master.  It may increase the overhead of the amount of static data you need to manage, but this is going to be a trade off between security, how much you can be bothered, and what the outcome of it all going wrong is.  Incidentally, this significantly promotes the idea of encoding useful data in hostnames and therefore Puppet certificate names (see my post on the simple things - this is an extension of stuff I hadn’t fully thought through there).

I personally have no problem with using facts to make decisions with low impact outcomes security wise.  I can’t immediately see is any way of making facts trustworthy given the current architecture though - Puppet runs as root, if you’re root you can run Puppet and have access to the private key and you can spoof facts.

Ultimately however, this comes down to is the good old fashioned security principles of not trusting user input, and sanitising the hell out of it before you do anything with it.

I recently wrote a Puppet report processor (mconotify) that processes Puppet reports and sends mcollective messages to trigger Puppet runs.  It was an attempt to build infrastructure where one part of an infrastructure can securely touch another part of the same infrastructure.

I’ve also been thinking about how to make catalog application also take into account parts of an infrastructure when applying state to a node.  Being able to test another configuration at apply time in a way that pauses a Puppet run while you wait for something else to happen sounded like a good idea to me.

One solution I came up with was to use the Puppet resource abstraction layer to do a sleep for an arbitrary length of time - if you have to wait 5 minutes, then

  sleep { 300: }

A more sophisticated approach would be to sleep for 5 minutes, but stop sleeping if your condition is satisfied:

  sleep { ‘until i need to wake up’:
    bedtime       => 300,
    wakeupfor     => ‘nc thedatabaseserver 3306 -w 1’,
    dozetime      => 10,
    failontimeout => true,
  }

In this example, we’ll sleep for a maximum of 5 minutes, but wake up every 10 seconds to netcat to the mysql port on a database server and if that exits successfully exit the sleep and carry on.  If the command doesn’t return 0 by the end of 5 minutes, we fail the resource (and anything in my catalog dependent on it will therefore not apply).  If you wanted to continue anyway, set failontimeout to false, and the thing won’t fail.  Simples.

You could use exec resource types, but it starts to look complicated in terms of the commands you have to run - wrapping it up in a type felt like the right solution here.

I think this is pretty neat.

The code is available on the Puppet Forge, as usual: http://forge.puppetlabs.com/fiddyspence/sleep