[Puppet Users] Searching for Nodes

Hi Brian,

The use case you are describing lends itself toward making use of
stored configuration and exported resources.

http://docs.puppetlabs.com/guides/exported_resources.html

http://projects.puppetlabs.com/projects/1/wiki/Using_Stored_Configuration

Additionally, if you are using haproxy as your load balancing
solution, there is already a module available that uses these features.

https://github.com/puppetlabs/puppetlabs-haproxy

Hope that helps.

Regards,
Bill

Hello Bill,

Thank you for the feedback. We are actually using haproxy, and will
undoubtedly use the stock book, it was just the example.

I guess I am confused as to the pattern. I have looked over the nagios
examples many times but I am still unsure. T

he final example would be how to distribute a database IP without
hard-coding it.

My understanding of the exported resources would be that the database needs
to export the configuration file, and the application servers would need to
import that file. This means that the database box needs to have
application level knowledge to create the config file needed.

What am I not understanding? Is the puppet pattern to set the $database_ip
variable?

Brian

At a high level, there are two basic alternatives:

1) The master "knows" the data with which nodes should be configured, and
dispenses it to each node as needed, or
2) nodes publish resources containing information about themselves that
other nodes may collect and use.

The two are not exclusive.

For the case of configuring application servers with a database IP, you
could operate either way. The master has to know which node is the
database server in order to configure that node appropriately, so that
information is in the master somewhere. If it is only a question of
converting a hostname to an IP address, or of structuring data and
manifests so that the database server can easily be changed, then those are
issues that can be handled.

On the other hand, it can be very convenient to use exported resources for
this sort of thing. The question, then, is what resource to export. If
the application server's configuration file needs information that the
database server cannot provide, then the exported resource solution is for
the database server to export a file fragment instead of an entire file
(see the Concat add-on module, available from the Forge).

Remember also that we speak loosely when we talk about information that the
database server can provide, or that the application server can provide.
The only things that are actually provided by any node during catalog
compilation are node facts. It is the *master* that has to have
"application-level" knowledge such as file format details. That
information is therefore available during the compilation of *any* node's
catalog.


John

You don't need to export resource per se, an empty define resource allows
you to export information as data.

define data(
$value,
) {}

node database {
@@data { 'ipaddress':
value => $::ipaddress, #obtained from facter, heck you don't really
need to do this because it's available in puppetdb.
}
...
}

But the problem is collection syntax only gather resource, so you need a
custom function to perform the query and treat the result as data. This is
the gap in the Puppet DSL, and there's no official solution if you want to
treat catalog/puppetdb as source of data rather than resource.

There's another module under RI repo that Dan Bode used as PoC for
openstack modules. It's delivered as a puppet face with matching puppet
functions: https://github.com/ripienaar/ruby-puppetdb

The face provides a way to treat puppetdb as source of data. For example,
for database_ip you can simply ask (and probably more interesting asking
what network, is it production, etc):
$ puppet query node --query '(Package[mysql-server] and
architecture=amd64)' --filter ipaddress

In the manifests this is:
$nodes = unique(query_active_nodes('Package[mysql-server] and
architecture=amd64', 'ipaddress'))

Some caveats:
1. Not really an official PuppetLabs project (i.e. experimental).
2. It's not really tested at scale and does data filter after getting a
large result set. Recent changes in PuppetDB allows more optimal query so
PuppetDB does the filter, but I'm pretty sure not taking advantage of it
yet.
3. PuppetDB exports on catalog compilation, so your database server might
not be online yet.

Thanks,

Nan

Thank you all for your feedback.

For anyone looking to do the same here is what i ended up with, I would
appreciate any recommendations if there is a better way to do this, I find
it dirty and think it should be done in the 'view' layer :)

==== Class for node that is providing the IP address

class primarydatabase {
include concat::setup

@@concat::fragment {'databaseIP' :
content => $::ipaddress,
}
}


==== Class for node that needs the IP address

class base {
include concat::setup

package { 'htop':
ensure => installed,
}

$db_file = "/root/database.yml"

concat{$db_file:
owner => root,
group => root,
mode => '0644',
}

concat::fragment { "${db_file}.header":
target => $db_file,
content => "# Managed By Puppet\n\n",
order => '01',
}

concat::fragment {"${db_file}_production_database_config" :
target => $db_file,
content => "production:\n host: ",
order => 100,
}

Concat::Fragment <<| title == 'databaseIP' |>> {
target => $db_file,
order => 101,
}
}