[Puppet Users] Referencing a variable from one class in another

Variable references like $apache::params::config_d are parse-order
dependant. The class apache::params has to be parsed on the master
before you reference the variable. (Order of application doesn't matter,
so you don't need to use 'require')

The fix is trivial, just add an "include apache::params" (or even just
"include apache") at the top of your kibana::apache class, like so:

# modules/kibana/manifests/apache.pp
class kibana::apache (
$version = $kibana::params::parameters['version'],
) {
include apache::params
@file { $kibana::params::apache_config:
...

However, this conflicts with the class {} style declarations in your
apache/manifests/init.pp file. For best results, you should switch those
to use the "include" method as well.

Thanks for the response.

Can multiple classes include the same class. Let's say I instantiate the apache class from manifests/nodes.pp which in turns includes apache::params. Can kibana include apache::params then as well with no conflict. I know you can't do this with the class {} style declarations. Also, I thought the class {} style declarations were the preferred way or is that just in the nodes.pp file?

Yes, with the "include" syntax, you can include a class multiple times
from different places with no conflicts. It's a no-op if the class is
already included.

The "include" method is the preferred syntax (in my opinion, but I'm
sure other share it), due to this feature - particularly if you're using
hiera to pull configuration in. The only time you should be using the
class {} syntax is if you need to pass in class parameters and can't use
hiera.

Yes, they can. That's the main selling point for the "include class"
syntax. And you are right, you can't use the class {...} syntax more than
once or you get a duplicate definition error.

However, let me warn you against going overboard with having classes
include other classes from other modules. It can be annoying to track down
where resources coming from for any given node if you've got cross module
inclusion: kibana includes httpd includes mod_ssl includes openssl includes
somethingelse includes ... How did this get on here?

A cleaner way might be to declare cross module relationships using the
Arrow operators:

class kibana::apache {
Class[apache::params] -> Class[kibana::apache]
...
}

And then you make a house rule to have all your classes instantiated in
your node definitions:

node woof {
class kibana
class apache::params
}

If apache::params is missing, you'll get an error saying so. It also fits
rather nicely into an ENC if you want to go in that direction now / later.

-Luke

While this is a good idea in general, it doesn't solve Luke's original
problem. In order to reference a variable $apache::params::something
from inside the kibana::apache class, you need the apache::params class
to be parsed on the puppet master before the kibana::apache class. This
is a parse-time ordering problem, not a run-time ordering problem.

The only way to force parse-time ordering right now is to do a direct
include, unfortunately.

Exactly.


Yes, but I'm not sure I would say "unfortunately" there. The problem is
not so much with any limitation of Puppet DSL in this area, but rather an
issue of module design and manifest set architecture.

I mean, one should be very careful and deliberate about designing modules
such that their classes need to rely on class variables of other modules'
classes. Indeed, it is probably a poor idea to implement such a design
unilaterally -- instead, the module providing the class variables should be
designed and implemented in anticipation of that usage as well. That
probably means centralizing all variables intended for cross-module
reference in one well-known class, documenting their names and value
ranges, and committing to avoiding incompatible changes there.


John

I'm not sure I fully agree with this from a design standpoint. In object-oriented programming, one of the design principles is that variables relating to the object are encapsulated within the object and exposed or not depending on how they should be accessed. IMHO, it also makes it more obfuscated when you're accessing say the SSL CA cert path variable and that's in some 'common' module that everything has to include. Granted it makes it easier on the module developer - just always in include the common module and your variables should be there - but it also makes it less explicit. I would argue, if you're writing a module that depends on using the SSL CA cert path you have some dependency on the SSL module and should have some understanding of what that module does and the ramifications of using that module, so you should explicitly include that module for that dependency. In just about every language you must include the external modules/libraries you depend on for functionality outside the standard norm. In puppet the standard norm - the stdlib.h equivalent if you will - I would consider to be facter variables. You want to use LDAP or SSL or Kerberos? You best include those modules explicitly and figure out what you can use from them - ldap.h <> ldap::params, ssl.h <> ssl::params, etc. Standardize how you create these public puppet 'headers' and use them explicitly and appropriately that way. At least that's my 2c.

How would you handle variables that wouldn't otherwise be tied to a module?

An example I ran into when I was doing our first deployment was the path to
various shells. They vary from OS to OS, but rarely (if ever) need a whole
module to manage them. The paths get referenced in many places, such as
when adding users or installing scripts (erb used in the bangpath). I've
found it useful to have things like $::site::params::bash and
$::site::params:tcsh for shells, and other site-wide variables for other
things. It means I only need to put the case logic to figure out the path
based on the OS in one place, and not have it scattered around several
modules that all need to figure out the same thing.

I include that in the module that installs the shell packages and configures them, in my case, I call it base. In other words, the variables should be as close to the things they affect or are affected by. Just because every node might include base (or site) doesn't mean every variable you'll ever want to use should be in there. That, to me, creates a messier and more confusing dependency relationship between modules that use a variable and what that variable ultimately affects.

If you have a module that installs and configures those shells, then great.
But many systems come with those shells preinstalled, and there's nothing
to manage. Perhaps shells were a bad example, since one of tcsh or bash
may need to be added to an OS by puppet. What about the path to sed?
That's part of the base OS for all systems I manage, but the path varies
from OS to OS. It seems overly cumbersome to me to create an entire
module just to assign one variable, and then repeat that for a dozen or so
other variables with similar circumstances. It seems cleaner to have a
single small module that contains site-wide definitions that aren't
obviously tied to modules of their own.

If the package or service comes standard with many OSes (my standard is the bare bones basic set, not the 'default') then I would argue they should go in the module that configures all of those base packages/services. However, I still don't think that that module is a suitable place for all possible variables that another package may want to use at some point in the future... possibly.

Take for instance examples I'm currently using: Apache's configuration directory (/etc/httpd/conf.d on RH, /etc/apache2/conf.d on Deb) or rsyslog directories or RADIUS directories or SSL certificate paths or any number of packages that are completely optional on some hosts. All of these are completely optional, some hosts may not install them, some may, some may install alternatives to them (syslog or syslog-ng), but when they're installed and used the variables within are very critical. I don't think these variables should be shoved into the base module just because it's already being used on every host. They should be left in the module that it matters to, because if you need that variable, you should be pulling in that module anyway - why put in an apache snippet (and need the apache config dir) if you're not already running apache and therefor pulling in that module.

Indeed so. Do be aware, however, that Puppet DSL is not an object-oriented
language. Among other things, it has no language-enforced encapsulation at
all, and Puppet "classes" are not classes in the OO sense of the term.
Puppet has no accessor methods (or any methods at all, actually), so the
only external interface that modules can provide consists of the data
relied upon and and declared by the module.

Inasmuch as one can apply OO principles even in contexts that are not
natively OO, however, my advice cleaves closely to OO orthodoxy by
recommending well-defined and documented component interfaces and
discretionary observation of encapsulation (since the language won't
enforce it). The only way to adhere more strictly to OO principles would
be for modules to avoid relying on each others' data at all. That's a nice
ideal, but it is often not practical to apply as an absolute requirement.
Such a recommendation would not have been responsive to the question,
anyway.




If OO is the ideal design approach (which is not at all clear, but you
brought it up) then it would dictate that related data be grouped into
objects. That's the most fundamental and incontrovertible principle of OO
-- you can have OO without inheritance, without polymorphism, even without
enforced encapsulation, but you can't have it without *objects*.

Puppet's analogs of objects are classes and resources. Ergo, if one wants
to pursue an OO strategy then classes should contain and own their own data.

More generally, modules aren't modular if they have to rely on all their
data being provided by some other module. Nevertheless, it doesn't really
matter which class in which module is documented to provide the data of
interest. As long as the module documents it and commits to those details
remaining stable, it amounts to the same thing I described.




And now that you drop the idea of a common data module, we come back around
to using other modules' classes' variables. Are you honestly arguing that
it is better to analyze class implementations and pull out variables
willy-nilly than to define and rely on documented interfaces? I think you
owe a penance at the OO altar.


John

I'm aware that Puppet DSL is much more declarative than OO, but in this particular light there is a lot of cosmetic lip-service to typical OO design - classes, inheritance (though limited), paramerterized instantiation of classes - which is why I brought that up. I'm fine with using those concepts and what their limitations are in the Puppet implementation. I'm also not advocating to adhere more strongly to OO ideals. At the end of the day, Puppet, for me at least, is a configuration management system. In that light it is more critical for me that it do that well than adhere to some design principle rigidly just because. And one of the things that makes a good config mgmt system work well is reducing the duplication of data that can cause inconsistencies. If a rigid adherence to OO (or declarative or imperative or whatever) design principles makes that hard or impossible, it should be carefully examined. A huge way I've seen config mgmt break down is when data is duplicated in multiple places and a need to change or update that data is needed and one or two spots are forgotten about making things break or behave badly causing me more time debugging what should have been a simple (and consistent) change.
Agreed (See above), but if every module has to define ssl_ca_path every time they need it, I don't want that methodology because when I need to change or update ssl_ca_path, I need to remember every module that defined and update it instead of updating in the one location that all dependent modules pull from. Sure I can define all this in one common module that every module that has a 'public' variable places stuff in, or I can define it in hiera and every module pulls it from there (though there are inheritance issues there that makes it less ideal), but at the end of the day I think we agree you should pick a method, document it, and stick to it.
I'm arguing that no more than I would argue having to delve into the source code of openssl in order to use it in my C program - that's what the header is for. I'm arguing to define your class implementations such that the nitty gritty of that class' implementation need not be inspected in order for another module to make use of it, not mash all the public bits in to one globally public class that has no nitty gritty bits to implement. In my example <module>::params is considered the header for the module (granted a header that exposes values, but that can't be helped due to the declarative nature of the DSL). There should be no implementation in that sub-module and even <module> should reference that 'header' to get the variables it needs to do its work. But I'll still pay penance at the OO altar for all my past transgressions against and abuses of it.

Well I'm glad that's cleared up, then.


Maybe we're just having a terminology problem. Puppet has no concept of
sub-modules, and the only construct available to hold reference-able,
non-global variables is the class. Indeed, even modules themselves are a
Puppet implementation detail -- the DSL has no concept of them except as
top-level namespaces (but top-level namespaces often map to classes, either
instead of or in addition to mapping to modules).

So, would you care to explain how your <module>::params then differs from
"mash[ing] all the public bits in to one globally public class that has no
nitty gritty bits to implement"? Are you suggesting separate ::params
classes shadowing multiple different classes in the same module? Are you
conflating class *parameters* with class *variables*?


John

Let's go back to my original example from http://pastie.org/5910079#. Not stated in that code snip (for conciseness) is a module, kibana. Among other things it needs to install an apache configuration to make it a useful piece of software and that configuration is in the kibana::apache sub-class in the form of a snippet that is tagged such that the apache module can instantiate it later on at the proper time (there's an alternative to this method). In order to do this, the kibana::apache class needs to know where to install this configuration file so that the apache process can load it. That location I chose to place in a variable, $config_d, in the apache module in a sub-class (sorry for the improper nomenclature before) apache::params. To solve my original problem I simply add:

include 'apache::params'

right above the @file snippet in kibana::apache and everything is happy. I think everyone can agree that is the right solution to the problem that I posed; however, both you and Luke strongly suggested against having modules include other module's variables willy nilly and instead move those variables that multiple modules need to reference into a globally included class, we'll call it globals::. This class, I assume (correct me if I assume wrong), would always be instantiated before all others, say in the nodes.pp file, to ensure that all subsequent modules could resolve the variable appropriately. I would assume in that class you would have all of these 'global' variables that multiple modules make use of so you'd have (eventually) a long list in this one class:

class globals {
$apache_config_d = '/etc/httpd/conf.d'
$openldap_schema_d = '/etc/openldap/schemas'
$rsyslog_d = '/etc/rsyslog.d'
$ssl_ca_path = '/etc/pki/tls/certs'
...
}

And in my kibana::apache class I would reference ${globals::apache_config_d}. If all of those assumptions are true, I'm curious why this solution is any better? The only benefit that I can see is that when writing a new module you don't trouble making sure you've loaded the proper prerequisites for variable resolution because it should already be done. You're still including another classes variables, albeit in this only ever one other class. The other alternative that has been alluded to in a pure programmatic way is that none of those variables should be shared between modules and each module should have their own local variable to use. I'll consider this proposed as only for the purist and not really tractable in any real complex environment.

What I proposed is only semantically different than the above but, in my mind, is a cleaner (you don't have one huge file that has all variables) and more explicit (you are required to include the class that has the variable you care about when writing your module). It leaves the variables closest to the module that has the most influence over that variable - the variable $ssl_ca_path is directly dictated by the openssl package (which can only be in one module) and so it should remain as close to the module that contains that package.

As another tangent, I mentioned there was an alternative to having the kibana apache configuration be served from kibana module and that is to move the kibana config into the apache module. That would solve the cross-module variable referencing, but again, in my head that removes the element further from the thing that most influences it - the kibana apache configuration certainly requires and makes use of the apache module, but the kibana module has more influence over the kibana apache configuration itself.

For this specific example I would not model it this way. I would make the
Apache module provide an interface to allow other modules to add to itself.
I would make something like an 'apache::vhost' or
'apache::extra_conf_file', which is just a wrapper around a Puppet File and
a notify => Service[]. Thus the implementation of how to create/manage
Apache config files (like their location) is wholly contained in the Apache
module and other modules don't have to "know" specific details.

That design may not scale out for more complex examples though. Looking at
my own modules I should really practice what I preach because all too often
I just drop a file in /etc/httpd/conf.d from anywhere I please ;-)

right above the @file snippet in kibana::apache and everything is happy. I

Ah! Now that's the difference I was looking for. Thanks!

No, neither Luke nor I recommended any such thing. I don't even see how
you could tag Luke with that, though I do now see how you could have
misunderstood my own remarks that way. My original comment about
"centralizing all variables intended for cross-module reference in one
well-known class, documenting their names and value ranges, and committing
to avoiding incompatible changes there" was made and intended to be
interpreted in the context of designing an individual module, so that the
"one well-known class" is *for that module*. My apologies for being
unclear.

A per-module ::params class, as you seem to be advocating, is completely
consistent with my recommendation. Even with that alternative, however, it
is best to minimize cross-module class references (as Luke previously
said), and I particularly like the example he just offered for a means of
doing so. I'm inclined to think that it would be less prone to tricky
parse-order issues and dependency cycles (or insufficient dependencies)
than what you describe doing in your kibana module.


John