FAQ
We're using Puppet as part of a broader toolchain that relies on delivering software for deployment using sets of Yum-based RPM packages. We've setup system, role and application specific Yum repositories on an environment-by-environment basis that ensure that the required set of RPM versions flow appropriately (e.g. from development to QA to staging and hence to production).

In this spirit we're packaging our Puppet modules as sets of RPMs too so the correct versions of the system, role and application specific modules flow along with everything else.

The problem arises when you consider the conflict that arises between the "natural" use of Yum-based RPM installation and the Puppet master's module delivery mechanisms.

Puppet allows "modulepath" to be set on an environment-by-environment basis, of course, thus supporting delivering different versions of modules from a master managing several environments.

The restriction lies with Yum/RPM's inability to allow multiple versions of the same (relocatable) package to be installed on the same system (even good old System V packages could do that!).

I'm looking for workarounds that aren't too egregious to either system!

Here are the ideas we've come up with so far:

Hack the RPM package names to include a version discriminator (e.g. "packageV1-1.0-noarch.rpm" rather than "package-1.0-noarch.rpm") to allow them all to be installed on Puppet master
Use Yum/RPM to install the modules directly on the client systems and find a way to restrict the Puppet master to managing the manifests rather than attempting to install the modules too.

Is the second method workable? It seems to be a blend between agent and apply modes.

We don't want to use apply mode since we really value using the master (even supplemented with Hiera) to act as the resource model provider to deliver configuration attributes to the agents as well as act as the node provider for Rundeck (used for distributed orchestration) using the Puppet/Rundeck plug-in (which doesn't seem to be environment aware - but that's another story!).

We'd appreciate any comments and feedback on this.

Thanks,

Anthony.




--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Search Discussions

  • Devzero2000 at May 27, 2012 at 6:10 am
    Sorry for the top posting.

    Imnsho, rpm had always permitted to have multiple package version if
    they not conflict, in fact the usual case is the kernel. Anyway your
    question is most rpm related: so if you like i suggest you to ask to a
    rpm mailing list.


    Best regards

    2012/5/27, Anthony Shortland <anthony@dtosolutions.com>:
    We're using Puppet as part of a broader toolchain that relies on delivering
    software for deployment using sets of Yum-based RPM packages. We've setup
    system, role and application specific Yum repositories on an
    environment-by-environment basis that ensure that the required set of RPM
    versions flow appropriately (e.g. from development to QA to staging and
    hence to production).

    In this spirit we're packaging our Puppet modules as sets of RPMs too so the
    correct versions of the system, role and application specific modules flow
    along with everything else.

    The problem arises when you consider the conflict that arises between the
    "natural" use of Yum-based RPM installation and the Puppet master's module
    delivery mechanisms.

    Puppet allows "modulepath" to be set on an environment-by-environment basis,
    of course, thus supporting delivering different versions of modules from a
    master managing several environments.

    The restriction lies with Yum/RPM's inability to allow multiple versions of
    the same (relocatable) package to be installed on the same system (even good
    old System V packages could do that!).

    I'm looking for workarounds that aren't too egregious to either system!

    Here are the ideas we've come up with so far:

    Hack the RPM package names to include a version discriminator (e.g.
    "packageV1-1.0-noarch.rpm" rather than "package-1.0-noarch.rpm") to allow
    them all to be installed on Puppet master
    Use Yum/RPM to install the modules directly on the client systems and find a
    way to restrict the Puppet master to managing the manifests rather than
    attempting to install the modules too.

    Is the second method workable? It seems to be a blend between agent and
    apply modes.

    We don't want to use apply mode since we really value using the master (even
    supplemented with Hiera) to act as the resource model provider to deliver
    configuration attributes to the agents as well as act as the node provider
    for Rundeck (used for distributed orchestration) using the Puppet/Rundeck
    plug-in (which doesn't seem to be environment aware - but that's another
    story!).

    We'd appreciate any comments and feedback on this.

    Thanks,

    Anthony.




    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.
    --
    Inviato dal mio dispositivo mobile

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Ohad Levy at May 29, 2012 at 5:22 am

    On Sun, May 27, 2012 at 9:10 AM, devzero2000 wrote:

    Sorry for the top posting.

    Imnsho, rpm had always permitted to have multiple package version if
    they not conflict, in fact the usual case is the kernel. Anyway your
    question is most rpm related: so if you like i suggest you to ask to a
    rpm mailing list.

    as far as i know, kernel is a very specific usage case, and while rpm does
    support it, yum does not (kernel multi version in yum is hard coded).

    Ohad
    Best regards

    2012/5/27, Anthony Shortland <anthony@dtosolutions.com>:
    We're using Puppet as part of a broader toolchain that relies on
    delivering
    software for deployment using sets of Yum-based RPM packages. We've setup
    system, role and application specific Yum repositories on an
    environment-by-environment basis that ensure that the required set of RPM
    versions flow appropriately (e.g. from development to QA to staging and
    hence to production).

    In this spirit we're packaging our Puppet modules as sets of RPMs too so the
    correct versions of the system, role and application specific modules flow
    along with everything else.

    The problem arises when you consider the conflict that arises between the
    "natural" use of Yum-based RPM installation and the Puppet master's module
    delivery mechanisms.

    Puppet allows "modulepath" to be set on an environment-by-environment basis,
    of course, thus supporting delivering different versions of modules from a
    master managing several environments.

    The restriction lies with Yum/RPM's inability to allow multiple versions of
    the same (relocatable) package to be installed on the same system (even good
    old System V packages could do that!).

    I'm looking for workarounds that aren't too egregious to either system!

    Here are the ideas we've come up with so far:

    Hack the RPM package names to include a version discriminator (e.g.
    "packageV1-1.0-noarch.rpm" rather than "package-1.0-noarch.rpm") to allow
    them all to be installed on Puppet master
    Use Yum/RPM to install the modules directly on the client systems and find a
    way to restrict the Puppet master to managing the manifests rather than
    attempting to install the modules too.

    Is the second method workable? It seems to be a blend between agent and
    apply modes.

    We don't want to use apply mode since we really value using the master (even
    supplemented with Hiera) to act as the resource model provider to deliver
    configuration attributes to the agents as well as act as the node provider
    for Rundeck (used for distributed orchestration) using the Puppet/Rundeck
    plug-in (which doesn't seem to be environment aware - but that's another
    story!).

    We'd appreciate any comments and feedback on this.

    Thanks,

    Anthony.




    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.
    --
    Inviato dal mio dispositivo mobile

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Kristof Willaert at May 28, 2012 at 12:03 am
    Hi,

    [snip]
    - Hack the RPM package names to include a version discriminator (e.g.
    "packageV1-1.0-noarch.rpm" rather than "package-1.0-noarch.rpm") to allow
    them all to be installed on Puppet maste

    This is a practice that is already used for the kernel, so it seems like a
    legitimate way of working.

    Some time ago, I faced the exact same problem you are describing. One
    possible solution I devised was:

    * Use the version of the module in the packagename (as you already
    considered) to make multiple versions on one system possible, and install
    all modules in the
    same basedir, using the version as part of the path, like
    /etc/puppet/modules/<modname>/<version>/

    * Create some script that mimics the functionality of the debian apache
    a2enmod, a2dismod scripts (they create symlinks for apache modules) that
    creates a symlink
    of the version you want in the modulepath of the environment you want
    (eg. /etc/puppet/modules/prd/users -> /etc/puppet/modules/users/0.1.2)

    You could also create a simple puppet class instead of the script, to
    manage these symlinks. If you use this with hiera, you get a nice yaml file
    that contains the
    deployed versions of your modules per environment.

    In the end I turned to a different solution (create a puppetmaster for each
    environment) because of a few other reasons. But I think the above should
    work without lots of problems.

    Kind regards,

    k

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Anthony Shortland at May 28, 2012 at 11:32 pm
    Very interesting that you should say that you created a Puppet master for each environment ... it's the obvious way to accommodate the limits of RPM packaging while retaining the ability to use Puppet in full agent mode.

    Is this considered a best practice? What other considerations drive the decision to have a Puppet master per environment?

    Anthony.
    On May 27, 2012, at 5:03 PM, Kristof Willaert wrote:

    Hi,

    [snip]
    Hack the RPM package names to include a version discriminator (e.g. "packageV1-1.0-noarch.rpm" rather than "package-1.0-noarch.rpm") to allow them all to be installed on Puppet maste
    This is a practice that is already used for the kernel, so it seems like a legitimate way of working.

    Some time ago, I faced the exact same problem you are describing. One possible solution I devised was:

    * Use the version of the module in the packagename (as you already considered) to make multiple versions on one system possible, and install all modules in the
    same basedir, using the version as part of the path, like /etc/puppet/modules/<modname>/<version>/

    * Create some script that mimics the functionality of the debian apache a2enmod, a2dismod scripts (they create symlinks for apache modules) that creates a symlink
    of the version you want in the modulepath of the environment you want (eg. /etc/puppet/modules/prd/users -> /etc/puppet/modules/users/0.1.2)

    You could also create a simple puppet class instead of the script, to manage these symlinks. If you use this with hiera, you get a nice yaml file that contains the
    deployed versions of your modules per environment.

    In the end I turned to a different solution (create a puppetmaster for each environment) because of a few other reasons. But I think the above should work without lots of problems.

    Kind regards,

    k

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Christopher Wood at May 29, 2012 at 12:31 am
    inline
    On Mon, May 28, 2012 at 04:32:03PM -0700, Anthony Shortland wrote:
    Very interesting that you should say that you created a Puppet master for
    each environment ... it's the obvious way to accommodate the limits of RPM
    packaging while retaining the ability to use Puppet in full agent mode.
    Is this considered a best practice? What other considerations drive the
    decision to have a Puppet master per environment?
    Being able to test puppetmaster upgrades with all supported builds without affecting production.

    Being able to leave my lab puppetmaster and some agents for a weekend with an angrier chaos monkey chewing on them, and not worry that it's affecting production.

    Etc., etc., doing things without touching production.
    Anthony.
    On May 27, 2012, at 5:03 PM, Kristof Willaert wrote:

    Hi,

    [snip]

    * Hack the RPM package names to include a version discriminator
    (e.g. "packageV1-1.0-noarch.rpm" rather than
    "package-1.0-noarch.rpm") to allow them all to be installed on
    Puppet maste

    This is a practice that is already used for the kernel, so it seems like
    a legitimate way of working.

    Some time ago, I faced the exact same problem you are describing. One
    possible solution I devised was:

    * Use the version of the module in the packagename (as you already
    considered) to make multiple versions on one system possible, and
    install all modules in the
    same basedir, using the version as part of the path, like
    /etc/puppet/modules/<modname>/<version>/

    * Create some script that mimics the functionality of the debian apache
    a2enmod, a2dismod scripts (they create symlinks for apache modules) that
    creates a symlink
    of the version you want in the modulepath of the environment you want
    (eg. /etc/puppet/modules/prd/users -> /etc/puppet/modules/users/0.1.2)

    You could also create a simple puppet class instead of the script, to
    manage these symlinks. If you use this with hiera, you get a nice yaml
    file that contains the
    deployed versions of your modules per environment.

    In the end I turned to a different solution (create a puppetmaster for
    each environment) because of a few other reasons. But I think the above
    should work without lots of problems.

    Kind regards,

    k
    --
    You received this message because you are subscribed to the Google
    Groups "Puppet Users" group.
    To post to this group, send email to [1]puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    [2]puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    [3]http://groups.google.com/group/puppet-users?hl=en.

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.

    References

    Visible links
    1. mailto:puppet-users@googlegroups.com
    2. mailto:puppet-users+unsubscribe@googlegroups.com
    3. http://groups.google.com/group/puppet-users?hl=en
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Kristof Willaert at May 29, 2012 at 8:54 am
    Hi,

    [snip]

    Very interesting that you should say that you created a Puppet master for
    each environment ... it's the obvious way to accommodate the limits of RPM
    packaging while retaining the ability to use Puppet in full agent mode.

    Is this considered a best practice? What other considerations drive the
    decision to have a Puppet master per environment?
    We are a hosting/service provider company and have a number of customers
    that have completely isolated setups. So a single puppetmaster was never
    really an option.

    We put some effort in the easy and automated creation of puppetmasters and
    module distribution (hence the RPMs :)), to be able to use Puppet agent
    mode.

    From there on, the decision to split them per environment (only for large
    customers ofcourse), was trivial. And it indeed allows us to test upgrades
    of our masters, while
    protecting us from "development" code accidently ending up on production
    machines.

    Don't know if it's a best practice. There is the overhead of a number of
    puppetmasters, and some loss of centralizatiion (for reports, ...), but
    that is something we knew from the start.

    Kind regards,

    k

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jcbollinger at May 29, 2012 at 1:35 pm

    On May 26, 8:56 pm, Anthony Shortland wrote:
    We're using Puppet as part of a broader toolchain that relies on delivering software for deployment using sets of Yum-based RPM packages. We've setup system, role and application specific Yum repositories on an environment-by-environment basis that ensure that the required set of RPM versions flow appropriately (e.g. from development to QA to staging and hence to production).

    In this spirit we're packaging our Puppet modules as sets of RPMs too so the correct versions of the system, role and application specific modules flow along with everything else.

    +1

    Hack the RPM package names to include a version discriminator (e.g. "packageV1-1.0-noarch.rpm" rather than "package-1.0-noarch.rpm") to allow them all to be installed on Puppet master

    I don't think that's too bad. Indeed, yum-based distros do exactly
    that for so-called "compatibility" libraries.

    Use Yum/RPM to install the modules directly on the client systems and find a way to restrict the Puppet master to managing the manifests rather than attempting to install the modules too.

    Turning off pluginsync would probably be all you need to make that
    work configuration-wise. I'd be a bit nervous about keeping all the
    plugins synchronized though. I can imagine schemes to make it work,
    but you need more care and admin effort that way.

    Or there's at least one other option: cut out Yum for installing these
    particular RPMs on the master. Since they're targeted only for the
    master, delivering them via Yum provides fewer advantages than many
    other uses of Yum. The underlying RPM system should not object as
    long as the packages don't conflict. I think this is more of a hack
    than your first option, however, only at a higher level.

    I think there is some confusion here surrounding the role and purpose
    of these RPMs. If the primary purpose of each one is to provide a
    puppetmaster the ability to support a particular system, role, or
    application (or combination of the above), then what you're
    characterizing as different versions of the same RPM in fact provide
    separate and distinct capabilities. As such they *should* have
    different names.

    Contrast that with the situation where the primary purpose of the RPMs
    is to provide Puppet modules to client systems. In that case, the
    different "versions" provide alternative, conflicting versions of
    logically the same features. In that case they should have the same
    name, and it follows naturally that yum objects to installing more
    than one.

    All things considered, I think your option 1 is right, good, and
    proper, not a hack at all.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedMay 27, '12 at 1:57a
activeMay 29, '12 at 1:35p
posts8
users6
websitepuppetlabs.com

People

Translate

site design / logo © 2022 Grokbase