FAQ
I'm testing a 'cleanup' stage which runs after Stage[main] and removes a bunch of package resources.

To do this, I tried a simple check of defined(Package[<foo>]) combined with a custom facter fact (called 'app_packages'):
class app::package::cleaner {

define check_and_remove {
if !defined(Package[$title]) {
package { $title:
ensure => absent
}
}
}

$apps = split($::app_packages, ',')
check_and_remove { $apps: }

}

node 'foo' {
class { 'app::package::cleaner': stage => 'cleanup' }
}
Unfortunately, this results in a dependency cycle. It appears that putting the Package[$title] resource reference in defined() actually invokes an implicit dependency between my cleanup helper resource in the cleanup stage and the original Package resource in the main stage.
Augeas[redacted] => Service[iptables] => Class[Iptables] => Stage[main] => Stage[cleanup] => Class[App::Package::Cleaner] => App::Package::Cleaner::Check_and_remove[package-434] => Package[package-434] => Exec[app-graceful-restart] => Class[App] => Stage[main]
Is this implicit dependency expected behaviour or am I doing something Bad(tm)?

Tom

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Jcbollinger at Jun 4, 2013 at 3:03 pm

    On Tuesday, June 4, 2013 1:22:08 AM UTC-5, Tom Lanyon wrote:
    I'm testing a 'cleanup' stage which runs after Stage[main] and removes a
    bunch of package resources.

    To do this, I tried a simple check of defined(Package[<foo>]) combined
    with a custom facter fact (called 'app_packages'):
    class app::package::cleaner {

    define check_and_remove {
    if !defined(Package[$title]) {
    package { $title:
    ensure => absent
    }
    }
    }

    $apps = split($::app_packages, ',')
    check_and_remove { $apps: }

    }

    node 'foo' {
    class { 'app::package::cleaner': stage => 'cleanup' }
    }
    Unfortunately, this results in a dependency cycle. It appears that
    putting the Package[$title] resource reference in defined() actually
    invokes an implicit dependency between my cleanup helper resource in the
    cleanup stage and the original Package resource in the main stage.
    Augeas[redacted] => Service[iptables] => Class[Iptables] => Stage[main]
    => Stage[cleanup] => Class[App::Package::Cleaner] =>
    App::Package::Cleaner::Check_and_remove[package-434] =>
    Package[package-434] => Exec[app-graceful-restart] => Class[App] =>
    Stage[main]
    Does it do that when Package[package-434] is already declared elsewhere, or
    only when it is not?


    Is this implicit dependency expected behaviour or am I doing something
    Bad(tm)?
    Both.

    Supposing that the target package is not declared elsewhere (so that the
    !defined() condition is true) the definition will declare the package
    itself to ensure it absent, and in that case you would expect a
    relationship between the defined-type instance and the resource declared by
    it. If elsewhere you have specific references to that package, applicable
    resource parameter defaults, or collectors that will match that package,
    then you can get relationships with it that are not evident from the
    defined type body.

    On the other hand, defined() is evil. Do not use it. Ever. I usually
    attribute its malignancy to the parse-order dependency it inherently
    creates -- which is indeed a serious problem -- but in this case I think
    trying to use it to approach your problem it has also obfuscated your
    manifests enough to confuse you about the scope and nature of some of your
    other declarations.

    Instead of using defined(), you can apply logic farther upstream to make
    the correct declaration in the first (one) place or to apply resource
    parameter overrides to the correct resources. Alternatively, you can
    simply determine by other means what packages need to be ensured absent,
    such as by filtering a list of possible packages against a list of packages
    that are supposed to be installed. Some of those options may still
    susceptible to the problem you observed, however, if relevant relationships
    spring from declarations elsewhere, as I described they may do.

    For the record, however, no order-of-application relationship should be
    implied by the reference itself. Therefore, when the referenced Package is
    declared elsewhere (so that the !defined() condition is false), there
    should be a relationship between
    App::Package::Cleaner::Check_and_remove[foo] and Package[foo] only if that
    relationship is declared somewhere else.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Tom Lanyon at Jun 4, 2013 at 11:02 pm
    Hi John,

    Thanks for the reply.
    On 05/06/2013, at 12:33 AM, jcbollinger wrote:
    On Tuesday, June 4, 2013 1:22:08 AM UTC-5, Tom Lanyon wrote:
    I'm testing a 'cleanup' stage which runs after Stage[main] and removes a bunch of package resources.

    To do this, I tried a simple check of defined(Package[<foo>]) combined with a custom facter fact (called 'app_packages'):
    class app::package::cleaner {

    define check_and_remove {
    if !defined(Package[$title]) {
    package { $title:
    ensure => absent
    }
    }
    }

    $apps = split($::app_packages, ',')
    check_and_remove { $apps: }

    }

    node 'foo' {
    class { 'app::package::cleaner': stage => 'cleanup' }
    }
    Unfortunately, this results in a dependency cycle. It appears that putting the Package[$title] resource reference in defined() actually invokes an implicit dependency between my cleanup helper resource in the cleanup stage and the original Package resource in the main stage.
    Augeas[redacted] => Service[iptables] => Class[Iptables] => Stage[main] => Stage[cleanup] => Class[App::Package::Cleaner] => App::Package::Cleaner::Check_and_remove[package-434] => Package[package-434] => Exec[app-graceful-restart] => Class[App] => Stage[main]
    Does it do that when Package[package-434] is already declared elsewhere, or only when it is not?
    Sorry, I should have been clearer that this occurs when Package[package-434] IS declared elsewhere. "!defined(Package[package-434])" therefore is false, so just by referencing the existing declaration within the defined() call it seems to incite an implicit dependency.

    Is this implicit dependency expected behaviour or am I doing something Bad(tm)?
    Both.

    Supposing that the target package is not declared elsewhere (so that the !defined() condition is true) the definition will declare the package itself to ensure it absent, and in that case you would expect a relationship between the defined-type instance and the resource declared by it. If elsewhere you have specific references to that package, applicable resource parameter defaults, or collectors that will match that package, then you can get relationships with it that are not evident from the defined type body.

    On the other hand, defined() is evil. Do not use it. Ever.
    I had this discussion with someone on #puppet IRC earlier and they ended up with "Oh, in your case, defined() is probably actually what you want."
    I usually attribute its malignancy to the parse-order dependency it inherently creates -- which is indeed a serious problem -- but in this case I think trying to use it to approach your problem it has also obfuscated your manifests enough to confuse you about the scope and nature of some of your other declarations.

    Instead of using defined(), you can apply logic farther upstream to make the correct declaration in the first (one) place or to apply resource parameter overrides to the correct resources. Alternatively, you can simply determine by other means what packages need to be ensured absent, such as by filtering a list of possible packages against a list of packages that are supposed to be installed. Some of those options may still susceptible to the problem you observed, however, if relevant relationships spring from declarations elsewhere, as I described they may do.
    I've tried this other ways, but here's an example of why farther upstream logic doesn't work:

    define myapp ($requested_package){

       package { $requested_package:
         ensure => present
       }

       define package_cleanup {
         $installed_package = $title

         if $installed_package != $requested_package {
           package { $installed_package:
             ensure => purged
           }
         }
       }

       # assuming a facter fact named 'installed_packages'
       package_cleanup { split($::installed_packages, ','): }
    }

    # now in the case of:
    # $::installed_packages = 'one,two,three'
    # with:
    myapp { 'oneA': requested_package => 'one' }
    myapp { 'twoA': requested_package => 'two' }
    myapp { 'oneB': requested_package => 'one' }

    # we'd end up with package conflicts because
    # Myapp[oneA] will define Package[one] (present)
    # then define Package[two], Package[three] (absent),
    # and Myapp[twoA] will try and define Package[two]
    # (present) and fail with a non-uniqueness error.

    I don't see how this is doable without defined() or some other check of the catalog to see what packages are "needed" elsewhere. Do you have any suggestions?

    For the record, however, no order-of-application relationship should be implied by the reference itself. Therefore, when the referenced Package is declared elsewhere (so that the !defined() condition is false), there should be a relationship between App::Package::Cleaner::Check_and_remove[foo] and Package[foo] only if that relationship is declared somewhere else.
    I'd hoped that using a Stage to run after everything else would sort this all out, but I'm now not sure this is correct and also see your earlier post about never using stages [with this and "defined() is evil", which pieces of Puppet /can/ we use safely?].

    Thanks,
    Tom

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jcbollinger at Jun 5, 2013 at 2:21 pm

    On Tuesday, June 4, 2013 6:01:58 PM UTC-5, Tom Lanyon wrote:
    Hi John,

    Thanks for the reply.
    On 05/06/2013, at 12:33 AM, jcbollinger wrote:
    On Tuesday, June 4, 2013 1:22:08 AM UTC-5, Tom Lanyon wrote:

    Unfortunately, this results in a dependency cycle. It appears that
    putting the Package[$title] resource reference in defined() actually
    invokes an implicit dependency between my cleanup helper resource in the
    cleanup stage and the original Package resource in the main stage.
    Augeas[redacted] => Service[iptables] => Class[Iptables] =>
    Stage[main] => Stage[cleanup] => Class[App::Package::Cleaner] =>
    App::Package::Cleaner::Check_and_remove[package-434] =>
    Package[package-434] => Exec[app-graceful-restart] => Class[App] =>
    Stage[main]
    Does it do that when Package[package-434] is already declared
    elsewhere, or only when it is not?

    Sorry, I should have been clearer that this occurs when
    Package[package-434] IS declared elsewhere.
    "!defined(Package[package-434])" therefore is false, so just by referencing
    the existing declaration within the defined() call it seems to incite an
    implicit dependency.
    If that's really what's happening then you should be able to create a
    simple test case that demonstrates it. That would be a worthy subject for
    a bug report.


    Is this implicit dependency expected behaviour or am I doing something
    Bad(tm)?
    Both.

    Supposing that the target package is not declared elsewhere (so that the
    !defined() condition is true) the definition will declare the package
    itself to ensure it absent, and in that case you would expect a
    relationship between the defined-type instance and the resource declared by
    it. If elsewhere you have specific references to that package, applicable
    resource parameter defaults, or collectors that will match that package,
    then you can get relationships with it that are not evident from the
    defined type body.
    On the other hand, defined() is evil. Do not use it. Ever.
    I had this discussion with someone on #puppet IRC earlier and they ended
    up with "Oh, in your case, defined() is probably actually what you want."

    No. defined() is never what you want. It may at times seem expedient, but
    it's bad news every time.


    I usually attribute its malignancy to the parse-order dependency it
    inherently creates -- which is indeed a serious problem -- but in this case
    I think trying to use it to approach your problem it has also obfuscated
    your manifests enough to confuse you about the scope and nature of some of
    your other declarations.
    Instead of using defined(), you can apply logic farther upstream to make
    the correct declaration in the first (one) place or to apply resource
    parameter overrides to the correct resources. Alternatively, you can
    simply determine by other means what packages need to be ensured absent,
    such as by filtering a list of possible packages against a list of packages
    that are supposed to be installed. Some of those options may still
    susceptible to the problem you observed, however, if relevant relationships
    spring from declarations elsewhere, as I described they may do.

    I've tried this other ways, but here's an example of why farther upstream
    logic doesn't work:

    define myapp ($requested_package){

    package { $requested_package:
    ensure => present
    }

    define package_cleanup {
    $installed_package = $title

    if $installed_package != $requested_package {
    package { $installed_package:
    ensure => purged
    }
    }
    }

    # assuming a facter fact named 'installed_packages'
    package_cleanup { split($::installed_packages, ','): }
    }

    I don't much like that general approach in the first place on account of
    the $requested_package parameter. That you encounter difficulty when you
    try something a bit dodgy should not be surprising.


    # now in the case of:
    # $::installed_packages = 'one,two,three'
    # with:
    myapp { 'oneA': requested_package => 'one' }
    myapp { 'twoA': requested_package => 'two' }
    myapp { 'oneB': requested_package => 'one' }

    # we'd end up with package conflicts because
    # Myapp[oneA] will define Package[one] (present)
    # then define Package[two], Package[three] (absent),
    # and Myapp[twoA] will try and define Package[two]
    # (present) and fail with a non-uniqueness error.

    I don't see how this is doable without defined() or some other check of
    the catalog to see what packages are "needed" elsewhere. Do you have any
    suggestions?
    In fact, despite my dissatisfaction with your approach, you can indeed do
    this without defined(), and without even disrupting your current structure
    very much. Here's one way I think would work:

    # This class ensures all known app packages are
    # by default purged
    class app::packages {
         $apps = split($::app_packages, ',')
         package { $apps:
             ensure => 'purged'
         }
    }

    # Overrides the requested package to be declared
    # present instead of purged.
    define app::myapp($requested_package) {
         include 'app::packages'
         Package<| title == $requested_package |> {
             ensure => 'present'
         }
    }

    # no separate package_cleanup required

    For the record, however, no order-of-application relationship should be
    implied by the reference itself. Therefore, when the referenced Package is
    declared elsewhere (so that the !defined() condition is false), there
    should be a relationship between
    App::Package::Cleaner::Check_and_remove[foo] and Package[foo] only if that
    relationship is declared somewhere else.

    I'd hoped that using a Stage to run after everything else would sort this
    all out, but I'm now not sure this is correct and also see your earlier
    post about never using stages [with this and "defined() is evil", which
    pieces of Puppet /can/ we use safely?].
    It's not uncommon for stages to cause cycles, but they will not resolve
    cycles that are already present without them, because stages only add
    relationships to those that are otherwise there. They are not evil in the
    same way that defined() is, however. Although I disfavor stages myself and
    generally do not recommend them to folks, I do not consider a manifest set
    that successfully uses them to be inherently flawed (unlike one that relies
    on defined()).

    Puppet is broad in scope and rich in features, so it should not be
    surprising that a few of its features are of questionable value. Pretty
    much any system of comparable size has such. To Puppetlabs's credit, they
    do deprecate and eventually remove features that they conclude are obsolete
    or irretrievably broken. They have floated the idea of doing this with
    defined(), in fact, and that may eventually happen.

    Other things I recommend you avoid:

        - most uses of 'import'. Bringing node definitions into site.pp is
        about the only appropriate use of 'import' that comes to mind. Most other
        potential uses are better served by the autoloader.
        - Lexical nesting of classes and/or defined types. This is partly for
        cooperating with the autoloader and partly just a matter of style, but it
        is widely considered a best practice to put each class and definition in
        its own file.
        - parameterized-style class declarations. (That is, declarations of the
        form "class { 'foo': param1 => bar }".) This one is controversial. I
        have written on it frequently in the past, so I will omit the recap unless
        you insist. Parameterized classes themselves are ok in Puppet 3, provided
        that you rely on hiera data bindings or maybe an ENC when you need to
        customize their parameters.

    John

    "Parameterized classes: the best thing since decaffeinated coffee"

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Tom Lanyon at Jun 12, 2013 at 2:15 pm
    Hi John,

    Sorry for the delayed reply.
    On 05/06/2013, at 11:51 PM, jcbollinger wrote:

    Sorry, I should have been clearer that this occurs when Package[package-434] IS declared elsewhere. "!defined(Package[package-434])" therefore is false, so just by referencing the existing declaration within the defined() call it seems to incite an implicit dependency.
    If that's really what's happening then you should be able to create a simple test case that demonstrates it. That would be a worthy subject for a bug report.
    I'll see what I can do.
    Is this implicit dependency expected behaviour or am I doing something Bad(tm)?
    Both.

    Supposing that the target package is not declared elsewhere (so that the !defined() condition is true) the definition will declare the package itself to ensure it absent, and in that case you would expect a relationship between the defined-type instance and the resource declared by it. If elsewhere you have specific references to that package, applicable resource parameter defaults, or collectors that will match that package, then you can get relationships with it that are not evident from the defined type body.

    On the other hand, defined() is evil. Do not use it. Ever.
    I had this discussion with someone on #puppet IRC earlier and they ended up with "Oh, in your case, defined() is probably actually what you want."
    No. defined() is never what you want. It may at times seem expedient, but it's bad news every time.
    OK, understood.
    define myapp ($requested_package){

    package { $requested_package:
    ensure => present
    }

    define package_cleanup {
    $installed_package = $title

    if $installed_package != $requested_package {
    package { $installed_package:
    ensure => purged
    }
    }
    }

    # assuming a facter fact named 'installed_packages'
    package_cleanup { split($::installed_packages, ','): }
    }
    I don't much like that general approach in the first place on account of the $requested_package parameter. That you encounter difficulty when you try something a bit dodgy should not be surprising.
    Can you explain this further so I can understand the issue?


    In fact, despite my dissatisfaction with your approach, you can indeed do this without defined(), and without even disrupting your current structure very much. Here's one way I think would work:

    # This class ensures all known app packages are
    # by default purged
    class app::packages {
    $apps = split($::app_packages, ',')
    package { $apps:
    ensure => 'purged'
    }
    }

    # Overrides the requested package to be declared
    # present instead of purged.
    define app::myapp($requested_package) {
    include 'app::packages'
    Package<| title == $requested_package |> {
    ensure => 'present'
    }
    }

    # no separate package_cleanup required

    OK, I wondered whether we could do something like this however - forgive my naivety - I still can't see how this could be a complete solution without something like defined().

    As an example... your above snippet works fine to ensure already installed packages remain installed, but what if we wanted to install a brand new version of app::myapp? Because a 'package' resource with title $requested_package does not yet exist, the Package<||> collector matches no resources and the new package is not installed. The only solution that I can come up with is to check whether such a resource is already defined and, if not, define one.

    Your guidance is appreciated.

    Tom

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jcbollinger at Jun 13, 2013 at 2:36 pm

    On Wednesday, June 12, 2013 9:15:22 AM UTC-5, Tom Lanyon wrote:
    On 05/06/2013, at 11:51 PM, jcbollinger [...] wrote:
    I don't much like that general approach in the first place on account of
    the $requested_package parameter. That you encounter difficulty when you
    try something a bit dodgy should not be surprising.

    Can you explain this further so I can understand the issue?
    Initially, it was mostly a gut feeling. After having had time to step back
    from the issue and return to it fresh, I think it's a combination of
    things, mostly revolving around what you're actually modeling, and how
    you're modeling it.

    Basically, the 'myapp' definition represents one package chosen from a list
    of mutually exclusive packages. If that's all it is, then its name is
    misleading -- it should be more generic -- and it should probably take the
    exclusive list as a second parameter. On the other hand, if it is indeed
    supposed to be something specific, then it doesn't take much advantage of
    that. In particular -- and here's where my previous comment came from --
    if it supposed to represent something specific to your application, then
    why doesn't it know anything about the application's package names?

    Also, if the point is supposed to be that only one version of the
    application can be installed at a time, and the definition is specific to
    that application, then it really ought to be a class instead.


    In fact, despite my dissatisfaction with your approach, you can indeed
    do this without defined(), and without even disrupting your current
    structure very much. Here's one way I think would work:
    # This class ensures all known app packages are
    # by default purged
    class app::packages {
    $apps = split($::app_packages, ',')
    package { $apps:
    ensure => 'purged'
    }
    }

    # Overrides the requested package to be declared
    # present instead of purged.
    define app::myapp($requested_package) {
    include 'app::packages'
    Package<| title == $requested_package |> {
    ensure => 'present'
    }
    }

    # no separate package_cleanup required

    OK, I wondered whether we could do something like this however - forgive
    my naivety - I still can't see how this could be a complete solution
    without something like defined().

    As an example... your above snippet works fine to ensure already installed
    packages remain installed, but what if we wanted to install a brand new
    version of app::myapp? Because a 'package' resource with title
    $requested_package does not yet exist, the Package<||> collector matches no
    resources and the new package is not installed. The only solution that I
    can come up with is to check whether such a resource is already defined
    and, if not, define one.
    You appear to have a serious misunderstanding. Resource collectors have no
    direct relationship with or dependency on which resources are already
    installed on the target system. They work exclusively with resource *
    declarations* in your manifests, and they do so at catalog compilation
    time. Moreover, they are independent of parse order (though the example
    anyway ensures a parse order that would work if collectors were parse-order
    dependent).

    Explanation of the example:

        - class app::packages declares all of the possible application packages,
        specifying the intended state for each one as 'purged'. If that is the
        only thing applied to the target node then it will cause the removal of
        each and every one of those packages that is installed. ('purged' is
        stronger than 'absent'. The former is more sure to remove the specified
        package, but the latter takes care to avoid causing any other packages to
        be removed, and therefore fails if any other package depends on the target
        package.) It is necessary that the list of possible packages include every
        one that you may want to have installed, so it needs to be updated whenever
        you introduce a new one that you want to manage. That was already a
        requirement for you, however, whether you recognized it or not.
        - Resources of defined type app::myapp ensure that class app::packages
        is declared on the target node (by declaring it itself)
        - Resources of defined type app::myapp override the declaration of the
        target package (made by class app::packages) so that its target state is
        'present' instead of 'purged'. This will cause it to be installed if it is
        not already present, and will avoid removing it if it is present.
        - To use the example, simply declare one or more instances of app::myapp
        for the target node, either in a node block or in some class assigned to
        the node. You may also declare class app::packages directly for any node,
        whether or not they declare any app::myapp instances. That is useful if
        you have nodes on which you want to ensure that no version of the
        application is installed.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Tom Lanyon at Jun 14, 2013 at 7:02 am
    Hi John
    On 14/06/2013, at 12:06 AM, jcbollinger wrote:

    On Wednesday, June 12, 2013 9:15:22 AM UTC-5, Tom Lanyon wrote:
    On 05/06/2013, at 11:51 PM, jcbollinger [...] wrote:
    I don't much like that general approach in the first place on account of the $requested_package parameter. That you encounter difficulty when you try something a bit dodgy should not be surprising.
    Can you explain this further so I can understand the issue?
    Initially, it was mostly a gut feeling. After having had time to step back from the issue and return to it fresh, I think it's a combination of things, mostly revolving around what you're actually modeling, and how you're modeling it.

    Basically, the 'myapp' definition represents one package chosen from a list of mutually exclusive packages. If that's all it is, then its name is misleading -- it should be more generic -- and it should probably take the exclusive list as a second parameter. On the other hand, if it is indeed supposed to be something specific, then it doesn't take much advantage of that. In particular -- and here's where my previous comment came from -- if it supposed to represent something specific to your application, then why doesn't it know anything about the application's package names?

    Also, if the point is supposed to be that only one version of the application can be installed at a time, and the definition is specific to that application, then it really ought to be a class instead.
    Alas, the whole intention of this is that multiple versions of the package must be installed at the same time. This is where the problem lies, in that we have no way to clean up old unused versions once they're no longer needed.

    We have instances of an application MyApp, which utilise a shared Package in many to one relationship:

         Package[one] ___ MyApp[app1]

         Package[two] ___ MyApp[app2]
                     \___ MyApp[app3]

         Package[three] _ MyApp[app4]
                       \_ MyApp[app5]
                       \_ MyApp[app6]

    In the above example, if we upgraded MyApp[app2] and MyApp[app3] to Package[three], we'd still have Package[two] installed on all of the hosts but no MyApp instances would be using it. This is the unused Package we're trying to clean up with Puppet (there's no longer a MyApp resource definition which references Package[two]).

    There is no mutually exclusive packages or any other such conflicts.

    In fact, despite my dissatisfaction with your approach, you can indeed do this without defined(), and without even disrupting your current structure very much. Here's one way I think would work:

    # This class ensures all known app packages are
    # by default purged
    class app::packages {
    $apps = split($::app_packages, ',')
    package { $apps:
    ensure => 'purged'
    }
    }

    # Overrides the requested package to be declared
    # present instead of purged.
    define app::myapp($requested_package) {
    include 'app::packages'
    Package<| title == $requested_package |> {
    ensure => 'present'
    }
    }

    # no separate package_cleanup required

    OK, I wondered whether we could do something like this however - forgive my naivety - I still can't see how this could be a complete solution without something like defined().

    As an example... your above snippet works fine to ensure already installed packages remain installed, but what if we wanted to install a brand new version of app::myapp? Because a 'package' resource with title $requested_package does not yet exist, the Package<||> collector matches no resources and the new package is not installed. The only solution that I can come up with is to check whether such a resource is already defined and, if not, define one.
    You appear to have a serious misunderstanding. Resource collectors have no direct relationship with or dependency on which resources are already installed on the target system. They work exclusively with resource declarations in your manifests, and they do so at catalog compilation time. Moreover, they are independent of parse order (though the example anyway ensures a parse order that would work if collectors were parse-order dependent).
    No, I understand this.
    Explanation of the example:
    • class app::packages declares all of the possible application packages, specifying the intended state for each one as 'purged'. If that is the only thing applied to the target node then it will cause the removal of each and every one of those packages that is installed. ('purged' is stronger than 'absent'. The former is more sure to remove the specified package, but the latter takes care to avoid causing any other packages to be removed, and therefore fails if any other package depends on the target package.) It is necessary that the list of possible packages include every one that you may want to have installed, so it needs to be updated whenever you introduce a new one that you want to manage. That was already a requirement for you, however, whether you recognized it or not.
    This is where it fell apart for me before, I didn't realise that you assumed $::app_packages was every possible package; in my first post I'd specified that $::app_packages was a fact generated on each node from the list of *currently installed* packages, so it contains all of the packages which have been previously been installed but not packages which may be installed in the future.

    I think it's going to be a managerial nightmare to update some such list of all possible packages, so might look for an alternate solution. I don't think Puppet really meets our needs here, unless there's a way we can programmatically parse out all of the versions of the package specified in our manifests to include them in the $::app_packages (so they get defined as 'purged') before building the rest of the catalog which would override them as 'present'. This feels like something that might be possible with the use of some combination of defined() or stages, but I'm keen to not run up against any caveats you've mentioned with those features.
    • Resources of defined type app::myapp ensure that class app::packages is declared on the target node (by declaring it itself)
    • Resources of defined type app::myapp override the declaration of the target package (made by class app::packages) so that its target state is 'present' instead of 'purged'. This will cause it to be installed if it is not already present, and will avoid removing it if it is present.
    • To use the example, simply declare one or more instances of app::myapp for the target node, either in a node block or in some class assigned to the node. You may also declare class app::packages directly for any node, whether or not they declare any app::myapp instances. That is useful if you have nodes on which you want to ensure that no version of the application is installed.

    John
    Thanks for your help.

    Cheers,
    Tom

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jcbollinger at Jun 14, 2013 at 2:14 pm

    On Friday, June 14, 2013 2:01:50 AM UTC-5, Tom Lanyon wrote:
    Hi John

    On 14/06/2013, at 12:06 AM, jcbollinger [...] wrote:

    This is where it fell apart for me before, I didn't realise that you
    assumed $::app_packages was every possible package; in my first post I'd
    specified that $::app_packages was a fact generated on each node from the
    list of *currently installed* packages, so it contains all of the packages
    which have been previously been installed but not packages which may be
    installed in the future.

    My apologies for the confusion.


    I think it's going to be a managerial nightmare to update some such list
    of all possible packages, so might look for an alternate solution. I don't
    think Puppet really meets our needs here, unless there's a way we can
    programmatically parse out all of the versions of the package specified in
    our manifests to include them in the $::app_packages (so they get defined
    as 'purged') before building the rest of the catalog which would override
    them as 'present'.


    I think you're looking at it the wrong way around. It is a fundamental
    problem for you if you don't know what all the packages you (may) want to
    manage are. This is not information that ought to be gleaned from your
    manifest set, nor even information that you ought to be collecting
    dynamically from clients, but rather information which you ought to have a
    priori. How you store it and keep it updated is a separate matter that I
    have heretofore avoided, but Puppet does have a capable external data
    interface, Hiera, that could reasonably be applied to the problem.

    Nevertheless, if it is not feasible for you to determine up front all the
    packages your apps may be using or may want to use, then you can get by
    with a smaller list, consisting of just the union of those packages
    currently installed and those you want to be installed. You already have
    those data at hand, and it is possible to merge them into the needed list.
    For example, rewriting my app::packages this way should help:

    class app::packages {
         $apps = unique(concat(split($::app_packages, ','), <wanted_packages>))
         package { $apps:
             ensure => 'purged'
         }
    }

    That might be paired with app::myapp declarations there or elsewhere of the
    form

    app::myapp { <wanted_packages>: }

    The <wanted_packages> represents an array of the names of the packages you
    want installed, which could be a variable (including a parameter) of some
    class in the 'app' module (even app::packages itself), or might be a
    function call (e.g. to an external data service) or maybe something else.
    The unique() and concat() functions are available from PuppetLabs's popular
    "stdlib" add-in module.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jcbollinger at Jun 14, 2013 at 2:28 pm

    On Friday, June 14, 2013 9:14:07 AM UTC-5, jcbollinger wrote:
    Nevertheless, if it is not feasible for you to determine up front all the
    packages your apps may be using or may want to use, then you can get by
    with a smaller list, consisting of just the union of those packages
    currently installed and those you want to be installed.

    Do note, by the way, that this general approach of relying on a dynamic,
    client-supplied list of packages that may need to be removed is inherently
    more risky than one based on a centrally managed, static list of packages.
    If your custom fact malfunctions (or is subverted) then you could end up
    removing packages you did not intend to remove, including packages
    essential to the client's operation.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedJun 4, '13 at 6:23a
activeJun 14, '13 at 2:28p
posts9
users2
websitepuppetlabs.com

2 users in discussion

Jcbollinger: 5 posts Tom Lanyon: 4 posts

People

Translate

site design / logo © 2022 Grokbase