FAQ
Hi,

I'm trying to build a module for haproxy which fetches all the
configuration data from Hiera to populate the haproxy config file. I've run
into a number of issues though when I try to use hashes. Ideally, I want to
use something like this:

haproxy_listeners :
"cluster1" :
ip : '192.168.0.2'
port : '80'
servers :
"server1" :
ip : '192.168.0.3'
port : '8080'

So a hash of clusters with each cluster containing a nested hash of
servers. Is this possible with Hiera/ERB? It's easy enough to iterate over
the first hash, but I can't work out how to extract the contents of the
nested hash. Or I might just be approaching this in entirely the wrong
way... Any help would be much appreciated.

Thanks,

Andy

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/vtr8gCCwuBsJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Search Discussions

  • Llowder at Jan 4, 2013 at 3:37 pm

    On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote:
    Hi,

    I'm trying to build a module for haproxy which fetches all the
    configuration data from Hiera to populate the haproxy config file. I've run
    into a number of issues though when I try to use hashes. Ideally, I want to
    use something like this:

    haproxy_listeners :
    "cluster1" :
    ip : '192.168.0.2'
    port : '80'
    servers :
    "server1" :
    ip : '192.168.0.3'
    port : '8080'

    So a hash of clusters with each cluster containing a nested hash of
    servers. Is this possible with Hiera/ERB? It's easy enough to iterate over
    the first hash, but I can't work out how to extract the contents of the
    nested hash. Or I might just be approaching this in entirely the wrong
    way... Any help would be much appreciated.
    I haven't used the function myself, but this looks like it would be a good
    case for a define + create_resources(), which I think is part of stdlib.
    You might need to restructure the hashes slightly, but I think that will be
    the best approach.


    Thanks,

    Andy
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/DPkuzR8Q8soJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Wolf Noble at Jan 4, 2013 at 4:38 pm
    not sure if this is the "best" way to do this, but, taking from a different module I have:


    hiera yaml:

    snmp_filesystems:
    root: {
    mountpoint: '/',
    threshold: '10%'
    }
    snmp_collector:
    test: {
    ip: '1.2.3.4',
    group: 'readonly',
    community: 'somecommunity'
    }

    class:

    class snmp::params(
    filesystems = hiera(snmp_filesystems, undef),
    collector = hiera(snmp_collector, undef)
    {
    #…
    }

    template:

    <%
    collector = scope.lookupvar('snmp::params::collector')
    filesystems = scope.lookupvar('snmp::params::filesystems')
    -%>

    <% collector.each_pair do |key, hash| -%>
    group <%=hash['group']-%> v2c <%=key%>
    <%end-%>

    # Filesystem monitoring enabled
    <% filesystems.each_pair do |key, hash| -%>
    # <%=key%>
    disk <%=hash['mountpoint']-%> <%=hash['threshold']%>
    <%end-%>




    does that help at all?


    Wolf Noble
    UNIX Team Lead
    Datapipe Managed IT Services
    1.201.792.4847 (international) x2910
    1.888.749.5821 (toll free) x2910



    On Jan 4, 2013, at 9:37 AM, llowder wrote:



    On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote:
    Hi,

    I'm trying to build a module for haproxy which fetches all the configuration data from Hiera to populate the haproxy config file. I've run into a number of issues though when I try to use hashes. Ideally, I want to use something like this:

    haproxy_listeners :
    "cluster1" :
    ip : '192.168.0.2'
    port : '80'
    servers :
    "server1" :
    ip : '192.168.0.3'
    port : '8080'

    So a hash of clusters with each cluster containing a nested hash of servers. Is this possible with Hiera/ERB? It's easy enough to iterate over the first hash, but I can't work out how to extract the contents of the nested hash. Or I might just be approaching this in entirely the wrong way... Any help would be much appreciated.


    I haven't used the function myself, but this looks like it would be a good case for a define + create_resources(), which I think is part of stdlib. You might need to restructure the hashes slightly, but I think that will be the best approach.


    Thanks,

    Andy

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/DPkuzR8Q8soJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

    ________________________________

    This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/legal/email_disclaimer/ for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Gavin Williams at Jan 4, 2013 at 4:47 pm
    I'm with Andy on this one... I'm doing something very similar with my
    NetApp volume provider (https://github.com/fatmcgav/fatmcgav-netapp/).

    I've created a define with the following contents:
    define util::netapp::volume (
    $ensure = present,
    $size,
    $aggr = 'aggr1',
    $snapresv = 0,
    $autoincrement = true,
    $snapschedule = {"minutes" => 0, "hours" => 0, "days" => 0, "weeks"
    => 0}
    ) {

    netapp_notify {"volume_define_${name}":
    message => "Processing Volume ${name}",
    }
    ->
    netapp_volume { "v_${name}":
    ensure => $ensure,
    initsize => $size,
    aggregate => $aggr,
    spaceres => "none",
    snapreserve => $snapresv,
    autoincrement => $autoincrement,
    options => {'convert_ucode' => 'on', 'no_atime_update' =>
    'on', 'try_first' => 'volume_grow'},
    snapschedule => $snapschedule
    }
    ->
    netapp_qtree { "q_${name}":
    ensure => $ensure,
    volume => "v_${name}"
    }
    ->
    netapp_export { "/vol/v_${name}/q_${name}":
    ensure => $ensure,
    persistent => true
    }

    }

    I've added a default hash to 'snapschedule' in the options list, but that
    can be over-ridden from the Hiera data.

    Then use the following to pull the data from hiera and call the define:
    create_resources( util::netapp::volume, hiera('volumes') )

    'Volumes' in hiera yaml looks like:
    volumes:
    vol1:
    ensure: present
    size: '500m'
    vol2:
    ensure: present
    size: '20g'
    snapschedule:
    minutes: 0
    hours: 36
    days: 0
    weeks: 0

    You can also use the 'hiera' command to test your yaml structure:
    $ hiera -c hiera.yaml volumes clientcert=act-star-nactl01
    {"vol1"=>{"ensure"=>"present", "size"=>"500m"},
    "vol2"=>{"ensure"=>"present", "size"=>"20g", "snapschedule"=>{"days"=>0,
    "weeks"=>0, "hours"=>36, "minutes"=>0}}}

    As you can see from the above output, *snapschedule* for *vol2* is a nested
    hash. This assumes that your resource provider can support hashes on the
    relevant param/property ;)

    HTH

    Gav
    On Friday, 4 January 2013 15:37:25 UTC, llowder wrote:


    On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote:

    Hi,

    I'm trying to build a module for haproxy which fetches all the
    configuration data from Hiera to populate the haproxy config file. I've run
    into a number of issues though when I try to use hashes. Ideally, I want to
    use something like this:

    haproxy_listeners :
    "cluster1" :
    ip : '192.168.0.2'
    port : '80'
    servers :
    "server1" :
    ip : '192.168.0.3'
    port : '8080'

    So a hash of clusters with each cluster containing a nested hash of
    servers. Is this possible with Hiera/ERB? It's easy enough to iterate over
    the first hash, but I can't work out how to extract the contents of the
    nested hash. Or I might just be approaching this in entirely the wrong
    way... Any help would be much appreciated.
    I haven't used the function myself, but this looks like it would be a good
    case for a define + create_resources(), which I think is part of stdlib.
    You might need to restructure the hashes slightly, but I think that will be
    the best approach.


    Thanks,

    Andy
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/UjWLd-yp7iIJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Andy Taylor at Jan 4, 2013 at 5:11 pm
    Thanks for your suggestions guys. I did consider using create_resource, but
    don't see how I can when I'm trying to apply this Hiera data to a single
    file. To expand on my initial post, what I need to do is create multiple
    config blocks within one file resource. So this Hiera data:

    haproxy_listeners :
    "cluster1" :
    ip : '192.168.0.2'
    port : '80'
    servers :
    "server1" :
    ip : '192.168.0.3'
    port : '8080'
    "server2" :
    ip : '192.168.0.4'
    port : '8080'
    "cluster2" :
    ip : '192.168.0.5'
    port : '80'
    servers :
    "server3"
    ip : '192.168.0.6'
    port : '8080'
    "server4"
    ip : '192.168.0.7'
    port : '8080'

    will result in this being generated in the haproxy config file:

    listen cluster1 192.168.0.2:80
    server server1 192.168.0.3:8080
    server server2 192.168.0.4:8080

    listen cluster2 192.168.0.5:80
    server server3 192.168.0.6:8080
    server server4 192.168.0.7:8080

    So I don't see how create_resources can handle this, as that's for creating
    multiple Puppet resources, as opposed to multiple blocks within a single
    file. The only alternative I can think of at the moment is using
    create_resources with a define which utilizes Augeas, but I don't know how
    well that will work.

    Thanks,

    Andy
    On Friday, 4 January 2013 16:47:13 UTC, Gavin Williams wrote:

    I'm with Andy on this one... I'm doing something very similar with my
    NetApp volume provider (https://github.com/fatmcgav/fatmcgav-netapp/).

    I've created a define with the following contents:
    define util::netapp::volume (
    $ensure = present,
    $size,
    $aggr = 'aggr1',
    $snapresv = 0,
    $autoincrement = true,
    $snapschedule = {"minutes" => 0, "hours" => 0, "days" => 0,
    "weeks" => 0}
    ) {

    netapp_notify {"volume_define_${name}":
    message => "Processing Volume ${name}",
    }
    ->
    netapp_volume { "v_${name}":
    ensure => $ensure,
    initsize => $size,
    aggregate => $aggr,
    spaceres => "none",
    snapreserve => $snapresv,
    autoincrement => $autoincrement,
    options => {'convert_ucode' => 'on', 'no_atime_update' =>
    'on', 'try_first' => 'volume_grow'},
    snapschedule => $snapschedule
    }
    ->
    netapp_qtree { "q_${name}":
    ensure => $ensure,
    volume => "v_${name}"
    }
    ->
    netapp_export { "/vol/v_${name}/q_${name}":
    ensure => $ensure,
    persistent => true
    }

    }

    I've added a default hash to 'snapschedule' in the options list, but that
    can be over-ridden from the Hiera data.

    Then use the following to pull the data from hiera and call the define:
    create_resources( util::netapp::volume, hiera('volumes') )

    'Volumes' in hiera yaml looks like:
    volumes:
    vol1:
    ensure: present
    size: '500m'
    vol2:
    ensure: present
    size: '20g'
    snapschedule:
    minutes: 0
    hours: 36
    days: 0
    weeks: 0

    You can also use the 'hiera' command to test your yaml structure:
    $ hiera -c hiera.yaml volumes clientcert=act-star-nactl01
    {"vol1"=>{"ensure"=>"present", "size"=>"500m"},
    "vol2"=>{"ensure"=>"present", "size"=>"20g", "snapschedule"=>{"days"=>0,
    "weeks"=>0, "hours"=>36, "minutes"=>0}}}

    As you can see from the above output, *snapschedule* for *vol2* is a
    nested hash. This assumes that your resource provider can support hashes on
    the relevant param/property ;)

    HTH

    Gav
    On Friday, 4 January 2013 15:37:25 UTC, llowder wrote:


    On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote:

    Hi,

    I'm trying to build a module for haproxy which fetches all the
    configuration data from Hiera to populate the haproxy config file. I've run
    into a number of issues though when I try to use hashes. Ideally, I want to
    use something like this:

    haproxy_listeners :
    "cluster1" :
    ip : '192.168.0.2'
    port : '80'
    servers :
    "server1" :
    ip : '192.168.0.3'
    port : '8080'

    So a hash of clusters with each cluster containing a nested hash of
    servers. Is this possible with Hiera/ERB? It's easy enough to iterate over
    the first hash, but I can't work out how to extract the contents of the
    nested hash. Or I might just be approaching this in entirely the wrong
    way... Any help would be much appreciated.
    I haven't used the function myself, but this looks like it would be a
    good case for a define + create_resources(), which I think is part of
    stdlib. You might need to restructure the hashes slightly, but I think that
    will be the best approach.


    Thanks,

    Andy
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/W3UBJBXuT24J.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Steven Nemetz at Jan 4, 2013 at 6:02 pm
    I'm using stdlib to help with this $listeners = hiera('haproxy_listeners', undef)$listener_keys = keys($listeners) then pass $listener_keys to a define to create all the instances
    The entre hash is in memory ($listeners) and the define will have the key it is working on ($name)So you can access anything in the data structure to build your resources.
    Steven
    Date: Fri, 4 Jan 2013 09:05:15 -0800
    From: andytayloruk@gmail.com
    To: puppet-users@googlegroups.com
    Subject: [Puppet Users] Re: Hiera hashes and arrays in ERB templates

    Thanks for your suggestions guys. I did consider using create_resource, but don't see how I can when I'm trying to apply this Hiera data to a single file. To expand on my initial post, what I need to do is create multiple config blocks within one file resource. So this Hiera data:
    haproxy_listeners : "cluster1" : ip : '192.168.0.2' port : '80' servers : "server1" : ip : '192.168.0.3' port : '8080' "server2" : ip : '192.168.0.4' port : '8080' "cluster2" : ip : '192.168.0.5' port : '80' servers : "server3" ip : '192.168.0.6' port : '8080' "server4" ip : '192.168.0.7' port : '8080'
    will result in this being generated in the haproxy config file:
    listen cluster1 192.168.0.2:80 server server1 192.168.0.3:8080 server server2 192.168.0.4:8080
    listen cluster2 192.168.0.5:80 server server3 192.168.0.6:8080 server server4 192.168.0.7:8080
    So I don't see how create_resources can handle this, as that's for creating multiple Puppet resources, as opposed to multiple blocks within a single file. The only alternative I can think of at the moment is using create_resources with a define which utilizes Augeas, but I don't know how well that will work.
    Thanks,
    Andy
    On Friday, 4 January 2013 16:47:13 UTC, Gavin Williams wrote:I'm with Andy on this one... I'm doing something very similar with my NetApp volume provider (https://github.com/fatmcgav/fatmcgav-netapp/).

    I've created a define with the following contents:
    define util::netapp::volume (
    $ensure = present,
    $size,
    $aggr = 'aggr1',
    $snapresv = 0,
    $autoincrement = true,
    $snapschedule = {"minutes" => 0, "hours" => 0, "days" => 0, "weeks" => 0}
    ) {

    netapp_notify {"volume_define_${name}":
    message => "Processing Volume ${name}",
    }
    ->
    netapp_volume { "v_${name}":
    ensure => $ensure,
    initsize => $size,
    aggregate => $aggr,
    spaceres => "none",
    snapreserve => $snapresv,
    autoincrement => $autoincrement,
    options => {'convert_ucode' => 'on', 'no_atime_update' => 'on', 'try_first' => 'volume_grow'},
    snapschedule => $snapschedule
    }
    ->
    netapp_qtree { "q_${name}":
    ensure => $ensure,
    volume => "v_${name}"
    }
    ->
    netapp_export { "/vol/v_${name}/q_${name}":
    ensure => $ensure,
    persistent => true
    }

    }

    I've added a default hash to 'snapschedule' in the options list, but that can be over-ridden from the Hiera data.

    Then use the following to pull the data from hiera and call the define:
    create_resources( util::netapp::volume, hiera('volumes') )

    'Volumes' in hiera yaml looks like:
    volumes:
    vol1:
    ensure: present
    size: '500m'
    vol2:
    ensure: present
    size: '20g'
    snapschedule:
    minutes: 0
    hours: 36
    days: 0
    weeks: 0

    You can also use the 'hiera' command to test your yaml structure:
    $ hiera -c hiera.yaml volumes clientcert=act-star-nactl01
    {"vol1"=>{"ensure"=>"present", "size"=>"500m"}, "vol2"=>{"ensure"=>"present", "size"=>"20g", "snapschedule"=>{"days"=>0, "weeks"=>0, "hours"=>36, "minutes"=>0}}}

    As you can see from the above output, snapschedule for vol2 is a nested hash. This assumes that your resource provider can support hashes on the relevant param/property ;)

    HTH

    Gav

    On Friday, 4 January 2013 15:37:25 UTC, llowder wrote:

    On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote:Hi,
    I'm trying to build a module for haproxy which fetches all the configuration data from Hiera to populate the haproxy config file. I've run into a number of issues though when I try to use hashes. Ideally, I want to use something like this:
    haproxy_listeners : "cluster1" : ip : '192.168.0.2' port : '80' servers : "server1" : ip : '192.168.0.3' port : '8080'
    So a hash of clusters with each cluster containing a nested hash of servers. Is this possible with Hiera/ERB? It's easy enough to iterate over the first hash, but I can't work out how to extract the contents of the nested hash. Or I might just be approaching this in entirely the wrong way... Any help would be much appreciated.

    I haven't used the function myself, but this looks like it would be a good case for a define + create_resources(), which I think is part of stdlib. You might need to restructure the hashes slightly, but I think that will be the best approach.
    Thanks,
    Andy



    --

    You received this message because you are subscribed to the Google Groups "Puppet Users" group.

    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/W3UBJBXuT24J.

    To post to this group, send email to puppet-users@googlegroups.com.

    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.


    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.


    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jcbollinger at Jan 4, 2013 at 10:02 pm

    On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote:
    Hi,

    I'm trying to build a module for haproxy which fetches all the
    configuration data from Hiera to populate the haproxy config file. I've run
    into a number of issues though when I try to use hashes. Ideally, I want to
    use something like this:

    haproxy_listeners :
    "cluster1" :
    ip : '192.168.0.2'
    port : '80'
    servers :
    "server1" :
    ip : '192.168.0.3'
    port : '8080'

    So a hash of clusters with each cluster containing a nested hash of
    servers. Is this possible with Hiera/ERB? It's easy enough to iterate over
    the first hash, but I can't work out how to extract the contents of the
    nested hash. Or I might just be approaching this in entirely the wrong
    way... Any help would be much appreciated.
    The data can be expressed in YAML, loaded into a Puppet variable via Hiera,
    and processed in an ERB template to construct your file. In other words,
    yes, it is possible.

    May I ask, however, what the advantage is to encoding the data in YAML and
    then transcoding it into the haproxy configuration language? I mean, if
    you also use bits and pieces of that object elsewhere in your manifests
    then perhaps it makes sense, but if all you want to do with it is generate
    the config file then I don't see the point. Why not just serve up the
    config as a static or mostly-static file?

    Leaving aside the question of merits of your design,

    - Note that your example data are a quadruply-nested hash, not just a
    doubly-nested one as you say. You have a hash of clusters, each of which
    is a hash itself and contains a third hash, each of whose values is *
    another* hash.
    - The same Ruby mechanisms that allow your template to iterate and
    otherwise inspect the outer hash also work for the inner ones.
    - Perhaps, though, you are tripping over the fact that the values in
    your outer hash are of uniform type and meaning, whereas the values in the
    next-level hash are of varying type and meaning. You probably don't
    actually want to iterate the second-level hash; instead, you want to
    retrieve specific values from it by key.

    So, you might do something of this general form (not intended to resemble
    an actual haproxy configuration file):

    haproxy.conf.erb:
    ============
    <% @clusters.each_pair | cluster, properties | { -%>
    <cluster name="<%= cluster %>"
    ip="<%= properties['ip'] %>"
    port="<%= properties['port'] %>"
    >
    <% properties['servers'].each_pair | server, server_props | { -%>
    <server name="<%= server %>"
    ip="<%= server_props['ip'] %>"
    port="<%= server_props['port'] %>"
    />
    <% } -%>
    </cluster>
    <% } -%>


    Once corrected for the inevitable typos, that should produce output similar
    to
    <cluster name="cluster1"
    ip="192.168.0.2"
    port="80"
    >
    <server name="server1"
    ip="192.168.0.3"
    port="8080"
    />
    </cluster>


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/uZSpX_tBuM8J.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedJan 4, '13 at 3:11p
activeJan 4, '13 at 10:02p
posts7
users6
websitepuppetlabs.com

People

Translate

site design / logo © 2022 Grokbase