FAQ
I've seen how the puppetdb module uses ec2 to execute beaker tests. I've
tried setting this up as well and am getting some errors.

Is there a working example of using the different hypervisors?

I see
this: https://github.com/puppetlabs/beaker/wiki/Creating-A-Test-Environment#ec2-support
   but the documentation doesn't suggest it's a polished feature :)

It might be helpful to document a project that has an example of all the
hypervisors, as each one will have different required / optional parameters
in the nodeset configuration file.

Indented below is the error when running `beaker --hosts
spec/acceptance/nodesets/ec2-west-el6-64mda-el6-64a.cfg` - I pulled that
example from the puppetdb project, and as per the documentation have a
.fog file set up.. I was hoping to get an authentication or configuration
error on first run.

Beaker::Hypervisor, found some ec2 boxes to create
Failed: errored in CLI.provision
#<TypeError: no implicit conversion of Symbol into Integer>

/Projects/live_modules/puppet-shawlib/.bundle/ruby/2.1.0/gems/beaker-1.19.1/bin/beaker:6
/Users/bswift/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15
/Users/bswift/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15


I don't have much to go on there..

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/9c985036-bd1d-4850-99cf-9200f40584ca%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Ken Barber at Oct 7, 2014 at 1:08 am

    I've seen how the puppetdb module uses ec2 to execute beaker tests. I've
    tried setting this up as well and am getting some errors.

    Is there a working example of using the different hypervisors?

    I see this:
    https://github.com/puppetlabs/beaker/wiki/Creating-A-Test-Environment#ec2-support
    but the documentation doesn't suggest it's a polished feature :)

    It might be helpful to document a project that has an example of all the
    hypervisors, as each one will have different required / optional parameters
    in the nodeset configuration file.

    Indented below is the error when running `beaker --hosts
    spec/acceptance/nodesets/ec2-west-el6-64mda-el6-64a.cfg` - I pulled that
    example from the puppetdb project, and as per the documentation have a .fog
    file set up.. I was hoping to get an authentication or configuration error
    on first run.

    Beaker::Hypervisor, found some ec2 boxes to create
    Failed: errored in CLI.provision
    #<TypeError: no implicit conversion of Symbol into Integer>
    /Projects/live_modules/puppet-shawlib/.bundle/ruby/2.1.0/gems/beaker-1.19.1/bin/beaker:6
    /Users/bswift/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15
    /Users/bswift/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15


    I don't have much to go on there..
    The exception seems a little truncated, is that it? BTW Instead of
    annotating what has happened in a paragraph, just provide the full
    example in your shell in a gist or something, it says the message much
    more simply :-). Sort of like this:
    https://gist.github.com/kbarber/97f45eba0f922497901a. Or better yet,
    if you can create a repository with the basics of the above so I can
    take a look that would be much much easier.

    I must warn you, the aws_sdk.rb code was written in a massive hurry,
    and thus has very little error protection so if everything isn't spot
    on, it might break.

    ken.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAE4bNT%3D9knDc5Ovg3_ws7K_4UifvPV1Djo%3D8FJFsZO3EPLDpjQ%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ken Barber at Oct 7, 2014 at 1:14 am

    I've seen how the puppetdb module uses ec2 to execute beaker tests. I've
    tried setting this up as well and am getting some errors.

    Is there a working example of using the different hypervisors?

    I see this:
    https://github.com/puppetlabs/beaker/wiki/Creating-A-Test-Environment#ec2-support
    but the documentation doesn't suggest it's a polished feature :)

    It might be helpful to document a project that has an example of all the
    hypervisors, as each one will have different required / optional parameters
    in the nodeset configuration file.

    Indented below is the error when running `beaker --hosts
    spec/acceptance/nodesets/ec2-west-el6-64mda-el6-64a.cfg` - I pulled that
    example from the puppetdb project, and as per the documentation have a .fog
    file set up.. I was hoping to get an authentication or configuration error
    on first run.

    Beaker::Hypervisor, found some ec2 boxes to create
    Failed: errored in CLI.provision
    #<TypeError: no implicit conversion of Symbol into Integer>
    /Projects/live_modules/puppet-shawlib/.bundle/ruby/2.1.0/gems/beaker-1.19.1/bin/beaker:6
    /Users/bswift/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15
    /Users/bswift/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15


    I don't have much to go on there..
    The exception seems a little truncated, is that it? BTW Instead of
    annotating what has happened in a paragraph, just provide the full
    example in your shell in a gist or something, it says the message much
    more simply :-). Sort of like this:
    https://gist.github.com/kbarber/97f45eba0f922497901a. Or better yet,
    if you can create a repository with the basics of the above so I can
    take a look that would be much much easier.

    I must warn you, the aws_sdk.rb code was written in a massive hurry,
    and thus has very little error protection so if everything isn't spot
    on, it might break.
    FWIW also - I wouldn't test the state of the puppetdb module testing
    pieces, we've been waiting for our QA & Module teams to automate those
    tests for a about a year now :-(. But honestly, they haven't been ran
    for quite some time - so I'd be dubious.

    The PuppetDB source code itself (ie. puppetdb server, not the module)
    however does use the AWS EC2 test stuff, and thats a better example to
    use.

    ken.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAE4bNT%3DgHWf5zD3j8GD8%2BYXsf5HCwYPvSXcH2Veka2O2_b5w_w%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.
  • Brettswift at Oct 7, 2014 at 1:39 am
    I was just testing the host config file from puppetdb coupled with the documentation on the beaker documentation. I was actually going to omit the error message. That's actually all of it except for the json output of the compiled beaker configs. I can send the full output in the morning.

    It looks like the Google Compute Engine docs are more complete... It doesn't matter where it runs. Mostly looking for a free tier cloud to get started with. I'm not sure aws micro would even be big enough anyways. But it'd be cool to get it working.

    Thanks



    Brett
    Sent from my iPhone

    On Oct 6, 2014, at 19:13, Ken Barber wrote:

    I've seen how the puppetdb module uses ec2 to execute beaker tests. I've
    tried setting this up as well and am getting some errors.

    Is there a working example of using the different hypervisors?

    I see this:
    https://github.com/puppetlabs/beaker/wiki/Creating-A-Test-Environment#ec2-support
    but the documentation doesn't suggest it's a polished feature :)

    It might be helpful to document a project that has an example of all the
    hypervisors, as each one will have different required / optional parameters
    in the nodeset configuration file.

    Indented below is the error when running `beaker --hosts
    spec/acceptance/nodesets/ec2-west-el6-64mda-el6-64a.cfg` - I pulled that
    example from the puppetdb project, and as per the documentation have a .fog
    file set up.. I was hoping to get an authentication or configuration error
    on first run.

    Beaker::Hypervisor, found some ec2 boxes to create
    Failed: errored in CLI.provision
    #<TypeError: no implicit conversion of Symbol into Integer>
    /Projects/live_modules/puppet-shawlib/.bundle/ruby/2.1.0/gems/beaker-1.19.1/bin/beaker:6
    /Users/bswift/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15
    /Users/bswift/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15


    I don't have much to go on there..
    The exception seems a little truncated, is that it? BTW Instead of
    annotating what has happened in a paragraph, just provide the full
    example in your shell in a gist or something, it says the message much
    more simply :-). Sort of like this:
    https://gist.github.com/kbarber/97f45eba0f922497901a. Or better yet,
    if you can create a repository with the basics of the above so I can
    take a look that would be much much easier.

    I must warn you, the aws_sdk.rb code was written in a massive hurry,
    and thus has very little error protection so if everything isn't spot
    on, it might break.
    FWIW also - I wouldn't test the state of the puppetdb module testing
    pieces, we've been waiting for our QA & Module teams to automate those
    tests for a about a year now :-(. But honestly, they haven't been ran
    for quite some time - so I'd be dubious.

    The PuppetDB source code itself (ie. puppetdb server, not the module)
    however does use the AWS EC2 test stuff, and thats a better example to
    use.

    ken.

    --
    You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
    To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/AzcyYneW820/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAE4bNT%3DgHWf5zD3j8GD8%2BYXsf5HCwYPvSXcH2Veka2O2_b5w_w%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/7D366F34-582B-4B85-9276-04DA77A7A7E4%40gmail.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ken Barber at Oct 7, 2014 at 1:44 am
    I was just testing the host config file from puppetdb coupled with the documentation on the beaker documentation.
    Those docs honestly look old, they are still mentioning blimpy which I
    effectively deprecated/superseded with the aws_sdk driver.
    I was actually going to omit the error message. That's actually all of it except for the json output of the compiled beaker configs. I can send the full output in the morning.
    Send the full output and the configuration and I can take a closer
    look. Anything less, I'll probably struggle.
    It looks like the Google Compute Engine docs are more complete... It doesn't matter where it runs. Mostly looking for a free tier cloud to get started with. I'm not sure aws micro would even be big enough anyways. But it'd be cool to get it working.
    Sure, well we use EC2 heavily so I'm happy to help you there, I know
    some people use Google Compute Engine also, but I have no intimate
    knowledge of how this one works.

    ken.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAE4bNTng8oLjKCaVS6RV%2BjBSHqWAgYSatP69fpxcNWF1Upmz%2BA%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.
  • Justin Stoller at Oct 7, 2014 at 1:58 am

    On Mon, Oct 6, 2014 at 6:44 PM, Ken Barber wrote:

    I was just testing the host config file from puppetdb coupled with the
    documentation on the beaker documentation.

    Those docs honestly look old, they are still mentioning blimpy which I
    effectively deprecated/superseded with the aws_sdk driver.
    I was actually going to omit the error message. That's actually all of
    it except for the json output of the compiled beaker configs. I can send
    the full output in the morning.

    Send the full output and the configuration and I can take a closer
    look. Anything less, I'll probably struggle.
    You should also include a redacted version of your ~/.fog

    It looks like the Google Compute Engine docs are more complete... It
    doesn't matter where it runs. Mostly looking for a free tier cloud to get
    started with. I'm not sure aws micro would even be big enough anyways.
    But it'd be cool to get it working.

    Sure, well we use EC2 heavily so I'm happy to help you there, I know
    some people use Google Compute Engine also, but I have no intimate
    knowledge of how this one works.

    ken.

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit
    https://groups.google.com/d/msgid/puppet-users/CAE4bNTng8oLjKCaVS6RV%2BjBSHqWAgYSatP69fpxcNWF1Upmz%2BA%40mail.gmail.com
    .
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CA%2B%3DBEqW3s608U-VhyhVnFnJ3CaqEZ7dcwbYvvJ-jg3iaCXEPOQ%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ken Barber at Oct 7, 2014 at 10:37 am

    I was just testing the host config file from puppetdb coupled with the documentation on the beaker documentation.
    Those docs honestly look old, they are still mentioning blimpy which I
    effectively deprecated/superseded with the aws_sdk driver.
    I was actually going to omit the error message. That's actually all of it except for the json output of the compiled beaker configs. I can send the full output in the morning.
    Send the full output and the configuration and I can take a closer
    look. Anything less, I'll probably struggle.
    It looks like the Google Compute Engine docs are more complete... It doesn't matter where it runs. Mostly looking for a free tier cloud to get started with. I'm not sure aws micro would even be big enough anyways. But it'd be cool to get it working.
    Sure, well we use EC2 heavily so I'm happy to help you there, I know
    some people use Google Compute Engine also, but I have no intimate
    knowledge of how this one works.
    Actually Brett, maybe this is a better approach. I've got a working
    new project here showing beaker + beaker-rspec with EC2 support:

    https://github.com/kbarber/sample-beaker

    And you can see how I've launched it here:

    https://gist.github.com/kbarber/850a7d88fce409592bab

    Perhaps a better example will set you straight :-). It does fail
    incidentally, but it is kind of meant to as an example. Perhaps you
    can start with this project skeleton and modify to taste.

    Now as Justin mentioned, you do need a ~/.fog file - this was
    primarily to be compatible with the old Blimpy driver, but alas, we
    don't use Fog any more. The file should look something like:

    :default:
       :aws_access_key_id: AAAAAA
       :aws_secret_access_key: BBBBBB

    (And obviously match your own EC2 access keys and secrets)

    Also pay close attention to the config/image_templates/ec2.yaml file
    ... these map names of images to AMIs today, and the AMIs provided are
    the ones we use for our own testing, but you might want to maintain
    your own list. This is entirely up to you, just be aware each AMI
    needs a small amount of pre-setup if you want to create your own (that
    is certain minimal things might need 'baking' into the image, but
    nothing drastic). Of course, you are free to use the ones here, but if
    you need customisation I'd suggest forking your own images :-).

    Let me know how you get on with that. I think generally speaking all
    of this needs an overhaul in regards to usability, a lot of this
    awkward layout is due to backwards compatibility from legacy elements.
    That aside, once you get the fundamental elements right it should be
    okay.

    ken.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAE4bNT%3DzPBPSQi04EWH8J7sbMqmV0OXititEcgbPGM8utnjzGg%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.
  • Brett Swift at Oct 7, 2014 at 2:29 pm
    That's great Ken.

    I'll have a look. My .fog file was correct but I was missing that
    ec2.yaml.

    I get the user experience thing, it'll evolve and I'll help if I can.

    Would I be right to assume you built your images with packer?

    Thanks!
    On Tuesday, 7 October 2014 04:37:43 UTC-6, Ken Barber wrote:

    I was just testing the host config file from puppetdb coupled with the
    documentation on the beaker documentation.
    Those docs honestly look old, they are still mentioning blimpy which I
    effectively deprecated/superseded with the aws_sdk driver.
    I was actually going to omit the error message. That's actually all of
    it except for the json output of the compiled beaker configs. I can send
    the full output in the morning.
    Send the full output and the configuration and I can take a closer
    look. Anything less, I'll probably struggle.
    It looks like the Google Compute Engine docs are more complete... It
    doesn't matter where it runs. Mostly looking for a free tier cloud to get
    started with. I'm not sure aws micro would even be big enough anyways.
    But it'd be cool to get it working.
    Sure, well we use EC2 heavily so I'm happy to help you there, I know
    some people use Google Compute Engine also, but I have no intimate
    knowledge of how this one works.
    Actually Brett, maybe this is a better approach. I've got a working
    new project here showing beaker + beaker-rspec with EC2 support:

    https://github.com/kbarber/sample-beaker

    And you can see how I've launched it here:

    https://gist.github.com/kbarber/850a7d88fce409592bab

    Perhaps a better example will set you straight :-). It does fail
    incidentally, but it is kind of meant to as an example. Perhaps you
    can start with this project skeleton and modify to taste.

    Now as Justin mentioned, you do need a ~/.fog file - this was
    primarily to be compatible with the old Blimpy driver, but alas, we
    don't use Fog any more. The file should look something like:

    :default:
    :aws_access_key_id: AAAAAA
    :aws_secret_access_key: BBBBBB

    (And obviously match your own EC2 access keys and secrets)

    Also pay close attention to the config/image_templates/ec2.yaml file
    ... these map names of images to AMIs today, and the AMIs provided are
    the ones we use for our own testing, but you might want to maintain
    your own list. This is entirely up to you, just be aware each AMI
    needs a small amount of pre-setup if you want to create your own (that
    is certain minimal things might need 'baking' into the image, but
    nothing drastic). Of course, you are free to use the ones here, but if
    you need customisation I'd suggest forking your own images :-).

    Let me know how you get on with that. I think generally speaking all
    of this needs an overhaul in regards to usability, a lot of this
    awkward layout is due to backwards compatibility from legacy elements.
    That aside, once you get the fundamental elements right it should be
    okay.

    ken.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/cea098e8-56e9-4c17-b02e-5cfc209be3d7%40googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ken Barber at Oct 7, 2014 at 2:43 pm

    That's great Ken.

    I'll have a look. My .fog file was correct but I was missing that
    ec2.yaml.

    I get the user experience thing, it'll evolve and I'll help if I can.

    Would I be right to assume you built your images with packer?
    All of those images predate packer, but we're using packer now. For
    example my colleague Wyatt is about to add Debian 7 testing using a
    packer template he has developed. So going forward yes, packer is the
    trick.

    ken.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAE4bNTnp56EsgFj9W2c%3DDnWFUWEz34asHdC06%2BvBV1cmEhYpLw%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.
  • Brett Swift at Oct 7, 2014 at 8:17 pm
    Nice. I'll look out for your packer project. I've been using it to build
    our vagrant boxes, but haven't yet built any for vSphere or external cloud.

    I'm getting this error:
    ruby/2.1.0/gems/aws-sdk-1.42.0/lib/aws/core/resource.rb:238:in `rescue in
    block in define_attribute_getter': unable to find the image
    (AWS::Core::Resource::NotFound)

    Are your AMI's public?
    Gist:
    https://gist.github.com/brettswift/176b802bfe31dae369e9


    Using your sample project, if I include the puppetlabs_spec_helper in the
    Gemfile, it doesn't error but completes rather quickly saying there are no
    tests (obviously). But it doesn't bawk on any ec2 configuration.




    On Tuesday, 7 October 2014 08:43:45 UTC-6, Ken Barber wrote:

    That's great Ken.

    I'll have a look. My .fog file was correct but I was missing that
    ec2.yaml.

    I get the user experience thing, it'll evolve and I'll help if I can.

    Would I be right to assume you built your images with packer?
    All of those images predate packer, but we're using packer now. For
    example my colleague Wyatt is about to add Debian 7 testing using a
    packer template he has developed. So going forward yes, packer is the
    trick.

    ken.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/abdbbcfe-b23d-4eb6-8e4c-0f53b68ac6a3%40googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ken Barber at Oct 7, 2014 at 9:15 pm

    Nice. I'll look out for your packer project. I've been using it to build
    our vagrant boxes, but haven't yet built any for vSphere or external cloud.
    Its not my project, its a private project we have. So I can't speak
    around if we'll release it or not - just to be clear :-).
    I'm getting this error:
    ruby/2.1.0/gems/aws-sdk-1.42.0/lib/aws/core/resource.rb:238:in `rescue in
    block in define_attribute_getter': unable to find the image
    (AWS::Core::Resource::NotFound)

    Are your AMI's public?
    Gist:
    https://gist.github.com/brettswift/176b802bfe31dae369e9
    I would presume not? Honestly, I would look into creating your own
    most probably. Not because they don't work, just that you could
    probably do a better quality job and then own the process yourself.
    Start with a good basis, like the endorsed Centos AMI's for example:

    http://wiki.centos.org/Cloud/AWS

    Then apply the necessary customisations on top with packer using the
    amazon-ebs builder as a good 'entry level' way of doing it:
    http://www.packer.io/docs/builders/amazon-ebs.html
    Using your sample project, if I include the puppetlabs_spec_helper in the
    Gemfile, it doesn't error but completes rather quickly saying there are no
    tests (obviously). But it doesn't bawk on any ec2 configuration.
    So puppetlabs_spec_helper assists with the unit testing side of things
    and has nothing directly to do with beaker, so look into rspec-puppet
    for the long story around that. puppetlabs_spec_helper provides a
    number of utilities and helps bridge the testing parts with various
    different versions of puppet basically, but the main piece of work
    that users should focus on is rspec-puppet.

    Basically from a user perspective the helper gives you a rake task:
    `rake spec` and a file like so:
    https://github.com/puppetlabs/puppetlabs-puppetdb/blob/master/.fixtures.yml
    that will help you automatically retrieve the other modules your
    module depends on during testing time. As opposed to just running
    `rspec spec/unit` for example. So yeah, chalk and cheese ...

    ken.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAE4bNTnCuq80Zj%2ByuzNQcqUErz7viufhhdPHsHpyicQkNkKqTg%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.
  • Brettswift at Oct 7, 2014 at 9:46 pm
    Hmmm. I'm pretty sure the puppelabs_spec_helper gave me a beaker target....




    Brett
    Sent from my iPhone

    On Oct 7, 2014, at 15:15, Ken Barber wrote:

    Nice. I'll look out for your packer project. I've been using it to build
    our vagrant boxes, but haven't yet built any for vSphere or external cloud.
    Its not my project, its a private project we have. So I can't speak
    around if we'll release it or not - just to be clear :-).
    I'm getting this error:
    ruby/2.1.0/gems/aws-sdk-1.42.0/lib/aws/core/resource.rb:238:in `rescue in
    block in define_attribute_getter': unable to find the image
    (AWS::Core::Resource::NotFound)

    Are your AMI's public?
    Gist:
    https://gist.github.com/brettswift/176b802bfe31dae369e9
    I would presume not? Honestly, I would look into creating your own
    most probably. Not because they don't work, just that you could
    probably do a better quality job and then own the process yourself.
    Start with a good basis, like the endorsed Centos AMI's for example:

    http://wiki.centos.org/Cloud/AWS

    Then apply the necessary customisations on top with packer using the
    amazon-ebs builder as a good 'entry level' way of doing it:
    http://www.packer.io/docs/builders/amazon-ebs.html
    Using your sample project, if I include the puppetlabs_spec_helper in the
    Gemfile, it doesn't error but completes rather quickly saying there are no
    tests (obviously). But it doesn't bawk on any ec2 configuration.
    So puppetlabs_spec_helper assists with the unit testing side of things
    and has nothing directly to do with beaker, so look into rspec-puppet
    for the long story around that. puppetlabs_spec_helper provides a
    number of utilities and helps bridge the testing parts with various
    different versions of puppet basically, but the main piece of work
    that users should focus on is rspec-puppet.

    Basically from a user perspective the helper gives you a rake task:
    `rake spec` and a file like so:
    https://github.com/puppetlabs/puppetlabs-puppetdb/blob/master/.fixtures.yml
    that will help you automatically retrieve the other modules your
    module depends on during testing time. As opposed to just running
    `rspec spec/unit` for example. So yeah, chalk and cheese ...

    ken.

    --
    You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
    To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/AzcyYneW820/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAE4bNTnCuq80Zj%2ByuzNQcqUErz7viufhhdPHsHpyicQkNkKqTg%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/D4E401BC-37FB-4DCC-8682-EF76FD68C300%40gmail.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ken Barber at Oct 7, 2014 at 10:26 pm

    Using your sample project, if I include the puppetlabs_spec_helper in the
    Gemfile, it doesn't error but completes rather quickly saying there are no
    tests (obviously). But it doesn't bawk on any ec2 configuration.
    So puppetlabs_spec_helper assists with the unit testing side of things
    and has nothing directly to do with beaker, so look into rspec-puppet
    for the long story around that. puppetlabs_spec_helper provides a
    number of utilities and helps bridge the testing parts with various
    different versions of puppet basically, but the main piece of work
    that users should focus on is rspec-puppet.

    Basically from a user perspective the helper gives you a rake task:
    `rake spec` and a file like so:
    https://github.com/puppetlabs/puppetlabs-puppetdb/blob/master/.fixtures.yml
    that will help you automatically retrieve the other modules your
    module depends on during testing time. As opposed to just running
    `rspec spec/unit` for example. So yeah, chalk and cheese ...
    Hmmm. I'm pretty sure the puppelabs_spec_helper gave me a beaker target....
    Yep you are right, it was added in January by blkperl. I didn't realize this.

    So in response to your original question:
    Using your sample project, if I include the puppetlabs_spec_helper in the
    Gemfile, it doesn't error but completes rather quickly saying there are no
    tests (obviously). But it doesn't bawk on any ec2 configuration.
    Perhaps its because its using the `default.yml` definition that it is
    not complaining. Perhaps that AMI is public and fine. Check
    `spec/acceptance/nodesets/default.yml` in your project for what this
    contains.

    All the rake task seems to do is run beaker with --color and the
    spec/acceptance directory as a path, so beakers defaults everywhere
    else kick in. Which makes sense, in this case there are several
    BEAKER_* style variables that are used to setup the details in CI for
    example. This is because CI software such as Jenkins uses environment
    variables as a primary API between the Jenkins job data and the jobs
    scripts being executed.

    https://github.com/puppetlabs/beaker/wiki/How-to-Write-a-Beaker-Test-for-a-Module

    The BEAKER_set allows for a basic way of executing a particular
    nodeset for example, which is rather nice. Perfect for a matrix job in
    Jenkins to iterate across your distros.

    ken.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAE4bNT%3DPeu-CEU50yFTgoyWhywHAPbqz6tbD_PVTKznv-FDPqw%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.
  • Brettswift at Oct 7, 2014 at 10:37 pm
    The default.yaml is in the gist I supplied. I used the el6 and centos6 64 from your sample project.

    An Ami search on aws by the name or Ami ID didn't turn up any results which is why I thought they were private.

    I'll try building an Ami with packer. Are there any conventions that need to go into the VM required by beaker?




    Brett
    Sent from my iPhone

    On Oct 7, 2014, at 16:25, Ken Barber wrote:

    Using your sample project, if I include the puppetlabs_spec_helper in the
    Gemfile, it doesn't error but completes rather quickly saying there are no
    tests (obviously). But it doesn't bawk on any ec2 configuration.
    So puppetlabs_spec_helper assists with the unit testing side of things
    and has nothing directly to do with beaker, so look into rspec-puppet
    for the long story around that. puppetlabs_spec_helper provides a
    number of utilities and helps bridge the testing parts with various
    different versions of puppet basically, but the main piece of work
    that users should focus on is rspec-puppet.

    Basically from a user perspective the helper gives you a rake task:
    `rake spec` and a file like so:
    https://github.com/puppetlabs/puppetlabs-puppetdb/blob/master/.fixtures.yml
    that will help you automatically retrieve the other modules your
    module depends on during testing time. As opposed to just running
    `rspec spec/unit` for example. So yeah, chalk and cheese ...
    Hmmm. I'm pretty sure the puppelabs_spec_helper gave me a beaker target....
    Yep you are right, it was added in January by blkperl. I didn't realize this.

    So in response to your original question:
    Using your sample project, if I include the puppetlabs_spec_helper in the
    Gemfile, it doesn't error but completes rather quickly saying there are no
    tests (obviously). But it doesn't bawk on any ec2 configuration.
    Perhaps its because its using the `default.yml` definition that it is
    not complaining. Perhaps that AMI is public and fine. Check
    `spec/acceptance/nodesets/default.yml` in your project for what this
    contains.

    All the rake task seems to do is run beaker with --color and the
    spec/acceptance directory as a path, so beakers defaults everywhere
    else kick in. Which makes sense, in this case there are several
    BEAKER_* style variables that are used to setup the details in CI for
    example. This is because CI software such as Jenkins uses environment
    variables as a primary API between the Jenkins job data and the jobs
    scripts being executed.

    https://github.com/puppetlabs/beaker/wiki/How-to-Write-a-Beaker-Test-for-a-Module

    The BEAKER_set allows for a basic way of executing a particular
    nodeset for example, which is rather nice. Perfect for a matrix job in
    Jenkins to iterate across your distros.

    ken.

    --
    You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
    To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/AzcyYneW820/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAE4bNT%3DPeu-CEU50yFTgoyWhywHAPbqz6tbD_PVTKznv-FDPqw%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/B5314982-8BA2-4583-9CE0-E838CD0B8679%40gmail.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ken Barber at Oct 8, 2014 at 10:53 am
    The default.yaml is in the gist I supplied. I used the el6 and centos6 64 from your sample project.
    Taking a look again, its even more confusing than that, the platform
    name you have used is pointing at one of the very very old AMI's. The
    newer ones we actually use are private, but I don't have the ability
    to make it public. Looks like its locked up in someone elses older
    amazon account. Sorry about that. This is why we are slowly replacing
    these older ones, the management of them has slipped, my fault mainly
    - we'll work on that for PuppetDB.
    An Ami search on aws by the name or Ami ID didn't turn up any results which is why I thought they were private.

    I'll try building an Ami with packer. Are there any conventions that need to go into the VM required by beaker?
    I think the main requirement to make your life easier, is to allow
    logging in from root (which can be often disabled) this is often done
    by changing just sshd_config (and hupping/restarting sshd), and
    removing the /root/.ssh/authorized_keys files also. Some distros
    support doing this kind of thing via cloud-init as well, so you can
    pass something like:

    #cloud-config
    disable-root: false

    Into the user_data field of packer for example, but not this will not
    persist on its own, you can change the configuration in /etc/cloud/
    and bake that setting into the image. This usually affects
    Debian/Ubuntu images I believe, I haven't built one of the newer
    centos images in a while so try firing up the image and checking if it
    has /etc/cloud/ in its build, this usually indicates its got some sort
    of cloud-init. Having said that, someone has a patch up for beaker to
    allow logging in as non-root and 'fixing' this:

    https://github.com/puppetlabs/beaker/pull/478

    Beyond that it depends on the tests and helpers you want to use. Both
    'curl' and 'git' are good tools to have around on an image, since a
    lot of tests can utilize these commands, having said that some tools
    on top of the image can most certainly be installed by beaker in the
    spec_helper_acceptance.rb there is a section for doing this _before_
    the suite runs.

    In fact, the base beaker stuff will attempt to install items for you,
    like timing is important so the code goes to lengths to make sure the
    the image has the right tools before it uses ntp to update the time.
    Having the tooling there (I believe it uses ntpdate? but look at your
    log output from beaker it will tell you) should make that exercise
    mildly faster.

    Starting simple, and trying to run tests against an image is usually a
    good move in general, and being additive when you need it. Like I said
    you have two choices most of the time - bake in a tool, or do it
    during the testing run _before_ suite ... its really dependant on the
    wider problem you are going to solve with the image. Baking items into
    the image will of course speed up the exercise at test time, at the
    cost of forcing you to remake it when it needs a change. Starting
    conservatively with what goes into the image I think is a good idea.

    After that you'll start to see the wider problems when you run your
    tests a lot ... such as the flakiness of public mirrors :-). But thats
    a story for another day :-).

    ken.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAE4bNT%3DmaOQ9QvoVC-e9DkQm%2Bs7iRDrgt3%3Dvn%2BHYU%3D_Zvsw%3DAQ%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedOct 6, '14 at 9:22p
activeOct 8, '14 at 10:53a
posts15
users3
websitepuppetlabs.com

People

Translate

site design / logo © 2022 Grokbase