FAQ
Hello,

Since a few days of git updates, I was getting the following error when
trying to upload the bosh-stemcell-aws-0.5.1.tgz “expected string value for
member 1 of key values of member 1 of option filters”. I seem to have fixed
it by patching the bosh-stemcell-aws-0.5.1.tgz manifest.


What's the impact of using an "old" stemcell for AWS
(bosh-stemcell-aws-0.5.1.tgz) ? I see there is a 0.6.2 micro stemcell in
review. Can it be used on AWS ? How can I otherwise convert the
bosh-stemcell-0.5.2.tgz to be used on aws ?
Is it worth putting some effort with aws-specific stemcells if public amis
are in preparation ?

BTW, is there another recommended way to use bosh on AWS than
creating-a-bosh-from-scratch.md<https://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-bosh-from-scratch.md>(i.e. using chef) ?

- creating-a-micro-bosh-from-stemcell.md<https://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-micro-bosh-from-stemcell.md>indicates a pending micro-bosh stemcell 0.6.1 to be published (maybe the 0.6.2
<http://reviews.cloudfoundry.org/7757>would match it?)
-

uploading-public-stemcell.md<https://github.com/drnic/bosh-getting-started/blob/master/uploading-public-stemcell.md>reference a bosh-stemcell-aws-0.6.2.tgz I can't find in the list of public
stemcells.
-

creating-a-micro-bosh-from-ami.md<https://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-micro-bosh-from-ami.md>mentions writing this is in progress. Is there missing prereqs to get this
working on eu-west-1 region ?

Some ideas, to make next diagnostics easier:
- maybe, the create_stemcell() method could check validity of the manifest
(expected presence of the architecture and root_device_name) and display a
hint message in this case ?
- the "bosh upload stemcell" command could try validating the created
stemcell by instanciating it and pinging the instance. This would avoid
diagnosing it in later deployments steps.


Diagnostics and applied patches follow, I hope this can help others on AWS

Thanks,

Guillaume.

I’m running on eu-west-1 using a bosh instance populated from the chef
deployment following creating-a-bosh-from-scratch.md<https://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-bosh-from-scratch.md>

Looking at the aki autodetection code<https://github.com/cloudfoundry/bosh/blob/master/aws_cpi/lib/cloud/aws/cloud.rb#L288>following
7489 <http://reviews.cloudfoundry.org/7489> change, I patched the stemcell
manifest from the original bosh-stemcell-aws-0.5.1.tgz to add the following
(using the arch and root device name from the public ami-3d1f2b49 used on
my bosh-chef instance)

cloud_properties:
architecture: "x86_64"
root_device_name: "/dev/sda1"

However, the instances did not seem to properly start (compilation jobs
keep timeouting). Looking into the aki autoselected, it was the 'aki-60695814'
corresponding to 'pv-grub-hd00_1.02-x86_64'. This seems to be an image for
a partition table while previously my sucessfull tests were having the
default raw unpartitionned disk image (cf aws doc<http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/UserProvidedkernels.html%20>).


I tried to use /dev/sda instead to have the find_aki method pick the hd0
aki instead, and this solved the compilation jobs timeouting.
Full stack trace of the original issue.

E, [2012-07-26T08:10:39.343383 #383]
[create_stemcell(/var/vcap/deploy/tmp/stemcell20120726-383-1yv79vv/image...)]
ERROR -- : expected string value for member 1 of key values of member 1 of
option filters (AWS::Core::OptionGrammar::FormatError)

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:93:in
`validate'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:209:in
`block in validate'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in
`each'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in
`validate'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:310:in
`block in validate'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:306:in
`each'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:306:in
`validate'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:209:in
`block in validate'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in
`each'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in
`validate'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:550:in
`block in validate'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:546:in
`each'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:546:in
`validate'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:561:in
`request_params'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/configured_client_methods.rb:50:in
`block (3 levels) in configure_client'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:409:in
`build_request'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:350:in
`block (3 levels) in client_request'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/response.rb:60:in
`call'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/response.rb:60:in
`rebuild_request'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/response.rb:55:in
`initialize'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:171:in
`new'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:171:in
`new_response'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:350:in
`block (2 levels) in client_request'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client_logging.rb:25:in
`block in log_client_request'

/var/vcap/deploy/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/benchmark.rb:294:in
`measure'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client_logging.rb:24:in
`log_client_request'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:338:in
`block in client_request'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:275:in
`return_or_raise'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:337:in
`client_request'

(eval):3:in `describe_images'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_aws_cpi-0.6.0/lib/cloud/aws/cloud.rb:362:in
`find_aki'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_aws_cpi-0.6.0/lib/cloud/aws/cloud.rb:321:in
`block in create_stemcell'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_common-0.4.0/lib/common/thread_formatter.rb:46:in
`with_thread_name'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_aws_cpi-0.6.0/lib/cloud/aws/cloud.rb:296:in
`create_stemcell'

/var/vcap/deploy/bosh/director/current/director/lib/director/jobs/update_stemcell.rb:68:in
`block in perform'

/var/vcap/deploy/bosh/director/current/director/lib/director/jobs/base_job.rb:46:in
`block in track_and_log'

/var/vcap/deploy/bosh/director/current/director/lib/director/event_log.rb:58:in
`track'

/var/vcap/deploy/bosh/director/current/director/lib/director/jobs/base_job.rb:44:in
`track_and_log'

/var/vcap/deploy/bosh/director/current/director/lib/director/jobs/update_stemcell.rb:66:in
`perform'

/var/vcap/deploy/bosh/director/current/director/lib/director/job_runner.rb:95:in
`perform_job'

/var/vcap/deploy/bosh/director/current/director/lib/director/job_runner.rb:29:in
`block in run'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_common-0.4.0/lib/common/thread_formatter.rb:46:in
`with_thread_name'

/var/vcap/deploy/bosh/director/current/director/lib/director/job_runner.rb:29:in
`run'

/var/vcap/deploy/bosh/director/current/director/lib/director/jobs/base_job.rb:8:in
`perform'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/job.rb:127:in
`perform'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:163:in
`perform'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:130:in
`block in work'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:116:in
`loop'

/var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:116:in
`work'

/var/vcap/deploy/bosh/director/current/director/bin/worker:77:in `<main>'

I

Search Discussions

  • Dr Nic Williams at Jul 26, 2012 at 8:56 pm
    Sorry that the current "create a bosh" articles are all flawed - many are written in advance of Martin releasing new stemcells. When they ship I will proof all the tutorials again and confirm they work.

    Cheers
    Nic


    Dr Nic Williams - VP Developer Evangelism
    Engine Yard
    The Leading Platform as a Service
    Mobile: +1 415 860 2185
    Skype: nicwilliams
    Twitter: @drnic

    On Thursday, July 26, 2012 at 1:28 PM, Guillaume Berche wrote:

    Hello,

    Since a few days of git updates, I was getting the following error when trying to upload the bosh-stemcell-aws-0.5.1.tgz “expected string value for member 1 of key values of member 1 of option filters”. I seem to have fixed it by patching the bosh-stemcell-aws-0.5.1.tgz manifest.





    What's the impact of using an "old" stemcell for AWS (bosh-stemcell-aws-0.5.1.tgz) ? I see there is a 0.6.2 micro stemcell in review. Can it be used on AWS ? How can I otherwise convert the bosh-stemcell-0.5.2.tgz to be used on aws ?


    Is it worth putting some effort with aws-specific stemcells if public amis are in preparation ?


    BTW, is there another recommended way to use bosh on AWS than creating-a-bosh-from-scratch.md (https://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-bosh-from-scratch.md) (i.e. using chef) ?


    creating-a-micro-bosh-from-stemcell.md (https://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-micro-bosh-from-stemcell.md) indicates a pending micro-bosh stemcell 0.6.1 to be published (maybe the 0.6.2 (http://reviews.cloudfoundry.org/7757)would match it?)

    uploading-public-stemcell.md (https://github.com/drnic/bosh-getting-started/blob/master/uploading-public-stemcell.md) reference a bosh-stemcell-aws-0.6.2.tgz I can't find in the list of public stemcells.



    creating-a-micro-bosh-from-ami.md (https://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-micro-bosh-from-ami.md) mentions writing this is in progress. Is there missing prereqs to get this working on eu-west-1 region ?




    Some ideas, to make next diagnostics easier:
    - maybe, the create_stemcell() method could check validity of the manifest (expected presence of the architecture and root_device_name) and display a hint message in this case ?
    - the "bosh upload stemcell" command could try validating the created stemcell by instanciating it and pinging the instance. This would avoid diagnosing it in later deployments steps.




    Diagnostics and applied patches follow, I hope this can help others on AWS

    Thanks,

    Guillaume.

    I’m running on eu-west-1 using a bosh instance populated from the chef deployment following creating-a-bosh-from-scratch.md (https://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-bosh-from-scratch.md)

    Looking at the aki autodetection code (https://github.com/cloudfoundry/bosh/blob/master/aws_cpi/lib/cloud/aws/cloud.rb#L288) following 7489 (http://reviews.cloudfoundry.org/7489) change, I patched the stemcell manifest from the original bosh-stemcell-aws-0.5.1.tgz to add the following (using the arch and root device name from the public ami-3d1f2b49 used on my bosh-chef instance)



    cloud_properties:
    architecture: "x86_64"
    root_device_name: "/dev/sda1"






    However, the instances did not seem to properly start (compilation jobs keep timeouting). Looking into the aki autoselected, it was the 'aki-60695814' corresponding to 'pv-grub-hd00_1.02-x86_64'. This seems to be an image for a partition table while previously my sucessfull tests were having the default raw unpartitionned disk image (cf aws doc (http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/UserProvidedkernels.html%20)).



    I tried to use /dev/sda instead to have the find_aki method pick the hd0 aki instead, and this solved the compilation jobs timeouting.


    Full stack trace of the original issue.


    E, [2012-07-26T08:10:39.343383 #383] [create_stemcell(/var/vcap/deploy/tmp/stemcell20120726-383-1yv79vv/image...)] ERROR -- : expected string value for member 1 of key values of member 1 of option filters (AWS::Core::OptionGrammar::FormatError)


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:93:in `validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:209:in `block in validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in `each'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in `validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:310:in `block in validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:306:in `each'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:306:in `validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:209:in `block in validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in `each'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in `validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:550:in `block in validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:546:in `each'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:546:in `validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:561:in `request_params'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/configured_client_methods.rb:50:in `block (3 levels) in configure_client'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:409:in `build_request'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:350:in `block (3 levels) in client_request'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/response.rb:60:in `call'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/response.rb:60:in `rebuild_request'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/response.rb:55:in `initialize'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:171:in `new'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:171:in `new_response'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:350:in `block (2 levels) in client_request'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client_logging.rb:25:in `block in log_client_request'


    /var/vcap/deploy/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/benchmark.rb:294:in `measure'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client_logging.rb:24:in `log_client_request'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:338:in `block in client_request'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:275:in `return_or_raise'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:337:in `client_request'


    (eval):3:in `describe_images'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_aws_cpi-0.6.0/lib/cloud/aws/cloud.rb:362:in `find_aki'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_aws_cpi-0.6.0/lib/cloud/aws/cloud.rb:321:in `block in create_stemcell'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_common-0.4.0/lib/common/thread_formatter.rb:46:in `with_thread_name'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_aws_cpi-0.6.0/lib/cloud/aws/cloud.rb:296:in `create_stemcell'


    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/update_stemcell.rb:68:in `block in perform'


    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/base_job.rb:46:in `block in track_and_log'


    /var/vcap/deploy/bosh/director/current/director/lib/director/event_log.rb:58:in `track'


    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/base_job.rb:44:in `track_and_log'


    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/update_stemcell.rb:66:in `perform'


    /var/vcap/deploy/bosh/director/current/director/lib/director/job_runner.rb:95:in `perform_job'


    /var/vcap/deploy/bosh/director/current/director/lib/director/job_runner.rb:29:in `block in run'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_common-0.4.0/lib/common/thread_formatter.rb:46:in `with_thread_name'


    /var/vcap/deploy/bosh/director/current/director/lib/director/job_runner.rb:29:in `run'


    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/base_job.rb:8:in `perform'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/job.rb:127:in `perform'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:163:in `perform'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:130:in `block in work'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:116:in `loop'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:116:in `work'


    /var/vcap/deploy/bosh/director/current/director/bin/worker:77:in `<main>'


    I




  • Guillaume Berche at Jul 27, 2012 at 7:16 pm
    Thanks Nic for the update.
    How can I otherwise convert the bosh-stemcell-0.5.2.tgz to be used on aws
    ?

    Sorry, I had missed the stemcell builder<https://github.com/cloudfoundry/bosh/tree/master/agent/misc/stemcell/build2>and the process seems quite different between Iaas providers. Still not
    clear what stemcells without aws nor vSphere extensions are used for.

    Guillaume.
    On Thursday, July 26, 2012 10:56:34 PM UTC+2, Dr Nic Williams wrote:

    Sorry that the current "create a bosh" articles are all flawed - many are
    written in advance of Martin releasing new stemcells. When they ship I will
    proof all the tutorials again and confirm they work.

    Cheers
    Nic


    Dr Nic Williams - VP Developer Evangelism
    Engine Yard
    The Leading Platform as a Service
    Mobile: +1 415 860 2185
    Skype: nicwilliams
    Twitter: @drnic

    On Thursday, July 26, 2012 at 1:28 PM, Guillaume Berche wrote:

    Hello,

    Since a few days of git updates, I was getting the following error when
    trying to upload the bosh-stemcell-aws-0.5.1.tgz “expected string value for
    member 1 of key values of member 1 of option filters”. I seem to have fixed
    it by patching the bosh-stemcell-aws-0.5.1.tgz manifest.


    What's the impact of using an "old" stemcell for AWS
    (bosh-stemcell-aws-0.5.1.tgz) ? I see there is a 0.6.2 micro stemcell in
    review. Can it be used on AWS ? How can I otherwise convert the
    bosh-stemcell-0.5.2.tgz to be used on aws ?
    Is it worth putting some effort with aws-specific stemcells if public
    amis are in preparation ?

    BTW, is there another recommended way to use bosh on AWS than
    creating-a-bosh-from-scratch.md<https://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-bosh-from-scratch.md>(i.e. using chef) ?

    - creating-a-micro-bosh-from-stemcell.md<https://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-micro-bosh-from-stemcell.md>indicates a pending micro-bosh stemcell 0.6.1 to be published (maybe the 0.6.2
    <http://reviews.cloudfoundry.org/7757>would match it?)
    -

    uploading-public-stemcell.md<https://github.com/drnic/bosh-getting-started/blob/master/uploading-public-stemcell.md>reference a bosh-stemcell-aws-0.6.2.tgz I can't find in the list of public
    stemcells.
    -

    creating-a-micro-bosh-from-ami.md<https://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-micro-bosh-from-ami.md>mentions writing this is in progress. Is there missing prereqs to get this
    working on eu-west-1 region ?

    Some ideas, to make next diagnostics easier:
    - maybe, the create_stemcell() method could check validity of the
    manifest (expected presence of the architecture and root_device_name) and
    display a hint message in this case ?
    - the "bosh upload stemcell" command could try validating the created
    stemcell by instanciating it and pinging the instance. This would avoid
    diagnosing it in later deployments steps.


    Diagnostics and applied patches follow, I hope this can help others on AWS

    Thanks,

    Guillaume.

    I’m running on eu-west-1 using a bosh instance populated from the chef
    deployment following creating-a-bosh-from-scratch.md<https://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-bosh-from-scratch.md>

    Looking at the aki autodetection code<https://github.com/cloudfoundry/bosh/blob/master/aws_cpi/lib/cloud/aws/cloud.rb#L288>following
    7489 <http://reviews.cloudfoundry.org/7489> change, I patched the
    stemcell manifest from the original bosh-stemcell-aws-0.5.1.tgz to add the
    following (using the arch and root device name from the publicami-3d1f2b49 used on my bosh-chef instance)

    cloud_properties:
    architecture: "x86_64"
    root_device_name: "/dev/sda1"

    However, the instances did not seem to properly start (compilation jobs
    keep timeouting). Looking into the aki autoselected, it was the 'aki-60695814'
    corresponding to 'pv-grub-hd00_1.02-x86_64'. This seems to be an image for
    a partition table while previously my sucessfull tests were having the
    default raw unpartitionned disk image (cf aws doc<http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/UserProvidedkernels.html%20>).


    I tried to use /dev/sda instead to have the find_aki method pick the hd0
    aki instead, and this solved the compilation jobs timeouting.
    Full stack trace of the original issue.

    E, [2012-07-26T08:10:39.343383 #383]
    [create_stemcell(/var/vcap/deploy/tmp/stemcell20120726-383-1yv79vv/image...)]
    ERROR -- : expected string value for member 1 of key values of member 1 of
    option filters (AWS::Core::OptionGrammar::FormatError)

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:93:in
    `validate'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:209:in
    `block in validate'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in
    `each'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in
    `validate'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:310:in
    `block in validate'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:306:in
    `each'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:306:in
    `validate'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:209:in
    `block in validate'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in
    `each'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in
    `validate'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:550:in
    `block in validate'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:546:in
    `each'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:546:in
    `validate'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:561:in
    `request_params'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/configured_client_methods.rb:50:in
    `block (3 levels) in configure_client'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:409:in
    `build_request'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:350:in
    `block (3 levels) in client_request'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/response.rb:60:in
    `call'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/response.rb:60:in
    `rebuild_request'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/response.rb:55:in
    `initialize'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:171:in
    `new'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:171:in
    `new_response'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:350:in
    `block (2 levels) in client_request'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client_logging.rb:25:in
    `block in log_client_request'

    /var/vcap/deploy/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/benchmark.rb:294:in
    `measure'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client_logging.rb:24:in
    `log_client_request'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:338:in
    `block in client_request'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:275:in
    `return_or_raise'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:337:in
    `client_request'

    (eval):3:in `describe_images'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_aws_cpi-0.6.0/lib/cloud/aws/cloud.rb:362:in
    `find_aki'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_aws_cpi-0.6.0/lib/cloud/aws/cloud.rb:321:in
    `block in create_stemcell'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_common-0.4.0/lib/common/thread_formatter.rb:46:in
    `with_thread_name'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_aws_cpi-0.6.0/lib/cloud/aws/cloud.rb:296:in
    `create_stemcell'

    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/update_stemcell.rb:68:in
    `block in perform'

    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/base_job.rb:46:in
    `block in track_and_log'

    /var/vcap/deploy/bosh/director/current/director/lib/director/event_log.rb:58:in
    `track'

    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/base_job.rb:44:in
    `track_and_log'

    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/update_stemcell.rb:66:in
    `perform'

    /var/vcap/deploy/bosh/director/current/director/lib/director/job_runner.rb:95:in
    `perform_job'

    /var/vcap/deploy/bosh/director/current/director/lib/director/job_runner.rb:29:in
    `block in run'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_common-0.4.0/lib/common/thread_formatter.rb:46:in
    `with_thread_name'

    /var/vcap/deploy/bosh/director/current/director/lib/director/job_runner.rb:29:in
    `run'

    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/base_job.rb:8:in
    `perform'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/job.rb:127:in
    `perform'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:163:in
    `perform'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:130:in
    `block in work'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:116:in
    `loop'

    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:116:in
    `work'

    /var/vcap/deploy/bosh/director/current/director/bin/worker:77:in `<main>'

    I



  • Dr Nic Williams at Jul 27, 2012 at 7:20 pm
    I haven't created a stemcell yet.

    I'm not sure yet either why there are IaaS specific stemcells.

    I assume in part that stemcell/IaaS #2 (AWS) and #3 (OpenStack on some hypervisor) are new that the abstractions and similarities are only being discovered now by Martin & co.

    That's if Martin has enough energy left from doing the AWS 0.6.X stemcells to continue work in that space :)

    Nic

    Dr Nic Williams - VP Developer Evangelism
    Engine Yard
    The Leading Platform as a Service
    Mobile: +1 415 860 2185
    Skype: nicwilliams
    Twitter: @drnic

    On Friday, July 27, 2012 at 12:16 PM, Guillaume Berche wrote:

    Thanks Nic for the update.
    How can I otherwise convert the bosh-stemcell-0.5.2.tgz to be used on aws ?
    Sorry, I had missed the stemcell builder (https://github.com/cloudfoundry/bosh/tree/master/agent/misc/stemcell/build2) and the process seems quite different between Iaas providers. Still not clear what stemcells without aws nor vSphere extensions are used for.

    Guillaume.
    On Thursday, July 26, 2012 10:56:34 PM UTC+2, Dr Nic Williams wrote:
    Sorry that the current "create a bosh" articles are all flawed - many are written in advance of Martin releasing new stemcells. When they ship I will proof all the tutorials again and confirm they work.

    Cheers
    Nic


    Dr Nic Williams - VP Developer Evangelism
    Engine Yard
    The Leading Platform as a Service
    Mobile: +1 415 860 2185
    Skype: nicwilliams
    Twitter: @drnic

    On Thursday, July 26, 2012 at 1:28 PM, Guillaume Berche wrote:

    Hello,

    Since a few days of git updates, I was getting the following error when trying to upload the bosh-stemcell-aws-0.5.1.tgz “expected string value for member 1 of key values of member 1 of option filters”. I seem to have fixed it by patching the bosh-stemcell-aws-0.5.1.tgz manifest.





    What's the impact of using an "old" stemcell for AWS (bosh-stemcell-aws-0.5.1.tgz) ? I see there is a 0.6.2 micro stemcell in review. Can it be used on AWS ? How can I otherwise convert the bosh-stemcell-0.5.2.tgz to be used on aws ?


    Is it worth putting some effort with aws-specific stemcells if public amis are in preparation ?


    BTW, is there another recommended way to use bosh on AWS than creating-a-bosh-from-scratch.md (https://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-bosh-from-scratch.md) (i.e. using chef) ?


    creating-a-micro-bosh-from-stemcell.md (https://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-micro-bosh-from-stemcell.md) indicates a pending micro-bosh stemcell 0.6.1 to be published (maybe the 0.6.2 (http://reviews.cloudfoundry.org/7757)would match it?)

    uploading-public-stemcell.md (https://github.com/drnic/bosh-getting-started/blob/master/uploading-public-stemcell.md) reference a bosh-stemcell-aws-0.6.2.tgz I can't find in the list of public stemcells.



    creating-a-micro-bosh-from-ami.md (https://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-micro-bosh-from-ami.md) mentions writing this is in progress. Is there missing prereqs to get this working on eu-west-1 region ?




    Some ideas, to make next diagnostics easier:
    - maybe, the create_stemcell() method could check validity of the manifest (expected presence of the architecture and root_device_name) and display a hint message in this case ?
    - the "bosh upload stemcell" command could try validating the created stemcell by instanciating it and pinging the instance. This would avoid diagnosing it in later deployments steps.




    Diagnostics and applied patches follow, I hope this can help others on AWS

    Thanks,

    Guillaume.

    I’m running on eu-west-1 using a bosh instance populated from the chef deployment following creating-a-bosh-from-scratch.md (https://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-bosh-from-scratch.md)

    Looking at the aki autodetection code (https://github.com/cloudfoundry/bosh/blob/master/aws_cpi/lib/cloud/aws/cloud.rb#L288) following 7489 (http://reviews.cloudfoundry.org/7489) change, I patched the stemcell manifest from the original bosh-stemcell-aws-0.5.1.tgz to add the following (using the arch and root device name from the public ami-3d1f2b49 used on my bosh-chef instance)



    cloud_properties:
    architecture: "x86_64"
    root_device_name: "/dev/sda1"






    However, the instances did not seem to properly start (compilation jobs keep timeouting). Looking into the aki autoselected, it was the 'aki-60695814' corresponding to 'pv-grub-hd00_1.02-x86_64'. This seems to be an image for a partition table while previously my sucessfull tests were having the default raw unpartitionned disk image (cf aws doc (http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/UserProvidedkernels.html%20)).



    I tried to use /dev/sda instead to have the find_aki method pick the hd0 aki instead, and this solved the compilation jobs timeouting.


    Full stack trace of the original issue.


    E, [2012-07-26T08:10:39.343383 #383] [create_stemcell(/var/vcap/deploy/tmp/stemcell20120726-383-1yv79vv/image...)] ERROR -- : expected string value for member 1 of key values of member 1 of option filters (AWS::Core::OptionGrammar::FormatError)


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:93:in `validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:209:in `block in validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in `each'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in `validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:310:in `block in validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:306:in `each'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:306:in `validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:209:in `block in validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in `each'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in `validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:550:in `block in validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:546:in `each'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:546:in `validate'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:561:in `request_params'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/configured_client_methods.rb:50:in `block (3 levels) in configure_client'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:409:in `build_request'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:350:in `block (3 levels) in client_request'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/response.rb:60:in `call'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/response.rb:60:in `rebuild_request'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/response.rb:55:in `initialize'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:171:in `new'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:171:in `new_response'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:350:in `block (2 levels) in client_request'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client_logging.rb:25:in `block in log_client_request'


    /var/vcap/deploy/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/benchmark.rb:294:in `measure'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client_logging.rb:24:in `log_client_request'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:338:in `block in client_request'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:275:in `return_or_raise'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:337:in `client_request'


    (eval):3:in `describe_images'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_aws_cpi-0.6.0/lib/cloud/aws/cloud.rb:362:in `find_aki'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_aws_cpi-0.6.0/lib/cloud/aws/cloud.rb:321:in `block in create_stemcell'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_common-0.4.0/lib/common/thread_formatter.rb:46:in `with_thread_name'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_aws_cpi-0.6.0/lib/cloud/aws/cloud.rb:296:in `create_stemcell'


    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/update_stemcell.rb:68:in `block in perform'


    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/base_job.rb:46:in `block in track_and_log'


    /var/vcap/deploy/bosh/director/current/director/lib/director/event_log.rb:58:in `track'


    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/base_job.rb:44:in `track_and_log'


    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/update_stemcell.rb:66:in `perform'


    /var/vcap/deploy/bosh/director/current/director/lib/director/job_runner.rb:95:in `perform_job'


    /var/vcap/deploy/bosh/director/current/director/lib/director/job_runner.rb:29:in `block in run'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_common-0.4.0/lib/common/thread_formatter.rb:46:in `with_thread_name'


    /var/vcap/deploy/bosh/director/current/director/lib/director/job_runner.rb:29:in `run'


    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/base_job.rb:8:in `perform'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/job.rb:127:in `perform'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:163:in `perform'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:130:in `block in work'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:116:in `loop'


    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:116:in `work'


    /var/vcap/deploy/bosh/director/current/director/bin/worker:77:in `<main>'


    I





  • Martin Englund at Jul 27, 2012 at 7:51 pm

    On Jul 27, 2012, at 12:19, Dr Nic Williams wrote:

    I haven't created a stemcell yet.
    cd agent
    rake stemcell2:basic[aws]
    I'm not sure yet either why there are IaaS specific stemcells.
    Because we need to use different ways to bootstrap the initial configuration for the VMs.
    I assume in part that stemcell/IaaS #2 (AWS) and #3 (OpenStack on some hypervisor) are new that the abstractions and similarities are only being discovered now by Martin & co.

    That's if Martin has enough energy left from doing the AWS 0.6.X stemcells to continue work in that space :)
    Pfft! I need a break from CPI stuff once I'm done with AWS, it is boring and tedious work ;)

    /M
    Nic

    Dr Nic Williams - VP Developer Evangelism
    Engine Yard
    The Leading Platform as a Service
    Mobile: +1 415 860 2185
    Skype: nicwilliams
    Twitter: @drnic
    On Friday, July 27, 2012 at 12:16 PM, Guillaume Berche wrote:

    Thanks Nic for the update.
    How can I otherwise convert the bosh-stemcell-0.5.2.tgz to be used on aws ?
    Sorry, I had missed the stemcell builder and the process seems quite different between Iaas providers. Still not clear what stemcells without aws nor vSphere extensions are used for.

    Guillaume.
    On Thursday, July 26, 2012 10:56:34 PM UTC+2, Dr Nic Williams wrote:

    Sorry that the current "create a bosh" articles are all flawed - many are written in advance of Martin releasing new stemcells. When they ship I will proof all the tutorials again and confirm they work.

    Cheers
    Nic


    Dr Nic Williams - VP Developer Evangelism
    Engine Yard
    The Leading Platform as a Service
    Mobile: +1 415 860 2185
    Skype: nicwilliams
    Twitter: @drnic
    On Thursday, July 26, 2012 at 1:28 PM, Guillaume Berche wrote:

    Hello,

    Since a few days of git updates, I was getting the following error when trying to upload the bosh-stemcell-aws-0.5.1.tgz “expected string value for member 1 of key values of member 1 of option filters”. I seem to have fixed it by patching the bosh-stemcell-aws-0.5.1.tgz manifest.

    What's the impact of using an "old" stemcell for AWS (bosh-stemcell-aws-0.5.1.tgz) ? I see there is a 0.6.2 micro stemcell in review. Can it be used on AWS ? How can I otherwise convert the bosh-stemcell-0.5.2.tgz to be used on aws ?
    Is it worth putting some effort with aws-specific stemcells if public amis are in preparation ?

    BTW, is there another recommended way to use bosh on AWS than creating-a-bosh-from-scratch.md (i.e. using chef) ?
    creating-a-micro-bosh-from-stemcell.md indicates a pending micro-bosh stemcell 0.6.1 to be published (maybe the 0.6.2 would match it?)
    uploading-public-stemcell.md reference a bosh-stemcell-aws-0.6.2.tgz I can't find in the list of public stemcells.
    creating-a-micro-bosh-from-ami.md mentions writing this is in progress. Is there missing prereqs to get this working on eu-west-1 region ?
    Some ideas, to make next diagnostics easier:
    - maybe, the create_stemcell() method could check validity of the manifest (expected presence of the architecture and root_device_name) and display a hint message in this case ?
    - the "bosh upload stemcell" command could try validating the created stemcell by instanciating it and pinging the instance. This would avoid diagnosing it in later deployments steps.

    Diagnostics and applied patches follow, I hope this can help others on AWS

    Thanks,

    Guillaume.

    I’m running on eu-west-1 using a bosh instance populated from the chef deployment following creating-a-bosh-from-scratch.md
    Looking at the aki autodetection code following 7489 change, I patched the stemcell manifest from the original bosh-stemcell-aws-0.5.1.tgz to add the following (using the arch and root device name from the public ami-3d1f2b49 used on my bosh-chef instance)
    cloud_properties:
    architecture: "x86_64"
    root_device_name: "/dev/sda1"
    However, the instances did not seem to properly start (compilation jobs keep timeouting). Looking into the aki autoselected, it was the 'aki-60695814' corresponding to 'pv-grub-hd00_1.02-x86_64'. This seems to be an image for a partition table while previously my sucessfull tests were having the default raw unpartitionned disk image (cf aws doc).
    I tried to use /dev/sda instead to have the find_aki method pick the hd0 aki instead, and this solved the compilation jobs timeouting.
    Full stack trace of the original issue.

    E, [2012-07-26T08:10:39.343383 #383] [create_stemcell(/var/vcap/deploy/tmp/stemcell20120726-383-1yv79vv/image...)] ERROR -- : expected string value for member 1 of key values of member 1 of option filters (AWS::Core::OptionGrammar::FormatError)
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:93:in `validate'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:209:in `block in validate'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in `each'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in `validate'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:310:in `block in validate'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:306:in `each'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:306:in `validate'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:209:in `block in validate'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in `each'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:207:in `validate'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:550:in `block in validate'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:546:in `each'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:546:in `validate'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/option_grammar.rb:561:in `request_params'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/configured_client_methods.rb:50:in `block (3 levels) in configure_client'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:409:in `build_request'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:350:in `block (3 levels) in client_request'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/response.rb:60:in `call'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/response.rb:60:in `rebuild_request'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/response.rb:55:in `initialize'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:171:in `new'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:171:in `new_response'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:350:in `block (2 levels) in client_request'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client_logging.rb:25:in `block in log_client_request'
    /var/vcap/deploy/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/benchmark.rb:294:in `measure'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client_logging.rb:24:in `log_client_request'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:338:in `block in client_request'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:275:in `return_or_raise'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/aws-sdk-1.3.8/lib/aws/core/client.rb:337:in `client_request'
    (eval):3:in `describe_images'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_aws_cpi-0.6.0/lib/cloud/aws/cloud.rb:362:in `find_aki'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_aws_cpi-0.6.0/lib/cloud/aws/cloud.rb:321:in `block in create_stemcell'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_common-0.4.0/lib/common/thread_formatter.rb:46:in `with_thread_name'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_aws_cpi-0.6.0/lib/cloud/aws/cloud.rb:296:in `create_stemcell'
    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/update_stemcell.rb:68:in `block in perform'
    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/base_job.rb:46:in `block in track_and_log'
    /var/vcap/deploy/bosh/director/current/director/lib/director/event_log.rb:58:in `track'
    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/base_job.rb:44:in `track_and_log'
    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/update_stemcell.rb:66:in `perform'
    /var/vcap/deploy/bosh/director/current/director/lib/director/job_runner.rb:95:in `perform_job'
    /var/vcap/deploy/bosh/director/current/director/lib/director/job_runner.rb:29:in `block in run'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/bosh_common-0.4.0/lib/common/thread_formatter.rb:46:in `with_thread_name'
    /var/vcap/deploy/bosh/director/current/director/lib/director/job_runner.rb:29:in `run'
    /var/vcap/deploy/bosh/director/current/director/lib/director/jobs/base_job.rb:8:in `perform'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/job.rb:127:in `perform'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:163:in `perform'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:130:in `block in work'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:116:in `loop'
    /var/vcap/deploy/bosh/director/shared/gems/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:116:in `work'
    /var/vcap/deploy/bosh/director/current/director/bin/worker:77:in `<main>'
    I

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupbosh-users @
postedJul 26, '12 at 8:28p
activeJul 27, '12 at 7:51p
posts5
users3

People

Translate

site design / logo © 2022 Grokbase