FAQ
Hi !
I need your help for a wierd problem...

I tried a demo with multiple VM :

VM A : Nats server + Cloud Controller + Health Manager + Gateways (Redis +
Mysql + Mongodb)
VM B : DEA + Router1
VM C : DEA
VM D : MySQL + Router2
VM E : MySQL_node + Redis_node

This strange configuration is just to try different things, but anyway the
problem is not there.

Everything worked well yesterday, but after some modifications with the A
deployment config (I just tried to leave the mongodb gateway), and some
changes on /etc/hosts of differents VM (because I noticed I had to target a
router and not the cloud_controller to use vmc...).

I restarted the different machines, and in then things became strange.

I can't provision any service now, when I try it :

vmc create-service mysql
Creating Service [mysql-ef04a]: Error 503: Unexpected response from service
gateway

Here are different logs :

-------------------------- Cloud Controller
-----------------------------------------------


user_email decoded from token is "thomas@bar.com"
cloud_controller --> [2012-08-08 10:29:50.224037] cc - pid=20622 tid=af41
fid=bc71 DEBUG -- Attempting to provision service:
{"type":"database","tier":"free","vendor":"mysql","version":"5.1","name":"mysql-ef04a"}
cloud_controller --> [2012-08-08 10:29:50.235038] cc - pid=20622 tid=af41
fid=bc71 ERROR -- Error talking to gateway: Can't decode gateway response.
status code:0, response body:
cloud_controller --> [2012-08-08 10:29:50.235216] cc exception pid=20622
tid=af41 fid=bc71 ERROR --
VCAP::Services::Api::ServiceGatewayClient::UnexpectedResponse<<Can't decode
gateway response. status code:0, response
body::/usr/local/rvm/gems/ruby-1.9.2-p320/bundler/gems/vcap-common-1dca468ec017/lib/services/api/clients/service_gateway_client.rb:173:in
`rescue in
perform_request',/usr/local/rvm/gems/ruby-1.9.2-p320/bundler/gems/vcap-common-1dca468ec017/lib/services/api/clients/service_gateway_client.rb:169:in
`perform_request',/usr/local/rvm/gems/ruby-1.9.2-p320/bundler/gems/vcap-common-1dca468ec017/lib/services/api/clients/service_gateway_client.rb:77:in
`provision',/root/cloudfoundry/cloud_controller/cloud_controller/app/models/service_config.rb:64:in
`block in
provision',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/connection_adapters/abstract/database_statements.rb:139:in
`transaction',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/transactions.rb:207:in
`transaction',/root/cloudfoundry/cloud_controller/cloud_controller/app/models/service_config.rb:45:in
`provision',/root/cloudfoundry/cloud_controller/cloud_controller/app/controllers/legacy_services_controller.rb:64:in
`provision',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/implicit_render.rb:4:in
`send_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/base.rb:150:in
`process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/rendering.rb:11:in
`process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/callbacks.rb:18:in
`block in
process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/callbacks.rb:441:in
`_run__537527495993718278__process_action__2846389414622395934__callbacks',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/callbacks.rb:410:in
`_run_process_action_callbacks',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/callbacks.rb:94:in
`run_callbacks',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/callbacks.rb:17:in
`process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/rescue.rb:17:in
`process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/instrumentation.rb:30:in
`block in
process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/notifications.rb:52:in
`block in
instrument',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/notifications/instrumenter.rb:21:in
`instrument',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/notifications.rb:52:in
`instrument',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/instrumentation.rb:29:in
`process_action',/root/cloudfoundry/cloud_controller/cloud_controller/app/controllers/application_controller.rb:16:in
`process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/base.rb:119:in
`process',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/rendering.rb:41:in
`process',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal.rb:138:in
`dispatch',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/rack_delegation.rb:14:in
`dispatch',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal.rb:178:in
`block in
action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/routing/route_set.rb:68:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/routing/route_set.rb:68:in
`dispatch',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/routing/route_set.rb:33:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/route_set.rb:148:in
`block in
call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/code_generation.rb:93:in
`block in
recognize',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/code_generation.rb:68:in
`optimized_each',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/code_generation.rb:92:in
`recognize',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/route_set.rb:139:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/routing/route_set.rb:499:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/best_standards_support.rb:17:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/head.rb:14:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-1.2.5/lib/rack/methodoverride.rb:24:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/params_parser.rb:21:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/cookies.rb:302:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/query_cache.rb:32:in
`block in
call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/connection_adapters/abstract/query_cache.rb:28:in
`cache',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/query_cache.rb:12:in
`cache',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/query_cache.rb:31:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/connection_adapters/abstract/connection_pool.rb:354:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/callbacks.rb:46:in
`block in
call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/callbacks.rb:416:in
`_run_call_callbacks',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/callbacks.rb:44:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-1.2.5/lib/rack/sendfile.rb:106:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/remote_ip.rb:48:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/show_exceptions.rb:47:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/railties-3.0.14/lib/rails/rack/logger.rb:13:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/rack/fiber_pool.rb:19:in
`block in
call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/fiber_pool.rb:48:in
`call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/fiber_pool.rb:48:in
`block (3 levels) in
initialize',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/fiber_pool.rb:47:in
`loop',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/fiber_pool.rb:47:in
`block (2 levels) in
initialize',/root/cloudfoundry/cloud_controller/cloud_controller/config/initializers/fiber_connection_pool.rb:11:in
`call',/root/cloudfoundry/cloud_controller/cloud_controller/config/initializers/fiber_connection_pool.rb:11:in
`block in initialize'>>
cloud_controller --> [2012-08-08 10:29:50.241568] cc_events - pid=20622
tid=af41 fid=bc71 INFO -- [2012-08-08 10:29:50 +0200, :USER, "thomas@bar.com",
"POST:/services", "mysql", "mysql-ef04a", :FAILED, "Unexpected response
from service gateway"]

----------------------------------------------------------------------------------------------------------------------------

------------------------------- MySQL Gateway
-----------------------------------------------------------------------


mysql_gateway --> [2012-08-08 10:28:48.608498] mysql_gateway - pid=21286
tid=4d90 fid=5eb3 ERROR -- Failed fetching handles, status=404
mysql_gateway --> [2012-08-08 10:28:49.510232] mysql_gateway - pid=21286
tid=4d90 fid=5eb3 INFO -- Fetching handles from cloud controller @
http://api.vcap.me/services/v1/offerings/mysql-5.1/handles
mysql_gateway --> [2012-08-08 10:28:49.618446] mysql_gateway - pid=21286
tid=4d90 fid=5eb3 ERROR -- Failed fetching handles, status=404
mysql_gateway --> [2012-08-08 10:28:50.446124] mysql_gateway - pid=21286
tid=4d90 fid=5eb3 DEBUG -- [MyaaS-Provisioner] Received node announcement:
{"available_capacity":47,"capacity_unit":1,"id":"mysql_node_1","plan":"free","supported_versions":["5.1"]}
mysql_gateway --> [2012-08-08 10:28:50.536661] mysql_gateway - pid=21286
tid=4d90 fid=5eb3 INFO -- Fetching handles from cloud controller @
http://api.vcap.me/services/v1/offerings/mysql-5.1/handles
mysql_gateway --> [2012-08-08 10:28:50.655898] mysql_gateway - pid=21286
tid=4d90 fid=5eb3 ERROR -- Failed fetching handles, status=404
mysql_gateway --> [2012-08-08 10:28:51.557835] mysql_gateway - pid=21286
tid=4d90 fid=5eb3 INFO -- Fetching handles from cloud controller @
http://api.vcap.me/services/v1/offerings/mysql-5.1/handles

---------------------------------------------------------------------------------------------------------------------

----------------------------- Mysql_nodes
----------------------------------------------------------------------

70 fid=28b6 DEBUG -- MyaaS-Node: Sending announcement for everyone
mysql_node --> [2012-08-08 11:01:54.029298] mysql_node_1 - pid=19783 tid=2c
70 fid=28b6 DEBUG -- MyaaS-Node: Sending announcement for everyone
mysql_node --> [2012-08-08 11:02:24.104521] mysql_node_1 - pid=19783 tid=2c
70 fid=28b6 DEBUG -- MyaaS-Node: Sending announcement for everyone

----------------------------------------------------------------------------------------------------------------

So the Cloud_controller receive some datas from the gateway, but don't know
why he can't decode it. The nodes broadcast their capacity but I just saw
this availability once in the gateway logs, during the request.

Do you have any Idea to help me ? Thanks in advance!

Search Discussions

  • Haipeng Wu at Aug 9, 2012 at 8:26 am
    change list below may help you.
    https://github.com/cloudfoundry/vcap/commit/e867e10f3e2043801e81ff1fa43f7df265a03426

    haipeng
    On Thu, Aug 9, 2012 at 4:11 PM, bbqjack wrote:

    Hi !
    I need your help for a wierd problem...

    I tried a demo with multiple VM :

    VM A : Nats server + Cloud Controller + Health Manager + Gateways (Redis +
    Mysql + Mongodb)
    VM B : DEA + Router1
    VM C : DEA
    VM D : MySQL + Router2
    VM E : MySQL_node + Redis_node

    This strange configuration is just to try different things, but anyway the
    problem is not there.

    Everything worked well yesterday, but after some modifications with the A
    deployment config (I just tried to leave the mongodb gateway), and some
    changes on /etc/hosts of differents VM (because I noticed I had to target a
    router and not the cloud_controller to use vmc...).

    I restarted the different machines, and in then things became strange.

    I can't provision any service now, when I try it :

    vmc create-service mysql
    Creating Service [mysql-ef04a]: Error 503: Unexpected response from
    service gateway

    Here are different logs :

    -------------------------- Cloud Controller ------------------------------
    **-----------------


    user_email decoded from token is "thomas@bar.com"
    cloud_controller --> [2012-08-08 10:29:50.224037] cc - pid=20622 tid=af41
    fid=bc71 DEBUG -- Attempting to provision service:
    {"type":"database","tier":"**free","vendor":"mysql","**
    version":"5.1","name":"mysql-**ef04a"}
    cloud_controller --> [2012-08-08 10:29:50.235038] cc - pid=20622 tid=af41
    fid=bc71 ERROR -- Error talking to gateway: Can't decode gateway response.
    status code:0, response body:
    cloud_controller --> [2012-08-08 10:29:50.235216] cc exception pid=20622
    tid=af41 fid=bc71 ERROR -- VCAP::Services::Api::**ServiceGatewayClient::**UnexpectedResponse<<Can't
    decode gateway response. status code:0, response body::/usr/local/rvm/gems/
    **ruby-1.9.2-p320/bundler/gems/**vcap-common-1dca468ec017/lib/**
    services/api/clients/service_**gateway_client.rb:173:in `rescue in
    perform_request',/usr/local/**rvm/gems/ruby-1.9.2-p320/**
    bundler/gems/vcap-common-**1dca468ec017/lib/services/api/**
    clients/service_gateway_**client.rb:169:in `perform_request',/usr/local/**
    rvm/gems/ruby-1.9.2-p320/**bundler/gems/vcap-common-**
    1dca468ec017/lib/services/api/**clients/service_gateway_**client.rb:77:in
    `provision',/root/**cloudfoundry/cloud_controller/**
    cloud_controller/app/models/**service_config.rb:64:in `block in
    provision',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    activerecord-3.0.14/lib/**active_record/connection_**
    adapters/abstract/database_**statements.rb:139:in
    `transaction',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    activerecord-3.0.14/lib/**active_record/transactions.rb:**207:in
    `transaction',/root/**cloudfoundry/cloud_controller/**
    cloud_controller/app/models/**service_config.rb:45:in `provision',/root/**
    cloudfoundry/cloud_controller/**cloud_controller/app/**
    controllers/legacy_services_**controller.rb:64:in
    `provision',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal/implicit_**render.rb:4:in
    `send_action',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/**abstract_controller/base.rb:**150:in
    `process_action',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal/rendering.rb:**11:in
    `process_action',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/**abstract_controller/callbacks.**rb:18:in `block
    in process_action',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/callbacks.rb:**441:in
    `_run__537527495993718278__**process_action__**2846389414622395934__**
    callbacks',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/callbacks.rb:**410:in
    `_run_process_action_**callbacks',/usr/local/rvm/**
    gems/ruby-1.9.2-p320/gems/**activesupport-3.0.14/lib/**
    active_support/callbacks.rb:**94:in `run_callbacks',/usr/local/**
    rvm/gems/ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/**
    abstract_controller/callbacks.**rb:17:in `process_action',/usr/local/**
    rvm/gems/ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/action_**
    controller/metal/rescue.rb:17:**in `process_action',/usr/local/**
    rvm/gems/ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/action_**
    controller/metal/**instrumentation.rb:30:in `block in
    process_action',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/notifications.**rb:52:in `block
    in instrument',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/notifications/**instrumenter.rb:21:in
    `instrument',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/notifications.**rb:52:in
    `instrument',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal/**instrumentation.rb:29:in
    `process_action',/root/**cloudfoundry/cloud_controller/**
    cloud_controller/app/**controllers/application_**controller.rb:16:in
    `process_action',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/**abstract_controller/base.rb:**119:in
    `process',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/**abstract_controller/rendering.**rb:41:in
    `process',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal.rb:138:in
    `dispatch',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal/rack_**delegation.rb:14:in
    `dispatch',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal.rb:178:in `block in
    action',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/routing/route_set.rb:**68:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/routing/route_set.rb:**68:in
    `dispatch',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/routing/route_set.rb:**33:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-**
    mount-0.6.14/lib/rack/mount/**route_set.rb:148:in `block in
    call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-**
    mount-0.6.14/lib/rack/mount/**code_generation.rb:93:in `block in
    recognize',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    rack-mount-0.6.14/lib/rack/**mount/code_generation.rb:68:in
    `optimized_each',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    rack-mount-0.6.14/lib/rack/**mount/code_generation.rb:92:in
    `recognize',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    rack-mount-0.6.14/lib/rack/**mount/route_set.rb:139:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/routing/route_set.rb:**499:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/middleware/best_**standards_support.rb:17:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/middleware/head.rb:**14:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-1.2.**
    5/lib/rack/methodoverride.rb:**24:in `call',/usr/local/rvm/gems/**
    ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/action_**
    dispatch/middleware/params_**parser.rb:21:in `call',/usr/local/rvm/gems/**
    ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/action_**
    dispatch/middleware/cookies.**rb:302:in `call',/usr/local/rvm/gems/**
    ruby-1.9.2-p320/gems/**activerecord-3.0.14/lib/**
    active_record/query_cache.rb:**32:in `block in call',/usr/local/rvm/gems/*
    *ruby-1.9.2-p320/gems/**activerecord-3.0.14/lib/**
    active_record/connection_**adapters/abstract/query_cache.**rb:28:in
    `cache',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    activerecord-3.0.14/lib/**active_record/query_cache.rb:**12:in
    `cache',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    activerecord-3.0.14/lib/**active_record/query_cache.rb:**31:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    activerecord-3.0.14/lib/**active_record/connection_**
    adapters/abstract/connection_**pool.rb:354:in `call',/usr/local/rvm/gems/*
    *ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/action_**
    dispatch/middleware/callbacks.**rb:46:in `block in
    call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/callbacks.rb:**416:in
    `_run_call_callbacks',/usr/**local/rvm/gems/ruby-1.9.2-**
    p320/gems/actionpack-3.0.14/**lib/action_dispatch/**middleware/callbacks.rb:44:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-1.2.**5/lib/rack/sendfile.rb:106:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/middleware/remote_ip.**rb:48:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/middleware/show_**exceptions.rb:47:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/railties-**
    3.0.14/lib/rails/rack/logger.**rb:13:in `call',/usr/local/rvm/gems/**
    ruby-1.9.2-p320/gems/rack-**fiber_pool-0.9.1/lib/rack/**fiber_pool.rb:19:in
    `block in call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-**
    fiber_pool-0.9.1/lib/fiber_**pool.rb:48:in `call',/usr/local/rvm/gems/**
    ruby-1.9.2-p320/gems/rack-**fiber_pool-0.9.1/lib/fiber_**pool.rb:48:in
    `block (3 levels) in initialize',/usr/local/rvm/**
    gems/ruby-1.9.2-p320/gems/**rack-fiber_pool-0.9.1/lib/**fiber_pool.rb:47:in
    `loop',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-**
    fiber_pool-0.9.1/lib/fiber_**pool.rb:47:in `block (2 levels) in
    initialize',/root/**cloudfoundry/cloud_controller/**
    cloud_controller/config/**initializers/fiber_connection_**pool.rb:11:in
    `call',/root/cloudfoundry/**cloud_controller/cloud_**controller/config/**
    initializers/fiber_connection_**pool.rb:11:in `block in initialize'>>
    cloud_controller --> [2012-08-08 10:29:50.241568] cc_events - pid=20622
    tid=af41 fid=bc71 INFO -- [2012-08-08 10:29:50 +0200, :USER, "
    thomas@bar.com", "POST:/services", "mysql", "mysql-ef04a", :FAILED,
    "Unexpected response from service gateway"]

    ------------------------------**------------------------------**
    ------------------------------**------------------------------**----

    ------------------------------**- MySQL Gateway
    ------------------------------**------------------------------**
    -----------


    mysql_gateway --> [2012-08-08 10:28:48.608498] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 ERROR -- Failed fetching handles, status=404
    mysql_gateway --> [2012-08-08 10:28:49.510232] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 INFO -- Fetching handles from cloud controller @
    http://api.vcap.me/services/**v1/offerings/mysql-5.1/handles<http://api.vcap.me/services/v1/offerings/mysql-5.1/handles>
    mysql_gateway --> [2012-08-08 10:28:49.618446] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 ERROR -- Failed fetching handles, status=404
    mysql_gateway --> [2012-08-08 10:28:50.446124] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 DEBUG -- [MyaaS-Provisioner] Received node announcement:
    {"available_capacity":47,"**capacity_unit":1,"id":"mysql_**
    node_1","plan":"free","**supported_versions":["5.1"]}
    mysql_gateway --> [2012-08-08 10:28:50.536661] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 INFO -- Fetching handles from cloud controller @
    http://api.vcap.me/services/**v1/offerings/mysql-5.1/handles<http://api.vcap.me/services/v1/offerings/mysql-5.1/handles>
    mysql_gateway --> [2012-08-08 10:28:50.655898] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 ERROR -- Failed fetching handles, status=404
    mysql_gateway --> [2012-08-08 10:28:51.557835] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 INFO -- Fetching handles from cloud controller @
    http://api.vcap.me/services/**v1/offerings/mysql-5.1/handles<http://api.vcap.me/services/v1/offerings/mysql-5.1/handles>

    ------------------------------**------------------------------**
    ------------------------------**---------------------------

    ----------------------------- Mysql_nodes ------------------------------**
    ------------------------------**----------

    70 fid=28b6 DEBUG -- MyaaS-Node: Sending announcement for everyone
    mysql_node --> [2012-08-08 11:01:54.029298] mysql_node_1 - pid=19783
    tid=2c 70 fid=28b6 DEBUG -- MyaaS-Node: Sending announcement for everyone
    mysql_node --> [2012-08-08 11:02:24.104521] mysql_node_1 - pid=19783
    tid=2c 70 fid=28b6 DEBUG -- MyaaS-Node: Sending announcement for everyone

    ------------------------------**------------------------------**
    ------------------------------**----------------------

    So the Cloud_controller receive some datas from the gateway, but don't
    know why he can't decode it. The nodes broadcast their capacity but I just
    saw this availability once in the gateway logs, during the request.

    Do you have any Idea to help me ? Thanks in advance!
  • Bbqjack at Aug 9, 2012 at 8:31 am
    Sorry, but I didn't really understand what I have to do... :/

    Le jeudi 9 août 2012 10:26:33 UTC+2, Haipeng Wu a écrit :
    change list below may help you.

    https://github.com/cloudfoundry/vcap/commit/e867e10f3e2043801e81ff1fa43f7df265a03426

    haipeng

    On Thu, Aug 9, 2012 at 4:11 PM, bbqjack <barbecu...@gmail.com<javascript:>
    wrote:
    Hi !
    I need your help for a wierd problem...

    I tried a demo with multiple VM :

    VM A : Nats server + Cloud Controller + Health Manager + Gateways (Redis
    + Mysql + Mongodb)
    VM B : DEA + Router1
    VM C : DEA
    VM D : MySQL + Router2
    VM E : MySQL_node + Redis_node

    This strange configuration is just to try different things, but anyway
    the problem is not there.

    Everything worked well yesterday, but after some modifications with the A
    deployment config (I just tried to leave the mongodb gateway), and some
    changes on /etc/hosts of differents VM (because I noticed I had to target a
    router and not the cloud_controller to use vmc...).

    I restarted the different machines, and in then things became strange.

    I can't provision any service now, when I try it :

    vmc create-service mysql
    Creating Service [mysql-ef04a]: Error 503: Unexpected response from
    service gateway

    Here are different logs :

    -------------------------- Cloud Controller ------------------------------
    **-----------------


    user_email decoded from token is "tho...@bar.com <javascript:>"
    cloud_controller --> [2012-08-08 10:29:50.224037] cc - pid=20622 tid=af41
    fid=bc71 DEBUG -- Attempting to provision service:
    {"type":"database","tier":"**free","vendor":"mysql","**
    version":"5.1","name":"mysql-**ef04a"}
    cloud_controller --> [2012-08-08 10:29:50.235038] cc - pid=20622 tid=af41
    fid=bc71 ERROR -- Error talking to gateway: Can't decode gateway response.
    status code:0, response body:
    cloud_controller --> [2012-08-08 10:29:50.235216] cc exception pid=20622
    tid=af41 fid=bc71 ERROR -- VCAP::Services::Api::**ServiceGatewayClient::*
    *UnexpectedResponse<<Can't decode gateway response. status code:0,
    response body::/usr/local/rvm/gems/**ruby-1.9.2-p320/bundler/gems/**
    vcap-common-1dca468ec017/lib/**services/api/clients/service_**gateway_client.rb:173:in
    `rescue in perform_request',/usr/local/**rvm/gems/ruby-1.9.2-p320/**
    bundler/gems/vcap-common-**1dca468ec017/lib/services/api/**
    clients/service_gateway_**client.rb:169:in `perform_request',/usr/local/*
    *rvm/gems/ruby-1.9.2-p320/**bundler/gems/vcap-common-**
    1dca468ec017/lib/services/api/**clients/service_gateway_**client.rb:77:in
    `provision',/root/**cloudfoundry/cloud_controller/**
    cloud_controller/app/models/**service_config.rb:64:in `block in
    provision',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    activerecord-3.0.14/lib/**active_record/connection_**
    adapters/abstract/database_**statements.rb:139:in
    `transaction',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    activerecord-3.0.14/lib/**active_record/transactions.rb:**207:in
    `transaction',/root/**cloudfoundry/cloud_controller/**
    cloud_controller/app/models/**service_config.rb:45:in `provision',/root/*
    *cloudfoundry/cloud_controller/**cloud_controller/app/**
    controllers/legacy_services_**controller.rb:64:in
    `provision',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal/implicit_**render.rb:4:in
    `send_action',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/**abstract_controller/base.rb:**150:in
    `process_action',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal/rendering.rb:**11:in
    `process_action',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/**abstract_controller/callbacks.**rb:18:in `block
    in process_action',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/callbacks.rb:**441:in
    `_run__537527495993718278__**process_action__**2846389414622395934__**
    callbacks',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/callbacks.rb:**410:in
    `_run_process_action_**callbacks',/usr/local/rvm/**
    gems/ruby-1.9.2-p320/gems/**activesupport-3.0.14/lib/**
    active_support/callbacks.rb:**94:in `run_callbacks',/usr/local/**
    rvm/gems/ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/**
    abstract_controller/callbacks.**rb:17:in `process_action',/usr/local/**
    rvm/gems/ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/action_**
    controller/metal/rescue.rb:17:**in `process_action',/usr/local/**
    rvm/gems/ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/action_**
    controller/metal/**instrumentation.rb:30:in `block in
    process_action',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/notifications.**rb:52:in
    `block in instrument',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/notifications/**instrumenter.rb:21:in
    `instrument',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/notifications.**rb:52:in
    `instrument',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal/**instrumentation.rb:29:in
    `process_action',/root/**cloudfoundry/cloud_controller/**
    cloud_controller/app/**controllers/application_**controller.rb:16:in
    `process_action',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/**abstract_controller/base.rb:**119:in
    `process',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/**abstract_controller/rendering.**rb:41:in
    `process',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal.rb:138:in
    `dispatch',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal/rack_**delegation.rb:14:in
    `dispatch',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal.rb:178:in `block in
    action',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/routing/route_set.rb:**68:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/routing/route_set.rb:**68:in
    `dispatch',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/routing/route_set.rb:**33:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-**
    mount-0.6.14/lib/rack/mount/**route_set.rb:148:in `block in
    call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-**
    mount-0.6.14/lib/rack/mount/**code_generation.rb:93:in `block in
    recognize',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    rack-mount-0.6.14/lib/rack/**mount/code_generation.rb:68:in
    `optimized_each',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    rack-mount-0.6.14/lib/rack/**mount/code_generation.rb:92:in
    `recognize',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    rack-mount-0.6.14/lib/rack/**mount/route_set.rb:139:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/routing/route_set.rb:**499:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/middleware/best_**standards_support.rb:17:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/middleware/head.rb:**14:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-1.2.**
    5/lib/rack/methodoverride.rb:**24:in `call',/usr/local/rvm/gems/**
    ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/action_**
    dispatch/middleware/params_**parser.rb:21:in `call',/usr/local/rvm/gems/*
    *ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/action_**
    dispatch/middleware/cookies.**rb:302:in `call',/usr/local/rvm/gems/**
    ruby-1.9.2-p320/gems/**activerecord-3.0.14/lib/**
    active_record/query_cache.rb:**32:in `block in call',/usr/local/rvm/gems/
    **ruby-1.9.2-p320/gems/**activerecord-3.0.14/lib/**
    active_record/connection_**adapters/abstract/query_cache.**rb:28:in
    `cache',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    activerecord-3.0.14/lib/**active_record/query_cache.rb:**12:in
    `cache',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    activerecord-3.0.14/lib/**active_record/query_cache.rb:**31:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    activerecord-3.0.14/lib/**active_record/connection_**
    adapters/abstract/connection_**pool.rb:354:in `call',/usr/local/rvm/gems/
    **ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/action_**
    dispatch/middleware/callbacks.**rb:46:in `block in
    call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/callbacks.rb:**416:in
    `_run_call_callbacks',/usr/**local/rvm/gems/ruby-1.9.2-**
    p320/gems/actionpack-3.0.14/**lib/action_dispatch/**middleware/callbacks.rb:44:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-1.2.**5/lib/rack/sendfile.rb:106:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/middleware/remote_ip.**rb:48:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/middleware/show_**exceptions.rb:47:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/railties-**
    3.0.14/lib/rails/rack/logger.**rb:13:in `call',/usr/local/rvm/gems/**
    ruby-1.9.2-p320/gems/rack-**fiber_pool-0.9.1/lib/rack/**fiber_pool.rb:19:in
    `block in call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-**
    fiber_pool-0.9.1/lib/fiber_**pool.rb:48:in `call',/usr/local/rvm/gems/**
    ruby-1.9.2-p320/gems/rack-**fiber_pool-0.9.1/lib/fiber_**pool.rb:48:in
    `block (3 levels) in initialize',/usr/local/rvm/**
    gems/ruby-1.9.2-p320/gems/**rack-fiber_pool-0.9.1/lib/**fiber_pool.rb:47:in
    `loop',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-**
    fiber_pool-0.9.1/lib/fiber_**pool.rb:47:in `block (2 levels) in
    initialize',/root/**cloudfoundry/cloud_controller/**
    cloud_controller/config/**initializers/fiber_connection_**pool.rb:11:in
    `call',/root/cloudfoundry/**cloud_controller/cloud_**controller/config/**
    initializers/fiber_connection_**pool.rb:11:in `block in initialize'>>
    cloud_controller --> [2012-08-08 10:29:50.241568] cc_events - pid=20622
    tid=af41 fid=bc71 INFO -- [2012-08-08 10:29:50 +0200, :USER, "
    tho...@bar.com <javascript:>", "POST:/services", "mysql", "mysql-ef04a",
    :FAILED, "Unexpected response from service gateway"]

    ------------------------------**------------------------------**
    ------------------------------**------------------------------**----

    ------------------------------**- MySQL Gateway
    ------------------------------**------------------------------**
    -----------


    mysql_gateway --> [2012-08-08 10:28:48.608498] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 ERROR -- Failed fetching handles, status=404
    mysql_gateway --> [2012-08-08 10:28:49.510232] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 INFO -- Fetching handles from cloud controller @
    http://api.vcap.me/services/**v1/offerings/mysql-5.1/handles<http://api.vcap.me/services/v1/offerings/mysql-5.1/handles>
    mysql_gateway --> [2012-08-08 10:28:49.618446] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 ERROR -- Failed fetching handles, status=404
    mysql_gateway --> [2012-08-08 10:28:50.446124] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 DEBUG -- [MyaaS-Provisioner] Received node announcement:
    {"available_capacity":47,"**capacity_unit":1,"id":"mysql_**
    node_1","plan":"free","**supported_versions":["5.1"]}
    mysql_gateway --> [2012-08-08 10:28:50.536661] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 INFO -- Fetching handles from cloud controller @
    http://api.vcap.me/services/**v1/offerings/mysql-5.1/handles<http://api.vcap.me/services/v1/offerings/mysql-5.1/handles>
    mysql_gateway --> [2012-08-08 10:28:50.655898] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 ERROR -- Failed fetching handles, status=404
    mysql_gateway --> [2012-08-08 10:28:51.557835] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 INFO -- Fetching handles from cloud controller @
    http://api.vcap.me/services/**v1/offerings/mysql-5.1/handles<http://api.vcap.me/services/v1/offerings/mysql-5.1/handles>

    ------------------------------**------------------------------**
    ------------------------------**---------------------------

    ----------------------------- Mysql_nodes ------------------------------*
    *------------------------------**----------

    70 fid=28b6 DEBUG -- MyaaS-Node: Sending announcement for everyone
    mysql_node --> [2012-08-08 11:01:54.029298] mysql_node_1 - pid=19783
    tid=2c 70 fid=28b6 DEBUG -- MyaaS-Node: Sending announcement for everyone
    mysql_node --> [2012-08-08 11:02:24.104521] mysql_node_1 - pid=19783
    tid=2c 70 fid=28b6 DEBUG -- MyaaS-Node: Sending announcement for everyone

    ------------------------------**------------------------------**
    ------------------------------**----------------------

    So the Cloud_controller receive some datas from the gateway, but don't
    know why he can't decode it. The nodes broadcast their capacity but I just
    saw this availability once in the gateway logs, during the request.

    Do you have any Idea to help me ? Thanks in advance!
  • Frank Lu at Aug 9, 2012 at 8:50 am
    hi,
    could you list the commit SHA1 of vcap, cloud_controller, services?
    thanks.
    On Thu, Aug 9, 2012 at 4:31 PM, bbqjack wrote:

    Sorry, but I didn't really understand what I have to do... :/

    Le jeudi 9 août 2012 10:26:33 UTC+2, Haipeng Wu a écrit :
    change list below may help you.
    https://github.com/**cloudfoundry/vcap/commit/**
    e867e10f3e2043801e81ff1fa43f7d**f265a03426<https://github.com/cloudfoundry/vcap/commit/e867e10f3e2043801e81ff1fa43f7df265a03426>

    haipeng
    On Thu, Aug 9, 2012 at 4:11 PM, bbqjack wrote:

    Hi !
    I need your help for a wierd problem...

    I tried a demo with multiple VM :

    VM A : Nats server + Cloud Controller + Health Manager + Gateways (Redis
    + Mysql + Mongodb)
    VM B : DEA + Router1
    VM C : DEA
    VM D : MySQL + Router2
    VM E : MySQL_node + Redis_node

    This strange configuration is just to try different things, but anyway
    the problem is not there.

    Everything worked well yesterday, but after some modifications with the
    A deployment config (I just tried to leave the mongodb gateway), and some
    changes on /etc/hosts of differents VM (because I noticed I had to target a
    router and not the cloud_controller to use vmc...).

    I restarted the different machines, and in then things became strange.

    I can't provision any service now, when I try it :

    vmc create-service mysql
    Creating Service [mysql-ef04a]: Error 503: Unexpected response from
    service gateway

    Here are different logs :

    -------------------------- Cloud Controller
    ------------------------------****-----------------


    user_email decoded from token is "tho...@bar.com"

    cloud_controller --> [2012-08-08 10:29:50.224037] cc - pid=20622
    tid=af41 fid=bc71 DEBUG -- Attempting to provision service:
    {"type":"database","tier":"**fre**e","vendor":"mysql","**version":**
    "5.1","name":"mysql-**ef04a"}
    cloud_controller --> [2012-08-08 10:29:50.235038] cc - pid=20622
    tid=af41 fid=bc71 ERROR -- Error talking to gateway: Can't decode gateway
    response. status code:0, response body:
    cloud_controller --> [2012-08-08 10:29:50.235216] cc exception pid=20622
    tid=af41 fid=bc71 ERROR -- VCAP::Services::Api::**ServiceGa**
    tewayClient::**UnexpectedRespons**e<<Can't decode gateway response.
    status code:0, response body::/usr/local/rvm/gems/**ruby**
    -1.9.2-p320/bundler/gems/**vcap-**common-1dca468ec017/lib/**servic**
    es/api/clients/service_**gateway**_client.rb:173:in `rescue in
    perform_request',/usr/local/**rv**m/gems/ruby-1.9.2-p320/**bundler**
    /gems/vcap-common-**1dca468ec017**/lib/services/api/**clients/**
    service_gateway_**client.rb:169:**in `perform_request',/usr/local/**r**
    vm/gems/ruby-1.9.2-p320/**bundle**r/gems/vcap-common-**1dca468ec01**
    7/lib/services/api/**clients/**service_gateway_**client.rb:77:**in
    `provision',/root/**cloudfoundry**/cloud_controller/**cloud_**
    controller/app/models/**service_**config.rb:64:in `block in
    provision',/usr/local/rvm/**gems**/ruby-1.9.2-p320/gems/**activere**
    cord-3.0.14/lib/**active_record/**connection_**adapters/abstract/**
    database_**statements.rb:139:in `transaction',/usr/local/rvm/**g**
    ems/ruby-1.9.2-p320/gems/**activ**erecord-3.0.14/lib/**active_**
    record/transactions.rb:**207:in `transaction',/root/**cloudfound**
    ry/cloud_controller/**cloud_**controller/app/models/**service_**config.rb:45:in
    `provision',/root/**cloudfoundry**/cloud_controller/**cloud_**
    controller/app/**controllers/**legacy_services_**controller.rb:**64:in
    `provision',/usr/local/rvm/**gem**s/ruby-1.9.2-p320/gems/**actionp**
    ack-3.0.14/lib/action_**controll**er/metal/implicit_**render.rb:4:**in
    `send_action',/usr/local/rvm/**g**ems/ruby-1.9.2-p320/gems/**actio**
    npack-3.0.14/lib/**abstract_**controller/base.rb:**150:in
    `process_action',/usr/local/**rv**m/gems/ruby-1.9.2-p320/gems/**ac**
    tionpack-3.0.14/lib/action_**con**troller/metal/rendering.rb:**11:**in
    `process_action',/usr/local/**rv**m/gems/ruby-1.9.2-p320/gems/**ac**
    tionpack-3.0.14/lib/**abstract_**controller/callbacks.**rb:18:in `block
    in process_action',/usr/local/**rvm**/gems/ruby-1.9.2-p320/gems/**act**
    ivesupport-3.0.14/lib/**active_**support/callbacks.rb:**441:in
    `_run__537527495993718278__**pro**cess_action__**28463894146223959**34__
    **callbacks',/usr/local/rvm/****gems/ruby-1.9.2-p320/gems/**acti**
    vesupport-3.0.14/lib/**active_**support/callbacks.rb:**410:in
    `_run_process_action_**callbacks**',/usr/local/rvm/**gems/ruby-1.**
    9.2-p320/gems/**activesupport-3.**0.14/lib/**active_support/**
    callbacks.rb:**94:in `run_callbacks',/usr/local/**rvm**
    /gems/ruby-1.9.2-p320/gems/**act**ionpack-3.0.14/lib/**abstract_**
    controller/callbacks.**rb:17:in `process_action',/usr/local/**rv**
    m/gems/ruby-1.9.2-p320/gems/**ac**tionpack-3.0.14/lib/action_**con**
    troller/metal/rescue.rb:17:**in `process_action',/usr/local/**rv**
    m/gems/ruby-1.9.2-p320/gems/**ac**tionpack-3.0.14/lib/action_**con**
    troller/metal/**instrumentation.**rb:30:in `block in
    process_action',/usr/local/**rvm**/gems/ruby-1.9.2-p320/gems/**act**
    ivesupport-3.0.14/lib/**active_**support/notifications.**rb:52:in
    `block in instrument',/usr/local/rvm/**gem**s/ruby-1.9.2-p320/gems/**
    actives**upport-3.0.14/lib/**active_**support/notifications/**instrume**nter.rb:21:in
    `instrument',/usr/local/rvm/**ge**ms/ruby-1.9.2-p320/gems/**active**
    support-3.0.14/lib/**active_**support/notifications.**rb:52:in
    `instrument',/usr/local/rvm/**ge**ms/ruby-1.9.2-p320/gems/**action**
    pack-3.0.14/lib/action_**control**ler/metal/**instrumentation.rb:**29:in
    `process_action',/root/**cloudfo**undry/cloud_controller/**cloud_**
    controller/app/**controllers/**application_**controller.rb:16:**in
    `process_action',/usr/local/**rv**m/gems/ruby-1.9.2-p320/gems/**ac**
    tionpack-3.0.14/lib/**abstract_**controller/base.rb:**119:in
    `process',/usr/local/rvm/gems/****ruby-1.9.2-p320/gems/**actionpac**
    k-3.0.14/lib/**abstract_**controller/rendering.**rb:41:in
    `process',/usr/local/rvm/gems/****ruby-1.9.2-p320/gems/**actionpac**
    k-3.0.14/lib/action_**controller**/metal.rb:138:in
    `dispatch',/usr/local/rvm/**gems**/ruby-1.9.2-p320/gems/**actionpa**
    ck-3.0.14/lib/action_**controlle**r/metal/rack_**delegation.rb:14:**in
    `dispatch',/usr/local/rvm/**gems**/ruby-1.9.2-p320/gems/**actionpa**
    ck-3.0.14/lib/action_**controlle**r/metal.rb:178:in `block in
    action',/usr/local/rvm/gems/**ru**by-1.9.2-p320/gems/**actionpack-**
    3.0.14/lib/action_**dispatch/**routing/route_set.rb:**68:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/**actionpack-**
    3.0.14/lib/action_**dispatch/**routing/route_set.rb:**68:in
    `dispatch',/usr/local/rvm/**gems**/ruby-1.9.2-p320/gems/**actionpa**
    ck-3.0.14/lib/action_**dispatch/**routing/route_set.rb:**33:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/rack-**mount-**
    0.6.14/lib/rack/mount/**route_**set.rb:148:in `block in
    call',/usr/local/rvm/gems/**ruby**-1.9.2-p320/gems/rack-**mount-0.**
    6.14/lib/rack/mount/**code_**generation.rb:93:in `block in
    recognize',/usr/local/rvm/**gems**/ruby-1.9.2-p320/gems/**rack-**
    mount-0.6.14/lib/rack/**mount/**code_generation.rb:68:in
    `optimized_each',/usr/local/**rv**m/gems/ruby-1.9.2-p320/gems/**ra**
    ck-mount-0.6.14/lib/rack/**mount**/code_generation.rb:92:in
    `recognize',/usr/local/rvm/**gem**s/ruby-1.9.2-p320/gems/**rack-**
    mount-0.6.14/lib/rack/**mount/**route_set.rb:139:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/**actionpack-**
    3.0.14/lib/action_**dispatch/**routing/route_set.rb:**499:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/**actionpack-**
    3.0.14/lib/action_**dispatch/**middleware/best_**standards_**support.rb:17:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/**actionpack-**
    3.0.14/lib/action_**dispatch/**middleware/head.rb:**14:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/rack-1.2.**5/**
    lib/rack/methodoverride.rb:**24:**in `call',/usr/local/rvm/gems/**rub**
    y-1.9.2-p320/gems/**actionpack-**3.0.14/lib/action_**dispatch/**
    middleware/params_**parser.rb:**21:in `call',/usr/local/rvm/gems/**rub**
    y-1.9.2-p320/gems/**actionpack-**3.0.14/lib/action_**dispatch/**
    middleware/cookies.**rb:302:in `call',/usr/local/rvm/gems/**rub**
    y-1.9.2-p320/gems/**activerecord**-3.0.14/lib/**active_record/**
    query_cache.rb:**32:in `block in call',/usr/local/rvm/gems/**ruby**
    -1.9.2-p320/gems/**activerecord-**3.0.14/lib/**active_record/**
    connection_**adapters/abstract/**query_cache.**rb:28:in
    `cache',/usr/local/rvm/gems/**ru**by-1.9.2-p320/gems/**activerecor**
    d-3.0.14/lib/**active_record/**query_cache.rb:**12:in
    `cache',/usr/local/rvm/gems/**ru**by-1.9.2-p320/gems/**activerecor**
    d-3.0.14/lib/**active_record/**query_cache.rb:**31:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/**activerecord**
    -3.0.14/lib/**active_record/**connection_**adapters/abstract/**
    connection_**pool.rb:354:in `call',/usr/local/rvm/gems/**rub**
    y-1.9.2-p320/gems/**actionpack-**3.0.14/lib/action_**dispatch/**
    middleware/callbacks.**rb:46:in `block in call',/usr/local/rvm/gems/**
    ruby**-1.9.2-p320/gems/**activesupport**-3.0.14/lib/**active_support/**
    callbacks.rb:**416:in `_run_call_callbacks',/usr/**loc**
    al/rvm/gems/ruby-1.9.2-**p320/**gems/actionpack-3.0.14/**lib/**
    action_dispatch/**middleware/**callbacks.rb:44:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/rack-1.2.**5/**lib/rack/sendfile.rb:106:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/**actionpack-**
    3.0.14/lib/action_**dispatch/**middleware/remote_ip.**rb:48:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/**actionpack-**
    3.0.14/lib/action_**dispatch/**middleware/show_**exceptions.rb:**47:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/railties-**3.**
    0.14/lib/rails/rack/logger.**rb:**13:in `call',/usr/local/rvm/gems/**rub
    **y-1.9.2-p320/gems/rack-**fiber_**pool-0.9.1/lib/rack/**fiber_**pool.rb:19:in
    `block in call',/usr/local/rvm/gems/**ruby**-1.9.2-p320/gems/rack-**
    fiber_**pool-0.9.1/lib/fiber_**pool.rb:**48:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/rack-**fiber_**
    pool-0.9.1/lib/fiber_**pool.rb:**48:in `block (3 levels) in
    initialize',/usr/local/rvm/**gem**s/ruby-1.9.2-p320/gems/**rack-**
    fiber_pool-0.9.1/lib/**fiber_**pool.rb:47:in `loop',/usr/local/rvm/gems/
    **rub**y-1.9.2-p320/gems/rack-**fiber_**pool-0.9.1/lib/fiber_**pool.rb:*
    *47:in `block (2 levels) in initialize',/root/**cloudfoundry**
    /cloud_controller/**cloud_**controller/config/**initializers**
    /fiber_connection_**pool.rb:11:**in `call',/root/cloudfoundry/**clou**
    d_controller/cloud_**controller/**config/**initializers/fiber_**
    connection_**pool.rb:11:in `block in initialize'>>
    cloud_controller --> [2012-08-08 10:29:50.241568] cc_events - pid=20622
    tid=af41 fid=bc71 INFO -- [2012-08-08 10:29:50 +0200, :USER, "
    tho...@bar.com", "POST:/services", "mysql", "mysql-ef04a", :FAILED,
    "Unexpected response from service gateway"]

    ------------------------------****------------------------------****
    ------------------------------****------------------------------****----

    ------------------------------****- MySQL Gateway
    ------------------------------****------------------------------****
    -----------


    mysql_gateway --> [2012-08-08 10:28:48.608498] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 ERROR -- Failed fetching handles, status=404
    mysql_gateway --> [2012-08-08 10:28:49.510232] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 INFO -- Fetching handles from cloud controller @
    http://api.vcap.me/services/****v1/offerings/mysql-5.1/handles<http://api.vcap.me/services/v1/offerings/mysql-5.1/handles>
    mysql_gateway --> [2012-08-08 10:28:49.618446] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 ERROR -- Failed fetching handles, status=404
    mysql_gateway --> [2012-08-08 10:28:50.446124] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 DEBUG -- [MyaaS-Provisioner] Received node announcement:
    {"available_capacity":47,"**capa**city_unit":1,"id":"mysql_**node_**
    1","plan":"free","**supported_**versions":["5.1"]}
    mysql_gateway --> [2012-08-08 10:28:50.536661] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 INFO -- Fetching handles from cloud controller @
    http://api.vcap.me/services/****v1/offerings/mysql-5.1/handles<http://api.vcap.me/services/v1/offerings/mysql-5.1/handles>
    mysql_gateway --> [2012-08-08 10:28:50.655898] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 ERROR -- Failed fetching handles, status=404
    mysql_gateway --> [2012-08-08 10:28:51.557835] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 INFO -- Fetching handles from cloud controller @
    http://api.vcap.me/services/****v1/offerings/mysql-5.1/handles<http://api.vcap.me/services/v1/offerings/mysql-5.1/handles>

    ------------------------------****------------------------------****
    ------------------------------****---------------------------

    ----------------------------- Mysql_nodes ------------------------------
    ****------------------------------****----------

    70 fid=28b6 DEBUG -- MyaaS-Node: Sending announcement for everyone
    mysql_node --> [2012-08-08 11:01:54.029298] mysql_node_1 - pid=19783
    tid=2c 70 fid=28b6 DEBUG -- MyaaS-Node: Sending announcement for everyone
    mysql_node --> [2012-08-08 11:02:24.104521] mysql_node_1 - pid=19783
    tid=2c 70 fid=28b6 DEBUG -- MyaaS-Node: Sending announcement for everyone

    ------------------------------****------------------------------****
    ------------------------------****----------------------

    So the Cloud_controller receive some datas from the gateway, but don't
    know why he can't decode it. The nodes broadcast their capacity but I just
    saw this availability once in the gateway logs, during the request.

    Do you have any Idea to help me ? Thanks in advance!

    --

    Frank(mingfan) Lu


    Ext. 8004

    Tel: 86-21-60349004

    VMware Cloudfoundry Service Team
    10F, KIC III, No. 333 Songhu Road, Yangpu District. Shanghai 200433
  • Haipeng Wu at Aug 9, 2012 at 8:50 am
    the change list is to bump services submodule, in order to keep
    synchronized with cloud_controller.
    make sure your vcap repo contain the change list.

    haipeng
    On Thu, Aug 9, 2012 at 4:31 PM, bbqjack wrote:

    Sorry, but I didn't really understand what I have to do... :/

    Le jeudi 9 août 2012 10:26:33 UTC+2, Haipeng Wu a écrit :
    change list below may help you.
    https://github.com/**cloudfoundry/vcap/commit/**
    e867e10f3e2043801e81ff1fa43f7d**f265a03426<https://github.com/cloudfoundry/vcap/commit/e867e10f3e2043801e81ff1fa43f7df265a03426>

    haipeng
    On Thu, Aug 9, 2012 at 4:11 PM, bbqjack wrote:

    Hi !
    I need your help for a wierd problem...

    I tried a demo with multiple VM :

    VM A : Nats server + Cloud Controller + Health Manager + Gateways (Redis
    + Mysql + Mongodb)
    VM B : DEA + Router1
    VM C : DEA
    VM D : MySQL + Router2
    VM E : MySQL_node + Redis_node

    This strange configuration is just to try different things, but anyway
    the problem is not there.

    Everything worked well yesterday, but after some modifications with the
    A deployment config (I just tried to leave the mongodb gateway), and some
    changes on /etc/hosts of differents VM (because I noticed I had to target a
    router and not the cloud_controller to use vmc...).

    I restarted the different machines, and in then things became strange.

    I can't provision any service now, when I try it :

    vmc create-service mysql
    Creating Service [mysql-ef04a]: Error 503: Unexpected response from
    service gateway

    Here are different logs :

    -------------------------- Cloud Controller
    ------------------------------****-----------------


    user_email decoded from token is "tho...@bar.com"

    cloud_controller --> [2012-08-08 10:29:50.224037] cc - pid=20622
    tid=af41 fid=bc71 DEBUG -- Attempting to provision service:
    {"type":"database","tier":"**fre**e","vendor":"mysql","**version":**
    "5.1","name":"mysql-**ef04a"}
    cloud_controller --> [2012-08-08 10:29:50.235038] cc - pid=20622
    tid=af41 fid=bc71 ERROR -- Error talking to gateway: Can't decode gateway
    response. status code:0, response body:
    cloud_controller --> [2012-08-08 10:29:50.235216] cc exception pid=20622
    tid=af41 fid=bc71 ERROR -- VCAP::Services::Api::**ServiceGa**
    tewayClient::**UnexpectedRespons**e<<Can't decode gateway response.
    status code:0, response body::/usr/local/rvm/gems/**ruby**
    -1.9.2-p320/bundler/gems/**vcap-**common-1dca468ec017/lib/**servic**
    es/api/clients/service_**gateway**_client.rb:173:in `rescue in
    perform_request',/usr/local/**rv**m/gems/ruby-1.9.2-p320/**bundler**
    /gems/vcap-common-**1dca468ec017**/lib/services/api/**clients/**
    service_gateway_**client.rb:169:**in `perform_request',/usr/local/**r**
    vm/gems/ruby-1.9.2-p320/**bundle**r/gems/vcap-common-**1dca468ec01**
    7/lib/services/api/**clients/**service_gateway_**client.rb:77:**in
    `provision',/root/**cloudfoundry**/cloud_controller/**cloud_**
    controller/app/models/**service_**config.rb:64:in `block in
    provision',/usr/local/rvm/**gems**/ruby-1.9.2-p320/gems/**activere**
    cord-3.0.14/lib/**active_record/**connection_**adapters/abstract/**
    database_**statements.rb:139:in `transaction',/usr/local/rvm/**g**
    ems/ruby-1.9.2-p320/gems/**activ**erecord-3.0.14/lib/**active_**
    record/transactions.rb:**207:in `transaction',/root/**cloudfound**
    ry/cloud_controller/**cloud_**controller/app/models/**service_**config.rb:45:in
    `provision',/root/**cloudfoundry**/cloud_controller/**cloud_**
    controller/app/**controllers/**legacy_services_**controller.rb:**64:in
    `provision',/usr/local/rvm/**gem**s/ruby-1.9.2-p320/gems/**actionp**
    ack-3.0.14/lib/action_**controll**er/metal/implicit_**render.rb:4:**in
    `send_action',/usr/local/rvm/**g**ems/ruby-1.9.2-p320/gems/**actio**
    npack-3.0.14/lib/**abstract_**controller/base.rb:**150:in
    `process_action',/usr/local/**rv**m/gems/ruby-1.9.2-p320/gems/**ac**
    tionpack-3.0.14/lib/action_**con**troller/metal/rendering.rb:**11:**in
    `process_action',/usr/local/**rv**m/gems/ruby-1.9.2-p320/gems/**ac**
    tionpack-3.0.14/lib/**abstract_**controller/callbacks.**rb:18:in `block
    in process_action',/usr/local/**rvm**/gems/ruby-1.9.2-p320/gems/**act**
    ivesupport-3.0.14/lib/**active_**support/callbacks.rb:**441:in
    `_run__537527495993718278__**pro**cess_action__**28463894146223959**34__
    **callbacks',/usr/local/rvm/****gems/ruby-1.9.2-p320/gems/**acti**
    vesupport-3.0.14/lib/**active_**support/callbacks.rb:**410:in
    `_run_process_action_**callbacks**',/usr/local/rvm/**gems/ruby-1.**
    9.2-p320/gems/**activesupport-3.**0.14/lib/**active_support/**
    callbacks.rb:**94:in `run_callbacks',/usr/local/**rvm**
    /gems/ruby-1.9.2-p320/gems/**act**ionpack-3.0.14/lib/**abstract_**
    controller/callbacks.**rb:17:in `process_action',/usr/local/**rv**
    m/gems/ruby-1.9.2-p320/gems/**ac**tionpack-3.0.14/lib/action_**con**
    troller/metal/rescue.rb:17:**in `process_action',/usr/local/**rv**
    m/gems/ruby-1.9.2-p320/gems/**ac**tionpack-3.0.14/lib/action_**con**
    troller/metal/**instrumentation.**rb:30:in `block in
    process_action',/usr/local/**rvm**/gems/ruby-1.9.2-p320/gems/**act**
    ivesupport-3.0.14/lib/**active_**support/notifications.**rb:52:in
    `block in instrument',/usr/local/rvm/**gem**s/ruby-1.9.2-p320/gems/**
    actives**upport-3.0.14/lib/**active_**support/notifications/**instrume**nter.rb:21:in
    `instrument',/usr/local/rvm/**ge**ms/ruby-1.9.2-p320/gems/**active**
    support-3.0.14/lib/**active_**support/notifications.**rb:52:in
    `instrument',/usr/local/rvm/**ge**ms/ruby-1.9.2-p320/gems/**action**
    pack-3.0.14/lib/action_**control**ler/metal/**instrumentation.rb:**29:in
    `process_action',/root/**cloudfo**undry/cloud_controller/**cloud_**
    controller/app/**controllers/**application_**controller.rb:16:**in
    `process_action',/usr/local/**rv**m/gems/ruby-1.9.2-p320/gems/**ac**
    tionpack-3.0.14/lib/**abstract_**controller/base.rb:**119:in
    `process',/usr/local/rvm/gems/****ruby-1.9.2-p320/gems/**actionpac**
    k-3.0.14/lib/**abstract_**controller/rendering.**rb:41:in
    `process',/usr/local/rvm/gems/****ruby-1.9.2-p320/gems/**actionpac**
    k-3.0.14/lib/action_**controller**/metal.rb:138:in
    `dispatch',/usr/local/rvm/**gems**/ruby-1.9.2-p320/gems/**actionpa**
    ck-3.0.14/lib/action_**controlle**r/metal/rack_**delegation.rb:14:**in
    `dispatch',/usr/local/rvm/**gems**/ruby-1.9.2-p320/gems/**actionpa**
    ck-3.0.14/lib/action_**controlle**r/metal.rb:178:in `block in
    action',/usr/local/rvm/gems/**ru**by-1.9.2-p320/gems/**actionpack-**
    3.0.14/lib/action_**dispatch/**routing/route_set.rb:**68:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/**actionpack-**
    3.0.14/lib/action_**dispatch/**routing/route_set.rb:**68:in
    `dispatch',/usr/local/rvm/**gems**/ruby-1.9.2-p320/gems/**actionpa**
    ck-3.0.14/lib/action_**dispatch/**routing/route_set.rb:**33:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/rack-**mount-**
    0.6.14/lib/rack/mount/**route_**set.rb:148:in `block in
    call',/usr/local/rvm/gems/**ruby**-1.9.2-p320/gems/rack-**mount-0.**
    6.14/lib/rack/mount/**code_**generation.rb:93:in `block in
    recognize',/usr/local/rvm/**gems**/ruby-1.9.2-p320/gems/**rack-**
    mount-0.6.14/lib/rack/**mount/**code_generation.rb:68:in
    `optimized_each',/usr/local/**rv**m/gems/ruby-1.9.2-p320/gems/**ra**
    ck-mount-0.6.14/lib/rack/**mount**/code_generation.rb:92:in
    `recognize',/usr/local/rvm/**gem**s/ruby-1.9.2-p320/gems/**rack-**
    mount-0.6.14/lib/rack/**mount/**route_set.rb:139:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/**actionpack-**
    3.0.14/lib/action_**dispatch/**routing/route_set.rb:**499:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/**actionpack-**
    3.0.14/lib/action_**dispatch/**middleware/best_**standards_**support.rb:17:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/**actionpack-**
    3.0.14/lib/action_**dispatch/**middleware/head.rb:**14:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/rack-1.2.**5/**
    lib/rack/methodoverride.rb:**24:**in `call',/usr/local/rvm/gems/**rub**
    y-1.9.2-p320/gems/**actionpack-**3.0.14/lib/action_**dispatch/**
    middleware/params_**parser.rb:**21:in `call',/usr/local/rvm/gems/**rub**
    y-1.9.2-p320/gems/**actionpack-**3.0.14/lib/action_**dispatch/**
    middleware/cookies.**rb:302:in `call',/usr/local/rvm/gems/**rub**
    y-1.9.2-p320/gems/**activerecord**-3.0.14/lib/**active_record/**
    query_cache.rb:**32:in `block in call',/usr/local/rvm/gems/**ruby**
    -1.9.2-p320/gems/**activerecord-**3.0.14/lib/**active_record/**
    connection_**adapters/abstract/**query_cache.**rb:28:in
    `cache',/usr/local/rvm/gems/**ru**by-1.9.2-p320/gems/**activerecor**
    d-3.0.14/lib/**active_record/**query_cache.rb:**12:in
    `cache',/usr/local/rvm/gems/**ru**by-1.9.2-p320/gems/**activerecor**
    d-3.0.14/lib/**active_record/**query_cache.rb:**31:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/**activerecord**
    -3.0.14/lib/**active_record/**connection_**adapters/abstract/**
    connection_**pool.rb:354:in `call',/usr/local/rvm/gems/**rub**
    y-1.9.2-p320/gems/**actionpack-**3.0.14/lib/action_**dispatch/**
    middleware/callbacks.**rb:46:in `block in call',/usr/local/rvm/gems/**
    ruby**-1.9.2-p320/gems/**activesupport**-3.0.14/lib/**active_support/**
    callbacks.rb:**416:in `_run_call_callbacks',/usr/**loc**
    al/rvm/gems/ruby-1.9.2-**p320/**gems/actionpack-3.0.14/**lib/**
    action_dispatch/**middleware/**callbacks.rb:44:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/rack-1.2.**5/**lib/rack/sendfile.rb:106:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/**actionpack-**
    3.0.14/lib/action_**dispatch/**middleware/remote_ip.**rb:48:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/**actionpack-**
    3.0.14/lib/action_**dispatch/**middleware/show_**exceptions.rb:**47:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/railties-**3.**
    0.14/lib/rails/rack/logger.**rb:**13:in `call',/usr/local/rvm/gems/**rub
    **y-1.9.2-p320/gems/rack-**fiber_**pool-0.9.1/lib/rack/**fiber_**pool.rb:19:in
    `block in call',/usr/local/rvm/gems/**ruby**-1.9.2-p320/gems/rack-**
    fiber_**pool-0.9.1/lib/fiber_**pool.rb:**48:in
    `call',/usr/local/rvm/gems/**rub**y-1.9.2-p320/gems/rack-**fiber_**
    pool-0.9.1/lib/fiber_**pool.rb:**48:in `block (3 levels) in
    initialize',/usr/local/rvm/**gem**s/ruby-1.9.2-p320/gems/**rack-**
    fiber_pool-0.9.1/lib/**fiber_**pool.rb:47:in `loop',/usr/local/rvm/gems/
    **rub**y-1.9.2-p320/gems/rack-**fiber_**pool-0.9.1/lib/fiber_**pool.rb:*
    *47:in `block (2 levels) in initialize',/root/**cloudfoundry**
    /cloud_controller/**cloud_**controller/config/**initializers**
    /fiber_connection_**pool.rb:11:**in `call',/root/cloudfoundry/**clou**
    d_controller/cloud_**controller/**config/**initializers/fiber_**
    connection_**pool.rb:11:in `block in initialize'>>
    cloud_controller --> [2012-08-08 10:29:50.241568] cc_events - pid=20622
    tid=af41 fid=bc71 INFO -- [2012-08-08 10:29:50 +0200, :USER, "
    tho...@bar.com", "POST:/services", "mysql", "mysql-ef04a", :FAILED,
    "Unexpected response from service gateway"]

    ------------------------------****------------------------------****
    ------------------------------****------------------------------****----

    ------------------------------****- MySQL Gateway
    ------------------------------****------------------------------****
    -----------


    mysql_gateway --> [2012-08-08 10:28:48.608498] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 ERROR -- Failed fetching handles, status=404
    mysql_gateway --> [2012-08-08 10:28:49.510232] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 INFO -- Fetching handles from cloud controller @
    http://api.vcap.me/services/****v1/offerings/mysql-5.1/handles<http://api.vcap.me/services/v1/offerings/mysql-5.1/handles>
    mysql_gateway --> [2012-08-08 10:28:49.618446] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 ERROR -- Failed fetching handles, status=404
    mysql_gateway --> [2012-08-08 10:28:50.446124] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 DEBUG -- [MyaaS-Provisioner] Received node announcement:
    {"available_capacity":47,"**capa**city_unit":1,"id":"mysql_**node_**
    1","plan":"free","**supported_**versions":["5.1"]}
    mysql_gateway --> [2012-08-08 10:28:50.536661] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 INFO -- Fetching handles from cloud controller @
    http://api.vcap.me/services/****v1/offerings/mysql-5.1/handles<http://api.vcap.me/services/v1/offerings/mysql-5.1/handles>
    mysql_gateway --> [2012-08-08 10:28:50.655898] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 ERROR -- Failed fetching handles, status=404
    mysql_gateway --> [2012-08-08 10:28:51.557835] mysql_gateway - pid=21286
    tid=4d90 fid=5eb3 INFO -- Fetching handles from cloud controller @
    http://api.vcap.me/services/****v1/offerings/mysql-5.1/handles<http://api.vcap.me/services/v1/offerings/mysql-5.1/handles>

    ------------------------------****------------------------------****
    ------------------------------****---------------------------

    ----------------------------- Mysql_nodes ------------------------------
    ****------------------------------****----------

    70 fid=28b6 DEBUG -- MyaaS-Node: Sending announcement for everyone
    mysql_node --> [2012-08-08 11:01:54.029298] mysql_node_1 - pid=19783
    tid=2c 70 fid=28b6 DEBUG -- MyaaS-Node: Sending announcement for everyone
    mysql_node --> [2012-08-08 11:02:24.104521] mysql_node_1 - pid=19783
    tid=2c 70 fid=28b6 DEBUG -- MyaaS-Node: Sending announcement for everyone

    ------------------------------****------------------------------****
    ------------------------------****----------------------

    So the Cloud_controller receive some datas from the gateway, but don't
    know why he can't decode it. The nodes broadcast their capacity but I just
    saw this availability once in the gateway logs, during the request.

    Do you have any Idea to help me ? Thanks in advance!
  • Bbqjack at Aug 9, 2012 at 9:07 am
    @Frank Lu : Where could I find those informations?

    @ Haipeng Wu : I checked in .../vcap/services, I didn't find any bump or
    bump services... So I'm confused, I'm totally lost.

    Thanks a lot for your patience guys !
  • Frank Lu at Aug 9, 2012 at 9:11 am
    For example, my vcap is in ~/cloudfoundry/vcap

    cd ~/cloudfoundry/vcap
    git log -1
    # then you will get the commit id.
    # such as: commit cb45bd75564321d9ad4c91fb66fd4427b5282762

    cd services
    git log -1

    cd ../../cloud_controller
    git log -1
    On Thu, Aug 9, 2012 at 5:07 PM, bbqjack wrote:

    @Frank Lu : Where could I find those informations?

    @ Haipeng Wu : I checked in .../vcap/services, I didn't find any bump or
    bump services... So I'm confused, I'm totally lost.

    Thanks a lot for your patience guys !


    --

    Frank(mingfan) Lu


    Ext. 8004

    Tel: 86-21-60349004

    VMware Cloudfoundry Service Team
    10F, KIC III, No. 333 Songhu Road, Yangpu District. Shanghai 200433
  • Bbqjack at Aug 9, 2012 at 9:18 am
    *root@cloudFoundry:~/cloudfoundry/vcap# git log -1*
    log -1
    commit 56edacab37407c40c1709884a593a57867604ae2
    Merge: 57183f6 64752d4
    Author: Figo Feng <ffeng@vmware.com>
    Date: Mon Jul 23 07:55:34 2012 +0000

    Merge "remove ip_route in vblob_node.yml"

    *root@cloudFoundry:~/cloudfoundry/vcap# cd services*
    *root@cloudFoundry:~/cloudfoundry/vcap/services# git log -1*

    log -1
    commit 7f530df679252f85d56f4c98063854099f93fc5f
    Author: Tang Rui <rtang@vmware.com>
    Date: Sun Jul 15 21:30:54 2012 -0600

    Fix the bug that the instance data can be deleted when start timeout

    Change-Id: I9e3f0bab81cffe3683956fb459807377face18f2

    *root@cloudFoundry:~/cloudfoundry/vcap/services# cd ../../cloud_controller/
    *
    *root@cloudFoundry:~/cloudfoundry/cloud_controller# git log -1*

    log -1
    commit 1a5a587c564f9e49c902169c7a20bdebf46a7c47
    Merge: 3c92a61 9c2b92e
    Author: Dale Olds <olds@rbcon.com>
    Date: Mon Aug 6 21:50:13 2012 +0000

    Merge "CFID-364: change default UAA scopes for cloud_controller to nil"

    Could this help you?
  • Frank Lu at Aug 9, 2012 at 9:24 am
    yes. you cloud_controller is new and including latest changes, while your
    vcap& services are out-of-date.
    could you pull the lastet changes in vcap and update submodules.

    if you need service_lifecycle feature, you need cherry-pick this change
    http://reviews.cloudfoundry.org/#/c/8175/ on the top of your updated vcap.
    On Thu, Aug 9, 2012 at 5:18 PM, bbqjack wrote:

    *root@cloudFoundry:~/cloudfoundry/vcap# git log -1*
    log -1
    commit 56edacab37407c40c1709884a593a57867604ae2
    Merge: 57183f6 64752d4
    Author: Figo Feng <ffeng@vmware.com>
    Date: Mon Jul 23 07:55:34 2012 +0000

    Merge "remove ip_route in vblob_node.yml"

    *root@cloudFoundry:~/cloudfoundry/vcap# cd services*
    *root@cloudFoundry:~/cloudfoundry/vcap/services# git log -1*

    log -1
    commit 7f530df679252f85d56f4c98063854099f93fc5f
    Author: Tang Rui <rtang@vmware.com>
    Date: Sun Jul 15 21:30:54 2012 -0600

    Fix the bug that the instance data can be deleted when start timeout

    Change-Id: I9e3f0bab81cffe3683956fb459807377face18f2

    *root@cloudFoundry:~/cloudfoundry/vcap/services# cd
    ../../cloud_controller/
    *
    *root@cloudFoundry:~/cloudfoundry/cloud_controller# git log -1*

    log -1
    commit 1a5a587c564f9e49c902169c7a20bdebf46a7c47
    Merge: 3c92a61 9c2b92e
    Author: Dale Olds <olds@rbcon.com>
    Date: Mon Aug 6 21:50:13 2012 +0000

    Merge "CFID-364: change default UAA scopes for cloud_controller to nil"

    Could this help you?

    --

    Frank(mingfan) Lu


    Ext. 8004

    Tel: 86-21-60349004

    VMware Cloudfoundry Service Team
    10F, KIC III, No. 333 Songhu Road, Yangpu District. Shanghai 200433
  • Bbqjack at Aug 9, 2012 at 10:02 am
    Oh Thanks ! I didn't know that (I'm not used to Git, it's a really new tool
    for me !)
    The problem is nearly solved.

    Now the mysql gateway receive correctly all messages from nodes BUT the
    cloud controller can't decode it. logs says me that something is missing in
    configuration but I can't find it...

    ---------------------------------
    cloud_controller --> [2012-08-09 11:44:02.269582] cc - pid=31121 tid=6d7f
    fid=bbfe DEBUG -- Attempting to provision service:
    {"type":"database","tier":"free","vendor":"mysql","version":"5.1","name":"mysql-a0399"}
    cloud_controller --> [2012-08-09 11:44:02.299817] cc - pid=31121 tid=6d7f
    fid=bbfe ERROR -- Error talking to gateway: Field: configuration, Error:
    Missing field configuration
    cloud_controller --> [2012-08-09 11:44:02.299976] cc exception pid=31121
    tid=6d7f fid=bbfe ERROR -- JsonMessage::ValidationError<<Field:
    configuration, Error: Missing field
    configuration:/usr/local/rvm/gems/ruby-1.9.2-p320/bundler/gems/vcap-common-1dca468ec017/lib/json_message.rb:102:in
    `from_decoded_json',/usr/local/rvm/gems/ruby-1.9.2-p320/bundler/gems/vcap-common-1dca468ec017/lib/json_message.rb:74:in
    `decode',/usr/local/rvm/gems/ruby-1.9.2-p320/bundler/gems/vcap-common-1dca468ec017/lib/services/api/clients/service_gateway_client.rb:78:in
    `provision',/root/cloudfoundry/cloud_controller/cloud_controller/app/models/service_config.rb:64:in
    `block in
    provision',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/connection_adapters/abstract/database_statements.rb:139:in
    `transaction',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/transactions.rb:207:in
    `transaction',/root/cloudfoundry/cloud_controller/cloud_controller/app/models/service_config.rb:45:in
    `provision',/root/cloudfoundry/cloud_controller/cloud_controller/app/controllers/legacy_services_controller.rb:64:in
    `provision',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/implicit_render.rb:4:in
    `send_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/base.rb:150:in
    `process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/rendering.rb:11:in
    `process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/callbacks.rb:18:in
    `block in
    process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/callbacks.rb:441:in
    `_run__3273360498142871003__process_action__4248041742907623254__callbacks',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/callbacks.rb:410:in
    `_run_process_action_callbacks',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/callbacks.rb:94:in
    `run_callbacks',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/callbacks.rb:17:in
    `process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/rescue.rb:17:in
    `process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/instrumentation.rb:30:in
    `block in
    process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/notifications.rb:52:in
    `block in
    instrument',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/notifications/instrumenter.rb:21:in
    `instrument',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/notifications.rb:52:in
    `instrument',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/instrumentation.rb:29:in
    `process_action',/root/cloudfoundry/cloud_controller/cloud_controller/app/controllers/application_controller.rb:16:in
    `process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/base.rb:119:in
    `process',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/rendering.rb:41:in
    `process',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal.rb:138:in
    `dispatch',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/rack_delegation.rb:14:in
    `dispatch',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal.rb:178:in
    `block in
    action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/routing/route_set.rb:68:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/routing/route_set.rb:68:in
    `dispatch',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/routing/route_set.rb:33:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/route_set.rb:148:in
    `block in
    call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/code_generation.rb:93:in
    `block in
    recognize',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/code_generation.rb:68:in
    `optimized_each',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/code_generation.rb:92:in
    `recognize',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/route_set.rb:139:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/routing/route_set.rb:499:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/best_standards_support.rb:17:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/head.rb:14:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-1.2.5/lib/rack/methodoverride.rb:24:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/params_parser.rb:21:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/cookies.rb:302:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/query_cache.rb:32:in
    `block in
    call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/connection_adapters/abstract/query_cache.rb:28:in
    `cache',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/query_cache.rb:12:in
    `cache',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/query_cache.rb:31:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/connection_adapters/abstract/connection_pool.rb:354:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/callbacks.rb:46:in
    `block in
    call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/callbacks.rb:416:in
    `_run_call_callbacks',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/callbacks.rb:44:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-1.2.5/lib/rack/sendfile.rb:106:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/remote_ip.rb:48:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/show_exceptions.rb:47:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/railties-3.0.14/lib/rails/rack/logger.rb:13:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/rack/fiber_pool.rb:19:in
    `block in
    call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/fiber_pool.rb:48:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/fiber_pool.rb:48:in
    `block (3 levels) in
    initialize',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/fiber_pool.rb:47:in
    `loop',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/fiber_pool.rb:47:in
    `block (2 levels) in
    initialize',/root/cloudfoundry/cloud_controller/cloud_controller/config/initializers/fiber_connection_pool.rb:11:in
    `call',/root/cloudfoundry/cloud_controller/cloud_controller/config/initializers/fiber_connection_pool.rb:11:in
    `block in initialize'>>
    cloud_controller --> [2012-08-09 11:44:02.305925] cc_events - pid=31121
    tid=6d7f fid=bbfe INFO -- [2012-08-09 11:44:02 +0200, :USER,
    "thomas@bar.com", "POST:/services", "mysql", "mysql-a0399", :FAILED,
    "Unexpected response from service gateway"]
    -----------------------

    I give you the cloud_controller configuration file, and the mysql_gateway
    one.

    I changed the real cloud_controller IP by 1.2.3.4.5



    Le jeudi 9 août 2012 11:23:52 UTC+2, Frank Lu a écrit :
    yes. you cloud_controller is new and including latest changes, while your
    vcap& services are out-of-date.
    could you pull the lastet changes in vcap and update submodules.

    if you need service_lifecycle feature, you need cherry-pick this change
    http://reviews.cloudfoundry.org/#/c/8175/ on the top of your updated vcap.

    On Thu, Aug 9, 2012 at 5:18 PM, bbqjack <barbecu...@gmail.com<javascript:>
    wrote:
    *root@cloudFoundry:~/cloudfoundry/vcap# git log -1*
    log -1
    commit 56edacab37407c40c1709884a593a57867604ae2
    Merge: 57183f6 64752d4
    Author: Figo Feng <ff...@vmware.com <javascript:>>
    Date: Mon Jul 23 07:55:34 2012 +0000

    Merge "remove ip_route in vblob_node.yml"

    *root@cloudFoundry:~/cloudfoundry/vcap# cd services*
    *root@cloudFoundry:~/cloudfoundry/vcap/services# git log -1*

    log -1
    commit 7f530df679252f85d56f4c98063854099f93fc5f
    Author: Tang Rui <rt...@vmware.com <javascript:>>
    Date: Sun Jul 15 21:30:54 2012 -0600

    Fix the bug that the instance data can be deleted when start timeout

    Change-Id: I9e3f0bab81cffe3683956fb459807377face18f2

    *root@cloudFoundry:~/cloudfoundry/vcap/services# cd
    ../../cloud_controller/
    *
    *root@cloudFoundry:~/cloudfoundry/cloud_controller# git log -1*

    log -1
    commit 1a5a587c564f9e49c902169c7a20bdebf46a7c47
    Merge: 3c92a61 9c2b92e
    Author: Dale Olds <ol...@rbcon.com <javascript:>>
    Date: Mon Aug 6 21:50:13 2012 +0000

    Merge "CFID-364: change default UAA scopes for cloud_controller to
    nil"

    Could this help you?

    --

    Frank(mingfan) Lu


    Ext. 8004

    Tel: 86-21-60349004

    VMware Cloudfoundry Service Team
    10F, KIC III, No. 333 Songhu Road, Yangpu District. Shanghai 200433

  • Frank Lu at Aug 9, 2012 at 1:47 pm
    from your log
    1) you cc use the latest vcap_common (1dca468ec017)
    2) but your provision response's format is out-of-date.
    this change has modified the provision response format:
    https://github.com/cloudfoundry/vcap-common/commit/b0cc19d5520c9849600dbd8cf2883bff66d7643e#L1R87)
    .
    I am afraid, your vcap/services are still not matched with your cc.

    On Thu, Aug 9, 2012 at 5:55 PM, bbqjack wrote:

    Oh Thanks ! I didn't know that (I'm not used to Git, it's a really new
    tool for me !)
    The problem is nearly solved.

    Now the mysql gateway receive correctly all messages from nodes BUT the
    cloud controller can't decode it. logs says me that something is missing in
    configuration but I can't find it...

    ---------------------------------
    cloud_controller --> [2012-08-09 11:44:02.269582] cc - pid=31121 tid=6d7f
    fid=bbfe DEBUG -- Attempting to provision service:
    {"type":"database","tier":"free","vendor":"mysql","version":"5.1","name":"mysql-a0399"}
    cloud_controller --> [2012-08-09 11:44:02.299817] cc - pid=31121 tid=6d7f
    fid=bbfe ERROR -- Error talking to gateway: Field: configuration, Error:
    Missing field configuration
    cloud_controller --> [2012-08-09 11:44:02.299976] cc exception pid=31121
    tid=6d7f fid=bbfe ERROR -- JsonMessage::ValidationError<<Field:
    configuration, Error: Missing field
    configuration:/usr/local/rvm/gems/ruby-1.9.2-p320/bundler/gems/vcap-common-1dca468ec017/lib/json_message.rb:102:in
    `from_decoded_json',/usr/local/rvm/gems/ruby-1.9.2-p320/bundler/gems/vcap-common-1dca468ec017/lib/json_message.rb:74:in
    `decode',/usr/local/rvm/gems/ruby-1.9.2-p320/bundler/gems/vcap-common-1dca468ec017/lib/services/api/clients/service_gateway_client.rb:78:in
    `provision',/root/cloudfoundry/cloud_controller/cloud_controller/app/models/service_config.rb:64:in
    `block in
    provision',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/connection_adapters/abstract/database_statements.rb:139:in
    `transaction',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/transactions.rb:207:in
    `transaction',/root/cloudfoundry/cloud_controller/cloud_controller/app/models/service_config.rb:45:in
    `provision',/root/cloudfoundry/cloud_controller/cloud_controller/app/controllers/legacy_services_controller.rb:64:in
    `provision',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/implicit_render.rb:4:in
    `send_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/base.rb:150:in
    `process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/rendering.rb:11:in
    `process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/callbacks.rb:18:in
    `block in
    process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/callbacks.rb:441:in
    `_run__3273360498142871003__process_action__4248041742907623254__callbacks',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/callbacks.rb:410:in
    `_run_process_action_callbacks',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/callbacks.rb:94:in
    `run_callbacks',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/callbacks.rb:17:in
    `process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/rescue.rb:17:in
    `process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/instrumentation.rb:30:in
    `block in
    process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/notifications.rb:52:in
    `block in
    instrument',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/notifications/instrumenter.rb:21:in
    `instrument',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/notifications.rb:52:in
    `instrument',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/instrumentation.rb:29:in
    `process_action',/root/cloudfoundry/cloud_controller/cloud_controller/app/controllers/application_controller.rb:16:in
    `process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/base.rb:119:in
    `process',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/rendering.rb:41:in
    `process',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal.rb:138:in
    `dispatch',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/rack_delegation.rb:14:in
    `dispatch',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal.rb:178:in
    `block in
    action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/routing/route_set.rb:68:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/routing/route_set.rb:68:in
    `dispatch',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/routing/route_set.rb:33:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/route_set.rb:148:in
    `block in
    call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/code_generation.rb:93:in
    `block in
    recognize',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/code_generation.rb:68:in
    `optimized_each',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/code_generation.rb:92:in
    `recognize',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/route_set.rb:139:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/routing/route_set.rb:499:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/best_standards_support.rb:17:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/head.rb:14:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-1.2.5/lib/rack/methodoverride.rb:24:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/params_parser.rb:21:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/cookies.rb:302:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/query_cache.rb:32:in
    `block in
    call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/connection_adapters/abstract/query_cache.rb:28:in
    `cache',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/query_cache.rb:12:in
    `cache',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/query_cache.rb:31:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/connection_adapters/abstract/connection_pool.rb:354:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/callbacks.rb:46:in
    `block in
    call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/callbacks.rb:416:in
    `_run_call_callbacks',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/callbacks.rb:44:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-1.2.5/lib/rack/sendfile.rb:106:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/remote_ip.rb:48:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/show_exceptions.rb:47:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/railties-3.0.14/lib/rails/rack/logger.rb:13:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/rack/fiber_pool.rb:19:in
    `block in
    call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/fiber_pool.rb:48:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/fiber_pool.rb:48:in
    `block (3 levels) in
    initialize',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/fiber_pool.rb:47:in
    `loop',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/fiber_pool.rb:47:in
    `block (2 levels) in
    initialize',/root/cloudfoundry/cloud_controller/cloud_controller/config/initializers/fiber_connection_pool.rb:11:in
    `call',/root/cloudfoundry/cloud_controller/cloud_controller/config/initializers/fiber_connection_pool.rb:11:in
    `block in initialize'>>
    cloud_controller --> [2012-08-09 11:44:02.305925] cc_events - pid=31121
    tid=6d7f fid=bbfe INFO -- [2012-08-09 11:44:02 +0200, :USER, "
    thomas@bar.com", "POST:/services", "mysql", "mysql-a0399", :FAILED,
    "Unexpected response from service gateway"]
    -----------------------

    I give you the cloud_controller configuration file, and the mysql_gateway
    one.

    I changed the real cloud_controller IP by 1.2.3.4.5



    Le jeudi 9 août 2012 11:23:52 UTC+2, Frank Lu a écrit :
    yes. you cloud_controller is new and including latest changes, while your
    vcap& services are out-of-date.
    could you pull the lastet changes in vcap and update submodules.

    if you need service_lifecycle feature, you need cherry-pick this change
    http://reviews.**cloudfoundry.org/#/c/8175/<http://reviews.cloudfoundry.org/#/c/8175/>on the top of your updated vcap.
    On Thu, Aug 9, 2012 at 5:18 PM, bbqjack wrote:

    *root@cloudFoundry:~/cloudfoundry/vcap# git log -1*
    log -1
    commit 56edacab37407c40c1709884a593a5**7867604ae2
    Merge: 57183f6 64752d4
    Author: Figo Feng <ff...@vmware.com>
    Date: Mon Jul 23 07:55:34 2012 +0000

    Merge "remove ip_route in vblob_node.yml"

    *root@cloudFoundry:~/cloudfoundry/vcap# cd services*
    *root@cloudFoundry:~/cloudfoundry/vcap/services# git log -1*

    log -1
    commit 7f530df679252f85d56f4c98063854**099f93fc5f
    Author: Tang Rui <rt...@vmware.com>
    Date: Sun Jul 15 21:30:54 2012 -0600

    Fix the bug that the instance data can be deleted when start timeout

    Change-Id: I9e3f0bab81cffe3683956fb459807**377face18f2

    *root@cloudFoundry:~/cloudfoundry/vcap/services# cd
    ../../cloud_controller/
    *
    *root@cloudFoundry:~/cloudfoundry/cloud_controller# git log -1*

    log -1
    commit 1a5a587c564f9e49c902169c7a20bd**ebf46a7c47
    Merge: 3c92a61 9c2b92e
    Author: Dale Olds <ol...@rbcon.com>
    Date: Mon Aug 6 21:50:13 2012 +0000

    Merge "CFID-364: change default UAA scopes for cloud_controller to
    nil"

    Could this help you?

    --

    Frank(mingfan) Lu


    Ext. 8004

    Tel: 86-21-60349004

    VMware Cloudfoundry Service Team
    10F, KIC III, No. 333 Songhu Road, Yangpu District. Shanghai 200433


    --

    Frank(mingfan) Lu


    Ext. 8004

    Tel: 86-21-60349004

    VMware Cloudfoundry Service Team
    10F, KIC III, No. 333 Songhu Road, Yangpu District. Shanghai 200433
  • Bbqjack at Aug 9, 2012 at 2:43 pm
    I did it in the vcap folder, but in
    ~/cloudfoundry/cloud_controller/cloud_controller I get this message

    ~/cloudfoundry/cloud_controller/cloud_controller# git submodule
    submodule
    You need to run this command from the toplevel of the working tree.


    Le jeudi 9 août 2012 15:39:25 UTC+2, Frank Lu a écrit :
    from your log
    1) you cc use the latest vcap_common (1dca468ec017)
    2) but your provision response's format is out-of-date.
    this change has modified the provision response format:
    https://github.com/cloudfoundry/vcap-common/commit/b0cc19d5520c9849600dbd8cf2883bff66d7643e#L1R87)
    .
    I am afraid, your vcap/services are still not matched with your
    cc.

    On Thu, Aug 9, 2012 at 5:55 PM, bbqjack <barbecu...@gmail.com<javascript:>
    wrote:
    Oh Thanks ! I didn't know that (I'm not used to Git, it's a really new
    tool for me !)
    The problem is nearly solved.

    Now the mysql gateway receive correctly all messages from nodes BUT the
    cloud controller can't decode it. logs says me that something is missing in
    configuration but I can't find it...

    ---------------------------------
    cloud_controller --> [2012-08-09 11:44:02.269582] cc - pid=31121 tid=6d7f
    fid=bbfe DEBUG -- Attempting to provision service:
    {"type":"database","tier":"free","vendor":"mysql","version":"5.1","name":"mysql-a0399"}
    cloud_controller --> [2012-08-09 11:44:02.299817] cc - pid=31121 tid=6d7f
    fid=bbfe ERROR -- Error talking to gateway: Field: configuration, Error:
    Missing field configuration
    cloud_controller --> [2012-08-09 11:44:02.299976] cc exception pid=31121
    tid=6d7f fid=bbfe ERROR -- JsonMessage::ValidationError<<Field:
    configuration, Error: Missing field
    configuration:/usr/local/rvm/gems/ruby-1.9.2-p320/bundler/gems/vcap-common-1dca468ec017/lib/json_message.rb:102:in
    `from_decoded_json',/usr/local/rvm/gems/ruby-1.9.2-p320/bundler/gems/vcap-common-1dca468ec017/lib/json_message.rb:74:in
    `decode',/usr/local/rvm/gems/ruby-1.9.2-p320/bundler/gems/vcap-common-1dca468ec017/lib/services/api/clients/service_gateway_client.rb:78:in
    `provision',/root/cloudfoundry/cloud_controller/cloud_controller/app/models/service_config.rb:64:in
    `block in
    provision',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/connection_adapters/abstract/database_statements.rb:139:in
    `transaction',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/transactions.rb:207:in
    `transaction',/root/cloudfoundry/cloud_controller/cloud_controller/app/models/service_config.rb:45:in
    `provision',/root/cloudfoundry/cloud_controller/cloud_controller/app/controllers/legacy_services_controller.rb:64:in
    `provision',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/implicit_render.rb:4:in
    `send_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/base.rb:150:in
    `process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/rendering.rb:11:in
    `process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/callbacks.rb:18:in
    `block in
    process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/callbacks.rb:441:in
    `_run__3273360498142871003__process_action__4248041742907623254__callbacks',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/callbacks.rb:410:in
    `_run_process_action_callbacks',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/callbacks.rb:94:in
    `run_callbacks',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/callbacks.rb:17:in
    `process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/rescue.rb:17:in
    `process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/instrumentation.rb:30:in
    `block in
    process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/notifications.rb:52:in
    `block in
    instrument',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/notifications/instrumenter.rb:21:in
    `instrument',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/notifications.rb:52:in
    `instrument',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/instrumentation.rb:29:in
    `process_action',/root/cloudfoundry/cloud_controller/cloud_controller/app/controllers/application_controller.rb:16:in
    `process_action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/base.rb:119:in
    `process',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/abstract_controller/rendering.rb:41:in
    `process',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal.rb:138:in
    `dispatch',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal/rack_delegation.rb:14:in
    `dispatch',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_controller/metal.rb:178:in
    `block in
    action',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/routing/route_set.rb:68:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/routing/route_set.rb:68:in
    `dispatch',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/routing/route_set.rb:33:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/route_set.rb:148:in
    `block in
    call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/code_generation.rb:93:in
    `block in
    recognize',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/code_generation.rb:68:in
    `optimized_each',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/code_generation.rb:92:in
    `recognize',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-mount-0.6.14/lib/rack/mount/route_set.rb:139:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/routing/route_set.rb:499:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/best_standards_support.rb:17:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/head.rb:14:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-1.2.5/lib/rack/methodoverride.rb:24:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/params_parser.rb:21:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/cookies.rb:302:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/query_cache.rb:32:in
    `block in
    call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/connection_adapters/abstract/query_cache.rb:28:in
    `cache',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/query_cache.rb:12:in
    `cache',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/query_cache.rb:31:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.0.14/lib/active_record/connection_adapters/abstract/connection_pool.rb:354:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/callbacks.rb:46:in
    `block in
    call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.0.14/lib/active_support/callbacks.rb:416:in
    `_run_call_callbacks',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/callbacks.rb:44:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-1.2.5/lib/rack/sendfile.rb:106:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/remote_ip.rb:48:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/actionpack-3.0.14/lib/action_dispatch/middleware/show_exceptions.rb:47:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/railties-3.0.14/lib/rails/rack/logger.rb:13:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/rack/fiber_pool.rb:19:in
    `block in
    call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/fiber_pool.rb:48:in
    `call',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/fiber_pool.rb:48:in
    `block (3 levels) in
    initialize',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/fiber_pool.rb:47:in
    `loop',/usr/local/rvm/gems/ruby-1.9.2-p320/gems/rack-fiber_pool-0.9.1/lib/fiber_pool.rb:47:in
    `block (2 levels) in
    initialize',/root/cloudfoundry/cloud_controller/cloud_controller/config/initializers/fiber_connection_pool.rb:11:in
    `call',/root/cloudfoundry/cloud_controller/cloud_controller/config/initializers/fiber_connection_pool.rb:11:in
    `block in initialize'>>
    cloud_controller --> [2012-08-09 11:44:02.305925] cc_events - pid=31121
    tid=6d7f fid=bbfe INFO -- [2012-08-09 11:44:02 +0200, :USER, "
    tho...@bar.com <javascript:>", "POST:/services", "mysql", "mysql-a0399",
    :FAILED, "Unexpected response from service gateway"]
    -----------------------

    I give you the cloud_controller configuration file, and the mysql_gateway
    one.

    I changed the real cloud_controller IP by 1.2.3.4.5



    Le jeudi 9 août 2012 11:23:52 UTC+2, Frank Lu a écrit :
    yes. you cloud_controller is new and including latest changes, while
    your vcap& services are out-of-date.
    could you pull the lastet changes in vcap and update submodules.

    if you need service_lifecycle feature, you need cherry-pick this change
    http://reviews.**cloudfoundry.org/#/c/8175/<http://reviews.cloudfoundry.org/#/c/8175/>on the top of your updated vcap.
    On Thu, Aug 9, 2012 at 5:18 PM, bbqjack wrote:

    *root@cloudFoundry:~/cloudfoundry/vcap# git log -1*
    log -1
    commit 56edacab37407c40c1709884a593a5**7867604ae2
    Merge: 57183f6 64752d4
    Author: Figo Feng <ff...@vmware.com>
    Date: Mon Jul 23 07:55:34 2012 +0000

    Merge "remove ip_route in vblob_node.yml"

    *root@cloudFoundry:~/cloudfoundry/vcap# cd services*
    *root@cloudFoundry:~/cloudfoundry/vcap/services# git log -1*

    log -1
    commit 7f530df679252f85d56f4c98063854**099f93fc5f
    Author: Tang Rui <rt...@vmware.com>
    Date: Sun Jul 15 21:30:54 2012 -0600

    Fix the bug that the instance data can be deleted when start timeout

    Change-Id: I9e3f0bab81cffe3683956fb459807**377face18f2

    *root@cloudFoundry:~/cloudfoundry/vcap/services# cd
    ../../cloud_controller/
    *
    *root@cloudFoundry:~/cloudfoundry/cloud_controller# git log -1*

    log -1
    commit 1a5a587c564f9e49c902169c7a20bd**ebf46a7c47
    Merge: 3c92a61 9c2b92e
    Author: Dale Olds <ol...@rbcon.com>
    Date: Mon Aug 6 21:50:13 2012 +0000

    Merge "CFID-364: change default UAA scopes for cloud_controller to
    nil"

    Could this help you?

    --

    Frank(mingfan) Lu


    Ext. 8004

    Tel: 86-21-60349004

    VMware Cloudfoundry Service Team
    10F, KIC III, No. 333 Songhu Road, Yangpu District. Shanghai 200433


    --

    Frank(mingfan) Lu


    Ext. 8004

    Tel: 86-21-60349004

    VMware Cloudfoundry Service Team
    10F, KIC III, No. 333 Songhu Road, Yangpu District. Shanghai 200433

  • Frank Lu at Aug 9, 2012 at 2:50 pm
    I think your cc is updated. but your vcap is still ood.

    cd ~/cloudfoundry/vcap
    git pull
    git submodule update services (and other submodules)

    Because you tried to deploy dev_setup in multi nodes, you had better update
    and re-run vcap_dev_setup to refresh your deployment in each node.

    Hope this coud help you.

    -frank

    On Thu, Aug 9, 2012 at 10:43 PM, bbqjack wrote:

    I did it in the vcap folder, but in
    ~/cloudfoundry/cloud_controller/cloud_controller I get this message

    ~/cloudfoundry/cloud_controller/cloud_controller# git submodule
    submodule
    You need to run this command from the toplevel of the working tree.


    Le jeudi 9 août 2012 15:39:25 UTC+2, Frank Lu a écrit :
    from your log
    1) you cc use the latest vcap_common (1dca468ec017)
    2) but your provision response's format is out-of-date.
    this change has modified the provision response format:
    https://github.com/**cloudfoundry/vcap-common/**commit/**
    b0cc19d5520c9849600dbd8cf2883b**ff66d7643e#L1R87<https://github.com/cloudfoundry/vcap-common/commit/b0cc19d5520c9849600dbd8cf2883bff66d7643e#L1R87>)
    .
    I am afraid, your vcap/services are still not matched with your
    cc.
    On Thu, Aug 9, 2012 at 5:55 PM, bbqjack wrote:

    Oh Thanks ! I didn't know that (I'm not used to Git, it's a really new
    tool for me !)
    The problem is nearly solved.

    Now the mysql gateway receive correctly all messages from nodes BUT the
    cloud controller can't decode it. logs says me that something is missing in
    configuration but I can't find it...

    ------------------------------**---
    cloud_controller --> [2012-08-09 11:44:02.269582] cc - pid=31121
    tid=6d7f fid=bbfe DEBUG -- Attempting to provision service:
    {"type":"database","tier":"**free","vendor":"mysql","**
    version":"5.1","name":"mysql-**a0399"}
    cloud_controller --> [2012-08-09 11:44:02.299817] cc - pid=31121
    tid=6d7f fid=bbfe ERROR -- Error talking to gateway: Field: configuration,
    Error: Missing field configuration
    cloud_controller --> [2012-08-09 11:44:02.299976] cc exception pid=31121
    tid=6d7f fid=bbfe ERROR -- JsonMessage::ValidationError<<**Field:
    configuration, Error: Missing field configuration:/usr/local/rvm/**
    gems/ruby-1.9.2-p320/bundler/**gems/vcap-common-1dca468ec017/**lib/json_message.rb:102:in
    `from_decoded_json',/usr/**local/rvm/gems/ruby-1.9.2-**
    p320/bundler/gems/vcap-common-**1dca468ec017/lib/json_message.**rb:74:in
    `decode',/usr/local/rvm/gems/**ruby-1.9.2-p320/bundler/gems/**
    vcap-common-1dca468ec017/lib/**services/api/clients/service_**gateway_client.rb:78:in
    `provision',/root/**cloudfoundry/cloud_controller/**
    cloud_controller/app/models/**service_config.rb:64:in `block in
    provision',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    activerecord-3.0.14/lib/**active_record/connection_**
    adapters/abstract/database_**statements.rb:139:in
    `transaction',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    activerecord-3.0.14/lib/**active_record/transactions.rb:**207:in
    `transaction',/root/**cloudfoundry/cloud_controller/**
    cloud_controller/app/models/**service_config.rb:45:in `provision',/root/
    **cloudfoundry/cloud_controller/**cloud_controller/app/**
    controllers/legacy_services_**controller.rb:64:in
    `provision',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal/implicit_**render.rb:4:in
    `send_action',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/**abstract_controller/base.rb:**150:in
    `process_action',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal/rendering.rb:**11:in
    `process_action',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/**abstract_controller/callbacks.**rb:18:in `block
    in process_action',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/callbacks.rb:**441:in
    `_run__3273360498142871003__**process_action__**4248041742907623254__**
    callbacks',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/callbacks.rb:**410:in
    `_run_process_action_**callbacks',/usr/local/rvm/**
    gems/ruby-1.9.2-p320/gems/**activesupport-3.0.14/lib/**
    active_support/callbacks.rb:**94:in `run_callbacks',/usr/local/**
    rvm/gems/ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/**
    abstract_controller/callbacks.**rb:17:in `process_action',/usr/local/**
    rvm/gems/ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/action_**
    controller/metal/rescue.rb:17:**in `process_action',/usr/local/**
    rvm/gems/ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/action_**
    controller/metal/**instrumentation.rb:30:in `block in
    process_action',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/notifications.**rb:52:in
    `block in instrument',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/notifications/**instrumenter.rb:21:in
    `instrument',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/notifications.**rb:52:in
    `instrument',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal/**instrumentation.rb:29:in
    `process_action',/root/**cloudfoundry/cloud_controller/**
    cloud_controller/app/**controllers/application_**controller.rb:16:in
    `process_action',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/**abstract_controller/base.rb:**119:in
    `process',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/**abstract_controller/rendering.**rb:41:in
    `process',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal.rb:138:in
    `dispatch',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal/rack_**delegation.rb:14:in
    `dispatch',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**controller/metal.rb:178:in `block in
    action',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/routing/route_set.rb:**68:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/routing/route_set.rb:**68:in
    `dispatch',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/routing/route_set.rb:**33:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-**
    mount-0.6.14/lib/rack/mount/**route_set.rb:148:in `block in
    call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-**
    mount-0.6.14/lib/rack/mount/**code_generation.rb:93:in `block in
    recognize',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    rack-mount-0.6.14/lib/rack/**mount/code_generation.rb:68:in
    `optimized_each',/usr/local/**rvm/gems/ruby-1.9.2-p320/gems/**
    rack-mount-0.6.14/lib/rack/**mount/code_generation.rb:92:in
    `recognize',/usr/local/rvm/**gems/ruby-1.9.2-p320/gems/**
    rack-mount-0.6.14/lib/rack/**mount/route_set.rb:139:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/routing/route_set.rb:**499:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/middleware/best_**standards_support.rb:17:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/middleware/head.rb:**14:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-1.2.**
    5/lib/rack/methodoverride.rb:**24:in `call',/usr/local/rvm/gems/**
    ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/action_**
    dispatch/middleware/params_**parser.rb:21:in `call',/usr/local/rvm/gems/
    **ruby-1.9.2-p320/gems/**actionpack-3.0.14/lib/action_**
    dispatch/middleware/cookies.**rb:302:in `call',/usr/local/rvm/gems/**
    ruby-1.9.2-p320/gems/**activerecord-3.0.14/lib/**
    active_record/query_cache.rb:**32:in `block in
    call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    activerecord-3.0.14/lib/**active_record/connection_**
    adapters/abstract/query_cache.**rb:28:in `cache',/usr/local/rvm/gems/**
    ruby-1.9.2-p320/gems/**activerecord-3.0.14/lib/**
    active_record/query_cache.rb:**12:in `cache',/usr/local/rvm/gems/**
    ruby-1.9.2-p320/gems/**activerecord-3.0.14/lib/**
    active_record/query_cache.rb:**31:in `call',/usr/local/rvm/gems/**
    ruby-1.9.2-p320/gems/**activerecord-3.0.14/lib/**
    active_record/connection_**adapters/abstract/connection_**pool.rb:354:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/middleware/callbacks.**rb:46:in
    `block in call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    activesupport-3.0.14/lib/**active_support/callbacks.rb:**416:in
    `_run_call_callbacks',/usr/**local/rvm/gems/ruby-1.9.2-**
    p320/gems/actionpack-3.0.14/**lib/action_dispatch/**middleware/callbacks.rb:44:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-1.2.**5/lib/rack/sendfile.rb:106:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/middleware/remote_ip.**rb:48:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/**
    actionpack-3.0.14/lib/action_**dispatch/middleware/show_**exceptions.rb:47:in
    `call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/railties-**
    3.0.14/lib/rails/rack/logger.**rb:13:in `call',/usr/local/rvm/gems/**
    ruby-1.9.2-p320/gems/rack-**fiber_pool-0.9.1/lib/rack/**fiber_pool.rb:19:in
    `block in call',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-**
    fiber_pool-0.9.1/lib/fiber_**pool.rb:48:in `call',/usr/local/rvm/gems/**
    ruby-1.9.2-p320/gems/rack-**fiber_pool-0.9.1/lib/fiber_**pool.rb:48:in
    `block (3 levels) in initialize',/usr/local/rvm/**
    gems/ruby-1.9.2-p320/gems/**rack-fiber_pool-0.9.1/lib/**fiber_pool.rb:47:in
    `loop',/usr/local/rvm/gems/**ruby-1.9.2-p320/gems/rack-**
    fiber_pool-0.9.1/lib/fiber_**pool.rb:47:in `block (2 levels) in
    initialize',/root/**cloudfoundry/cloud_controller/**
    cloud_controller/config/**initializers/fiber_connection_**pool.rb:11:in
    `call',/root/cloudfoundry/**cloud_controller/cloud_**controller/config/*
    *initializers/fiber_connection_**pool.rb:11:in `block in initialize'>>
    cloud_controller --> [2012-08-09 11:44:02.305925] cc_events - pid=31121
    tid=6d7f fid=bbfe INFO -- [2012-08-09 11:44:02 +0200, :USER, "
    tho...@bar.com", "POST:/services", "mysql", "mysql-a0399", :FAILED,
    "Unexpected response from service gateway"]
    -----------------------

    I give you the cloud_controller configuration file, and the
    mysql_gateway one.

    I changed the real cloud_controller IP by 1.2.3.4.5



    Le jeudi 9 août 2012 11:23:52 UTC+2, Frank Lu a écrit :
    yes. you cloud_controller is new and including latest changes, while
    your vcap& services are out-of-date.
    could you pull the lastet changes in vcap and update submodules.

    if you need service_lifecycle feature, you need cherry-pick this change
    http://reviews.**cloudfou**ndry.org/#/c/8175/<http://reviews.cloudfoundry.org/#/c/8175/>on the top of your updated vcap.
    On Thu, Aug 9, 2012 at 5:18 PM, bbqjack wrote:

    *root@cloudFoundry:~/cloudfoundry/vcap# git log -1*
    log -1
    commit 56edacab37407c40c1709884a593a5****7867604ae2
    Merge: 57183f6 64752d4
    Author: Figo Feng <ff...@vmware.com>
    Date: Mon Jul 23 07:55:34 2012 +0000

    Merge "remove ip_route in vblob_node.yml"

    *root@cloudFoundry:~/cloudfoundry/vcap# cd services*
    *root@cloudFoundry:~/cloudfoundry/vcap/services# git log -1*

    log -1
    commit 7f530df679252f85d56f4c98063854****099f93fc5f
    Author: Tang Rui <rt...@vmware.com>
    Date: Sun Jul 15 21:30:54 2012 -0600

    Fix the bug that the instance data can be deleted when start
    timeout

    Change-Id: I9e3f0bab81cffe3683956fb459807****377face18f2

    *root@cloudFoundry:~/cloudfoundry/vcap/services# cd
    ../../cloud_controller/
    *
    *root@cloudFoundry:~/cloudfoundry/cloud_controller# git log -1*

    log -1
    commit 1a5a587c564f9e49c902169c7a20bd****ebf46a7c47
    Merge: 3c92a61 9c2b92e
    Author: Dale Olds <ol...@rbcon.com>
    Date: Mon Aug 6 21:50:13 2012 +0000

    Merge "CFID-364: change default UAA scopes for cloud_controller to
    nil"

    Could this help you?

    --

    Frank(mingfan) Lu


    Ext. 8004

    Tel: 86-21-60349004

    VMware Cloudfoundry Service Team
    10F, KIC III, No. 333 Songhu Road, Yangpu District. Shanghai 200433


    --

    Frank(mingfan) Lu


    Ext. 8004

    Tel: 86-21-60349004

    VMware Cloudfoundry Service Team
    10F, KIC III, No. 333 Songhu Road, Yangpu District. Shanghai 200433


    --

    Frank(mingfan) Lu


    Ext. 8004

    Tel: 86-21-60349004

    VMware Cloudfoundry Service Team
    10F, KIC III, No. 333 Songhu Road, Yangpu District. Shanghai 200433

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupvcap-dev @
postedAug 9, '12 at 8:11a
activeAug 9, '12 at 2:50p
posts13
users3

3 users in discussion

Bbqjack: 6 posts Frank Lu: 5 posts Haipeng Wu: 2 posts

People

Translate

site design / logo © 2021 Grokbase