I'm trying to boot a stemcell built from HEAD and it is timing out. But in
/var/vcap/bosh/log/current it seems to only get to running mke2fs; and
never gets to downloading packages/jobs and setting up monit.

Dump of "current" log at https://gist.github.com/4460317

Could mke2fs take so long that 300s isn't long enough of a time out for a
new VM & new persistent disk?

Other ideas?

Nic

--
Dr Nic Williams
http://drnicwilliams.com
cell +1 (415) 860-2185

Search Discussions

  • Martin Englund at Jan 5, 2013 at 8:33 am
    If there is a EBS problem, yes

    After it creates the ephemeral disk here:
    https://github.com/cloudfoundry/bosh/blob/master/agent/lib/agent/bootstrap.rb#L168

    it logs a message "Swapon and mount data partition" which isn't shown
    in your log, so it must be that. Does it happen consistently?

    /M
    On Sat, Jan 5, 2013 at 8:25 AM, Dr Nic Williams wrote:
    I'm trying to boot a stemcell built from HEAD and it is timing out. But in
    /var/vcap/bosh/log/current it seems to only get to running mke2fs; and never
    gets to downloading packages/jobs and setting up monit.

    Dump of "current" log at https://gist.github.com/4460317

    Could mke2fs take so long that 300s isn't long enough of a time out for a
    new VM & new persistent disk?

    Other ideas?

    Nic

    --
    Dr Nic Williams
    http://drnicwilliams.com
    cell +1 (415) 860-2185


    --
    cheers,
    /Martin
    --
    Martin Englund, Staff Engineer, Cloud Foundry, VMware Inc.
    "The question is not if you are paranoid, it is if you are paranoid enough."
  • Dr Nic Williams at Jan 5, 2013 at 3:48 pm
    It happened twice in a row and then I went to bed. Will try again today.
    On Saturday, January 5, 2013, Martin Englund wrote:

    If there is a EBS problem, yes

    After it creates the ephemeral disk here:

    https://github.com/cloudfoundry/bosh/blob/master/agent/lib/agent/bootstrap.rb#L168

    it logs a message "Swapon and mount data partition" which isn't shown
    in your log, so it must be that. Does it happen consistently?

    /M
    On Sat, Jan 5, 2013 at 8:25 AM, Dr Nic Williams wrote:
    I'm trying to boot a stemcell built from HEAD and it is timing out. But in
    /var/vcap/bosh/log/current it seems to only get to running mke2fs; and never
    gets to downloading packages/jobs and setting up monit.

    Dump of "current" log at https://gist.github.com/4460317

    Could mke2fs take so long that 300s isn't long enough of a time out for a
    new VM & new persistent disk?

    Other ideas?

    Nic

    --
    Dr Nic Williams
    http://drnicwilliams.com
    cell +1 (415) 860-2185


    --
    cheers,
    /Martin
    --
    Martin Englund, Staff Engineer, Cloud Foundry, VMware Inc.
    "The question is not if you are paranoid, it is if you are paranoid
    enough."

    --
    Dr Nic Williams
    http://drnicwilliams.com
    cell +1 (415) 860-2185
  • Dr Nic Williams at Jan 5, 2013 at 4:21 pm
    Still seeing the issue this morning.

    The df listing is:
    # df
    Filesystem 1K-blocks Used Available Use% Mounted on
    /dev/xvda1 1325436 978048 280056 78% /
    none 3809240 124 3809116 1% /dev
    none 3822980 0 3822980 0% /dev/shm
    none 3822980 48 3822932 1% /var/run
    none 3822980 0 3822980 0% /var/lock
    none 3822980 0 3822980 0% /lib/init/rw
    none 1325436 978048 280056 78%
    /var/lib/ureadahead/debugfs

    There is only one EBS volume created during the deploy (the root volume).

    On Sat, Jan 5, 2013 at 7:48 AM, Dr Nic Williams wrote:

    It happened twice in a row and then I went to bed. Will try again today.

    On Saturday, January 5, 2013, Martin Englund wrote:

    If there is a EBS problem, yes

    After it creates the ephemeral disk here:

    https://github.com/cloudfoundry/bosh/blob/master/agent/lib/agent/bootstrap.rb#L168

    it logs a message "Swapon and mount data partition" which isn't shown
    in your log, so it must be that. Does it happen consistently?

    /M

    On Sat, Jan 5, 2013 at 8:25 AM, Dr Nic Williams <drnicwilliams@gmail.com>
    wrote:
    I'm trying to boot a stemcell built from HEAD and it is timing out. But in
    /var/vcap/bosh/log/current it seems to only get to running mke2fs; and never
    gets to downloading packages/jobs and setting up monit.

    Dump of "current" log at https://gist.github.com/4460317

    Could mke2fs take so long that 300s isn't long enough of a time out for a
    new VM & new persistent disk?

    Other ideas?

    Nic

    --
    Dr Nic Williams
    http://drnicwilliams.com
    cell +1 (415) 860-2185


    --
    cheers,
    /Martin
    --
    Martin Englund, Staff Engineer, Cloud Foundry, VMware Inc.
    "The question is not if you are paranoid, it is if you are paranoid
    enough."

    --
    Dr Nic Williams
    http://drnicwilliams.com
    cell +1 (415) 860-2185

    --
    Dr Nic Williams
    http://drnicwilliams.com
    cell +1 (415) 860-2185
  • Dr Nic Williams at Jan 6, 2013 at 7:21 am
    I went back and recreated my microbosh from 0.7.0 and uploaded the 0.6.7
    stemcell; and I'm still getting this issue.

    Is anyone else seeing this on AWS or conversely, anyone successfully
    booting VMs on AWS?

    Nic

    On Sat, Jan 5, 2013 at 8:21 AM, Dr Nic Williams wrote:

    Still seeing the issue this morning.

    The df listing is:
    # df
    Filesystem 1K-blocks Used Available Use% Mounted on
    /dev/xvda1 1325436 978048 280056 78% /
    none 3809240 124 3809116 1% /dev
    none 3822980 0 3822980 0% /dev/shm
    none 3822980 48 3822932 1% /var/run
    none 3822980 0 3822980 0% /var/lock
    none 3822980 0 3822980 0% /lib/init/rw
    none 1325436 978048 280056 78%
    /var/lib/ureadahead/debugfs

    There is only one EBS volume created during the deploy (the root volume).

    On Sat, Jan 5, 2013 at 7:48 AM, Dr Nic Williams wrote:

    It happened twice in a row and then I went to bed. Will try again today.

    On Saturday, January 5, 2013, Martin Englund wrote:

    If there is a EBS problem, yes

    After it creates the ephemeral disk here:

    https://github.com/cloudfoundry/bosh/blob/master/agent/lib/agent/bootstrap.rb#L168

    it logs a message "Swapon and mount data partition" which isn't shown
    in your log, so it must be that. Does it happen consistently?

    /M

    On Sat, Jan 5, 2013 at 8:25 AM, Dr Nic Williams <drnicwilliams@gmail.com>
    wrote:
    I'm trying to boot a stemcell built from HEAD and it is timing out. But in
    /var/vcap/bosh/log/current it seems to only get to running mke2fs; and never
    gets to downloading packages/jobs and setting up monit.

    Dump of "current" log at https://gist.github.com/4460317

    Could mke2fs take so long that 300s isn't long enough of a time out for a
    new VM & new persistent disk?

    Other ideas?

    Nic

    --
    Dr Nic Williams
    http://drnicwilliams.com
    cell +1 (415) 860-2185


    --
    cheers,
    /Martin
    --
    Martin Englund, Staff Engineer, Cloud Foundry, VMware Inc.
    "The question is not if you are paranoid, it is if you are paranoid
    enough."

    --
    Dr Nic Williams
    http://drnicwilliams.com
    cell +1 (415) 860-2185

    --
    Dr Nic Williams
    http://drnicwilliams.com
    cell +1 (415) 860-2185


    --
    Dr Nic Williams
    http://drnicwilliams.com
    cell +1 (415) 860-2185
  • Dr Nic Williams at Jan 6, 2013 at 10:34 pm
    After 24 hrs of this odd bug it seems to have disappeared.

    On Sat, Jan 5, 2013 at 11:21 PM, Dr Nic Williams wrote:

    I went back and recreated my microbosh from 0.7.0 and uploaded the 0.6.7
    stemcell; and I'm still getting this issue.

    Is anyone else seeing this on AWS or conversely, anyone successfully
    booting VMs on AWS?

    Nic

    On Sat, Jan 5, 2013 at 8:21 AM, Dr Nic Williams wrote:

    Still seeing the issue this morning.

    The df listing is:
    # df
    Filesystem 1K-blocks Used Available Use% Mounted on
    /dev/xvda1 1325436 978048 280056 78% /
    none 3809240 124 3809116 1% /dev
    none 3822980 0 3822980 0% /dev/shm
    none 3822980 48 3822932 1% /var/run
    none 3822980 0 3822980 0% /var/lock
    none 3822980 0 3822980 0% /lib/init/rw
    none 1325436 978048 280056 78%
    /var/lib/ureadahead/debugfs

    There is only one EBS volume created during the deploy (the root volume).

    On Sat, Jan 5, 2013 at 7:48 AM, Dr Nic Williams wrote:

    It happened twice in a row and then I went to bed. Will try again today.

    On Saturday, January 5, 2013, Martin Englund wrote:

    If there is a EBS problem, yes

    After it creates the ephemeral disk here:

    https://github.com/cloudfoundry/bosh/blob/master/agent/lib/agent/bootstrap.rb#L168

    it logs a message "Swapon and mount data partition" which isn't shown
    in your log, so it must be that. Does it happen consistently?

    /M

    On Sat, Jan 5, 2013 at 8:25 AM, Dr Nic Williams <
    drnicwilliams@gmail.com> wrote:
    I'm trying to boot a stemcell built from HEAD and it is timing out. But in
    /var/vcap/bosh/log/current it seems to only get to running mke2fs; and never
    gets to downloading packages/jobs and setting up monit.

    Dump of "current" log at https://gist.github.com/4460317

    Could mke2fs take so long that 300s isn't long enough of a time out for a
    new VM & new persistent disk?

    Other ideas?

    Nic

    --
    Dr Nic Williams
    http://drnicwilliams.com
    cell +1 (415) 860-2185


    --
    cheers,
    /Martin
    --
    Martin Englund, Staff Engineer, Cloud Foundry, VMware Inc.
    "The question is not if you are paranoid, it is if you are paranoid
    enough."

    --
    Dr Nic Williams
    http://drnicwilliams.com
    cell +1 (415) 860-2185

    --
    Dr Nic Williams
    http://drnicwilliams.com
    cell +1 (415) 860-2185


    --
    Dr Nic Williams
    http://drnicwilliams.com
    cell +1 (415) 860-2185


    --
    Dr Nic Williams
    http://drnicwilliams.com
    cell +1 (415) 860-2185

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupbosh-dev @
postedJan 5, '13 at 7:25a
activeJan 6, '13 at 10:34p
posts6
users2

2 users in discussion

Dr Nic Williams: 5 posts Martin Englund: 1 post

People

Translate

site design / logo © 2021 Grokbase