FAQ
Hi,

According to
https://github.com/cloudfoundry/dea/blob/master/lib/dea/agent.rb#L707, disk
limit is force to be 1GB even if disk quota settings in CloudController
database is larger than 1GB.

What the purpose for 1GB limitations?

We'd like to set enforce_ulimit for file descriptors, memory, and disks but
we have 4GB disk limit per an application.

F.Y.I, we customize limits properties in CCDB to optimize our
infrastructure: https://github.com/rakutentech/vcap/commit/1cfcac55b95a646046287721d0923c29abd67376
where we host single tenant DEA with fixed size of VM.

Search Discussions

  • Anfernee Gui at Jul 4, 2012 at 7:08 am
    Hi,

    Ulimit ­f just limits the size of a single opened file (only affects on
    write), AFAIK. It didn't limit the disk quota.
    The disk quota enforcement code is here:
    https://github.com/cloudfoundry/dea/blob/master/lib/dea/agent.rb#L1421

    Thanks,
    Anfernee

    From: yssk22 <yssk22@gmail.com>
    Reply-To: <vcap-dev@cloudfoundry.org>
    Date: Tue, 3 Jul 2012 23:52:59 -0700 (PDT)
    To: <vcap-dev@cloudfoundry.org>
    Subject: [vcap-dev] What is the purpose for forcing 1GB disk limit on
    'enforce_limit' environment?

    Hi,

    According to
    https://github.com/cloudfoundry/dea/blob/master/lib/dea/agent.rb#L707, disk
    limit is force to be 1GB even if disk quota settings in CloudController
    database is larger than 1GB.

    What the purpose for 1GB limitations?

    We'd like to set enforce_ulimit for file descriptors, memory, and disks but
    we have 4GB disk limit per an application.

    F.Y.I, we customize limits properties in CCDB to optimize our
    infrastructure:
    https://github.com/rakutentech/vcap/commit/1cfcac55b95a646046287721d0923c29a
    bd67376
    where we host single tenant DEA with fixed size of VM.
  • Yohei Sasaki at Jul 4, 2012 at 7:47 am
    oh, thanks. I got it!

    so do you think some log files such as stdout.log or stderr.log should
    be limited up to 1GB even if there are much more resources in
    enforce_ulimit environment? or should this be configurable on dea.yml?

    I think all of limits hard coded in agent.rb should be configurable..

    Thanks.

    2012/7/4 Anfernee Gui <agui@rbcon.com>:
    Hi,

    Ulimit –f just limits the size of a single opened file (only affects on
    write), AFAIK. It didn't limit the disk quota.
    The disk quota enforcement code is here:
    https://github.com/cloudfoundry/dea/blob/master/lib/dea/agent.rb#L1421

    Thanks,
    Anfernee

    From: yssk22 <yssk22@gmail.com>
    Reply-To: <vcap-dev@cloudfoundry.org>
    Date: Tue, 3 Jul 2012 23:52:59 -0700 (PDT)
    To: <vcap-dev@cloudfoundry.org>
    Subject: [vcap-dev] What is the purpose for forcing 1GB disk limit on
    'enforce_limit' environment?

    Hi,

    According to
    https://github.com/cloudfoundry/dea/blob/master/lib/dea/agent.rb#L707, disk
    limit is force to be 1GB even if disk quota settings in CloudController
    database is larger than 1GB.

    What the purpose for 1GB limitations?

    We'd like to set enforce_ulimit for file descriptors, memory, and disks but
    we have 4GB disk limit per an application.

    F.Y.I, we customize limits properties in CCDB to optimize our
    infrastructure:
    https://github.com/rakutentech/vcap/commit/1cfcac55b95a646046287721d0923c29abd67376
    where we host single tenant DEA with fixed size of VM.


    --
    Yohei SASAKI
    http://www.yssk22.info/
  • Anfernee Gui at Jul 4, 2012 at 9:24 am
    After DEA is moved into warden, most of these code will be abandoned.
    There will be no ulimit. In warden, those options will possibly be
    configurable, I guess..

    Thanks,
    Anfernee
    On 7/4/12 3:47 PM, "Yohei Sasaki" wrote:

    oh, thanks. I got it!

    so do you think some log files such as stdout.log or stderr.log should
    be limited up to 1GB even if there are much more resources in
    enforce_ulimit environment? or should this be configurable on dea.yml?

    I think all of limits hard coded in agent.rb should be configurable..

    Thanks.

    2012/7/4 Anfernee Gui <agui@rbcon.com>:
    Hi,

    Ulimit ­f just limits the size of a single opened file (only affects on
    write), AFAIK. It didn't limit the disk quota.
    The disk quota enforcement code is here:
    https://github.com/cloudfoundry/dea/blob/master/lib/dea/agent.rb#L1421

    Thanks,
    Anfernee

    From: yssk22 <yssk22@gmail.com>
    Reply-To: <vcap-dev@cloudfoundry.org>
    Date: Tue, 3 Jul 2012 23:52:59 -0700 (PDT)
    To: <vcap-dev@cloudfoundry.org>
    Subject: [vcap-dev] What is the purpose for forcing 1GB disk limit on
    'enforce_limit' environment?

    Hi,

    According to
    https://github.com/cloudfoundry/dea/blob/master/lib/dea/agent.rb#L707,
    disk
    limit is force to be 1GB even if disk quota settings in CloudController
    database is larger than 1GB.

    What the purpose for 1GB limitations?

    We'd like to set enforce_ulimit for file descriptors, memory, and disks
    but
    we have 4GB disk limit per an application.

    F.Y.I, we customize limits properties in CCDB to optimize our
    infrastructure:

    https://github.com/rakutentech/vcap/commit/1cfcac55b95a646046287721d0923c
    29abd67376
    where we host single tenant DEA with fixed size of VM.


    --
    Yohei SASAKI
    http://www.yssk22.info/
  • Yohei Sasaki at Jul 4, 2012 at 9:34 am
    Thanks, I understand your answer but...

    I think warden should be the one of options for DEA deployment...
    There should be the following options:

    A. Physical Server + DEA + warden
    B. Physical Server + DEA + single-tenant option
    C. Physical Server + DEA + multi-tenant option
    D. Virtual Server + DEA + warden
    E. Virtual Sever + DEA + single-tenant option
    F. Virtual Sever + DEA + multi-tenant option

    B options would be rare cases, I don't know there are demands for D
    option. but ulimit configration is useful for B, C, E, and D. Actually
    we deployed production environment with E, and development environment
    with F.

    Or does CloudFoundry focus on one of them?

    2012/7/4 Anfernee Gui <agui@rbcon.com>:
    After DEA is moved into warden, most of these code will be abandoned.
    There will be no ulimit. In warden, those options will possibly be
    configurable, I guess..

    Thanks,
    Anfernee
    On 7/4/12 3:47 PM, "Yohei Sasaki" wrote:

    oh, thanks. I got it!

    so do you think some log files such as stdout.log or stderr.log should
    be limited up to 1GB even if there are much more resources in
    enforce_ulimit environment? or should this be configurable on dea.yml?

    I think all of limits hard coded in agent.rb should be configurable..

    Thanks.

    2012/7/4 Anfernee Gui <agui@rbcon.com>:
    Hi,

    Ulimit ­f just limits the size of a single opened file (only affects on
    write), AFAIK. It didn't limit the disk quota.
    The disk quota enforcement code is here:
    https://github.com/cloudfoundry/dea/blob/master/lib/dea/agent.rb#L1421

    Thanks,
    Anfernee

    From: yssk22 <yssk22@gmail.com>
    Reply-To: <vcap-dev@cloudfoundry.org>
    Date: Tue, 3 Jul 2012 23:52:59 -0700 (PDT)
    To: <vcap-dev@cloudfoundry.org>
    Subject: [vcap-dev] What is the purpose for forcing 1GB disk limit on
    'enforce_limit' environment?

    Hi,

    According to
    https://github.com/cloudfoundry/dea/blob/master/lib/dea/agent.rb#L707,
    disk
    limit is force to be 1GB even if disk quota settings in CloudController
    database is larger than 1GB.

    What the purpose for 1GB limitations?

    We'd like to set enforce_ulimit for file descriptors, memory, and disks
    but
    we have 4GB disk limit per an application.

    F.Y.I, we customize limits properties in CCDB to optimize our
    infrastructure:

    https://github.com/rakutentech/vcap/commit/1cfcac55b95a646046287721d0923c
    29abd67376
    where we host single tenant DEA with fixed size of VM.


    --
    Yohei SASAKI
    http://www.yssk22.info/


    --
    Yohei SASAKI
    http://www.yssk22.info/
  • Yongkun Anfernee Gui at Jul 4, 2012 at 10:24 am
    I am not sure I understand you correctly. But warden might not be proper to
    be
    together with single-tenant and multi-tenant. Warden (ulimit) is used to
    limit
    user apps in a sand box, which is useful in multi-tenant env.

    So it's a question of to use warden or ulimit, to support single or multi
    tenant.
    BTW, physical/virtual server makes no difference for DEA.

    Thanks,
    Anfernee
    On Wed, Jul 4, 2012 at 5:34 PM, Yohei Sasaki wrote:

    Thanks, I understand your answer but...

    I think warden should be the one of options for DEA deployment...
    There should be the following options:

    A. Physical Server + DEA + warden
    B. Physical Server + DEA + single-tenant option
    C. Physical Server + DEA + multi-tenant option
    D. Virtual Server + DEA + warden
    E. Virtual Sever + DEA + single-tenant option
    F. Virtual Sever + DEA + multi-tenant option

    B options would be rare cases, I don't know there are demands for D
    option. but ulimit configration is useful for B, C, E, and D. Actually
    we deployed production environment with E, and development environment
    with F.

    Or does CloudFoundry focus on one of them?

    2012/7/4 Anfernee Gui <agui@rbcon.com>:
    After DEA is moved into warden, most of these code will be abandoned.
    There will be no ulimit. In warden, those options will possibly be
    configurable, I guess..

    Thanks,
    Anfernee
    On 7/4/12 3:47 PM, "Yohei Sasaki" wrote:

    oh, thanks. I got it!

    so do you think some log files such as stdout.log or stderr.log should
    be limited up to 1GB even if there are much more resources in
    enforce_ulimit environment? or should this be configurable on dea.yml?

    I think all of limits hard coded in agent.rb should be configurable..

    Thanks.

    2012/7/4 Anfernee Gui <agui@rbcon.com>:
    Hi,

    Ulimit ­f just limits the size of a single opened file (only affects on
    write), AFAIK. It didn't limit the disk quota.
    The disk quota enforcement code is here:
    https://github.com/cloudfoundry/dea/blob/master/lib/dea/agent.rb#L1421

    Thanks,
    Anfernee

    From: yssk22 <yssk22@gmail.com>
    Reply-To: <vcap-dev@cloudfoundry.org>
    Date: Tue, 3 Jul 2012 23:52:59 -0700 (PDT)
    To: <vcap-dev@cloudfoundry.org>
    Subject: [vcap-dev] What is the purpose for forcing 1GB disk limit on
    'enforce_limit' environment?

    Hi,

    According to
    https://github.com/cloudfoundry/dea/blob/master/lib/dea/agent.rb#L707,
    disk
    limit is force to be 1GB even if disk quota settings in CloudController
    database is larger than 1GB.

    What the purpose for 1GB limitations?

    We'd like to set enforce_ulimit for file descriptors, memory, and disks
    but
    we have 4GB disk limit per an application.

    F.Y.I, we customize limits properties in CCDB to optimize our
    infrastructure:
    https://github.com/rakutentech/vcap/commit/1cfcac55b95a646046287721d0923c
    29abd67376
    where we host single tenant DEA with fixed size of VM.


    --
    Yohei SASAKI
    http://www.yssk22.info/


    --
    Yohei SASAKI
    http://www.yssk22.info/
  • Andrea Campi at Jul 4, 2012 at 10:36 am

    On Wed, Jul 4, 2012 at 12:24 PM, Yongkun Anfernee Gui wrote:
    I am not sure I understand you correctly. But warden might not be proper to
    be
    together with single-tenant and multi-tenant. Warden (ulimit) is used to
    limit
    user apps in a sand box, which is useful in multi-tenant env.
    Warden is much more than ulimit though, it also means cgroups, per-app
    firewall, per-app fs mounts, per-app nginx...

    These features may be less important if you have a single tenant DEA
    for an app you absolutely trust.
    Using warden in every case keeps the single- and multi-tenants
    identical, and enforces the security model and the separation of the
    "app-space" from the host (DEA) OS.
    Those advantages are so big, and personally I don't see any downside
    to running within warden; so in my book, always using warden makes
    total sense.
  • Yohei Sasaki at Jul 4, 2012 at 11:28 am
    Let me clarify.

    In our environment, we deploy single-tenant DEAs, each of which is a
    small VMs on a large physical server. so in terms of security among
    apps, we can protect each other by VM-level seperation.

    But in a DEA, we have several 'special' processes for applications
    such as log collector daemon, SMTP gateway daemon, ...etc. So our
    purpose to use 'ulimit' is separate these special processes and
    deployed apps.

    In this context, should we use warden on our small VM? I think it's
    less useful and ulimit configuration on dea.yml is useful.

    2012/7/4 Andrea Campi <andrea.campi@zephirworks.com>:
    On Wed, Jul 4, 2012 at 12:24 PM, Yongkun Anfernee Gui wrote:
    I am not sure I understand you correctly. But warden might not be proper to
    be
    together with single-tenant and multi-tenant. Warden (ulimit) is used to
    limit
    user apps in a sand box, which is useful in multi-tenant env.
    Warden is much more than ulimit though, it also means cgroups, per-app
    firewall, per-app fs mounts, per-app nginx...

    These features may be less important if you have a single tenant DEA
    for an app you absolutely trust.
    Using warden in every case keeps the single- and multi-tenants
    identical, and enforces the security model and the separation of the
    "app-space" from the host (DEA) OS.
    Those advantages are so big, and personally I don't see any downside
    to running within warden; so in my book, always using warden makes
    total sense.
  • Andrea Campi at Jul 4, 2012 at 4:27 pm

    On Wed, Jul 4, 2012 at 1:28 PM, Yohei Sasaki wrote:
    Let me clarify.

    In our environment, we deploy single-tenant DEAs, each of which is a
    small VMs on a large physical server. so in terms of security among
    apps, we can protect each other by VM-level seperation.

    But in a DEA, we have several 'special' processes for applications
    such as log collector daemon, SMTP gateway daemon, ...etc. So our
    purpose to use 'ulimit' is separate these special processes and
    deployed apps.

    In this context, should we use warden on our small VM? I think it's
    less useful and ulimit configuration on dea.yml is useful.
    Even if you separate VMs, isolating the application code running "in
    the DEA" from your 'special' processes; so that even if the app
    misbehaves, get compromised etc your processes won't be compromised.

    Let me turn that around: would using Warden hurt? I don't think so,
    LXC / cgroups are very light-weight.
    The CF project is designed for a use-case that is different from
    yours, in particular multi-tenancy.
    So it's pretty much a given that going forward there will be more
    isolation, and as far I can tell from code in the review code, it will
    be mandatory.
    Let's say it does and the burden falls on you to patch CF to run
    without Warden--do you want that so badly that you would be willing to
    commit time and resources to develop and maintain that?

    Andrea
  • Yssk22 at Jul 5, 2012 at 1:45 pm
    Thanks,

    I agree to use warden for isolation. But I wondered we needed it in a many
    small VMs. And when warden is completely available in DEA, we need to
    redesign VMs (to be large ones) for it.

    Anyway I can get to know where CF goes. Thanks!

    2012年7月5日木曜日 1時27分16秒 UTC+9 Andrea Campi:
    On Wed, Jul 4, 2012 at 1:28 PM, Yohei Sasaki wrote:
    Let me clarify.

    In our environment, we deploy single-tenant DEAs, each of which is a
    small VMs on a large physical server. so in terms of security among
    apps, we can protect each other by VM-level seperation.

    But in a DEA, we have several 'special' processes for applications
    such as log collector daemon, SMTP gateway daemon, ...etc. So our
    purpose to use 'ulimit' is separate these special processes and
    deployed apps.

    In this context, should we use warden on our small VM? I think it's
    less useful and ulimit configuration on dea.yml is useful.
    Even if you separate VMs, isolating the application code running "in
    the DEA" from your 'special' processes; so that even if the app
    misbehaves, get compromised etc your processes won't be compromised.

    Let me turn that around: would using Warden hurt? I don't think so,
    LXC / cgroups are very light-weight.
    The CF project is designed for a use-case that is different from
    yours, in particular multi-tenancy.
    So it's pretty much a given that going forward there will be more
    isolation, and as far I can tell from code in the review code, it will
    be mandatory.
    Let's say it does and the burden falls on you to patch CF to run
    without Warden--do you want that so badly that you would be willing to
    commit time and resources to develop and maintain that?

    Andrea

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupvcap-dev @
postedJul 4, '12 at 6:53a
activeJul 5, '12 at 1:45p
posts10
users3

People

Translate

site design / logo © 2021 Grokbase