FAQ
Hello everybody. I'm a huge fan of node.js and ve're actively using it in
our project for creating various services.

Recently we've faced a problem of significant ram overhead required to run
each node.js instance (about 40Mb). This is an issue since we are running a
number of different node.js services on one machine, and the memory is
limited.

So what we trying to use is run multiple services in single node.js process
using vm.runInNewContext(), but compared this approach is missing some
functionality:

1) We cannot monitor amount of memory allocated by each subprocess.
2) We cannot cancel all setTimeouts and process.nextTicks for given process
to effectively stop its execution.
3) We cannot get a list of open sockets and file descriptors for given
process.

With this functionality we could use node.js similar to erlang - run a
number of processes inside a single vm - and use some erlang features (like
'let it fail') that would certanly lead us to world domination ;)

So, is there any projects targeted to solve similar problem?

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Search Discussions

  • Bradley Meck at Oct 13, 2012 at 1:00 pm
    Isolates were taken a look at a while ago. They are similar to what you
    want, but still unable to do so. It would generally be more sane to just
    scale out processes was what was decided. The edge cases of giving multiple
    "light weight" isolates proved to be very difficult to prevent Isolates
    from affecting each other (env variables, cwd, ulimits, etc.). If you are
    running large numbers of processes and having memory issues I might
    recommend you look into where that memory is coming from, node itself is
    around 10Mb overhead. The rest is most likely some kind of cache or
    non-GCable objects in your application. Note: Code source is cached, and
    coffeescript can compound things in unexpected ways (not always bad,
    inspect yourself).
    On Saturday, October 13, 2012 4:29:17 AM UTC-5, Alexey Guskov wrote:

    Hello everybody. I'm a huge fan of node.js and ve're actively using it in
    our project for creating various services.

    Recently we've faced a problem of significant ram overhead required to run
    each node.js instance (about 40Mb). This is an issue since we are running a
    number of different node.js services on one machine, and the memory is
    limited.

    So what we trying to use is run multiple services in single node.js
    process using vm.runInNewContext(), but compared this approach is missing
    some functionality:

    1) We cannot monitor amount of memory allocated by each subprocess.
    2) We cannot cancel all setTimeouts and process.nextTicks for given
    process to effectively stop its execution.
    3) We cannot get a list of open sockets and file descriptors for given
    process.

    With this functionality we could use node.js similar to erlang - run a
    number of processes inside a single vm - and use some erlang features (like
    'let it fail') that would certanly lead us to world domination ;)

    So, is there any projects targeted to solve similar problem?
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Alex Kocharin at Oct 13, 2012 at 1:09 pm
    Hello Alexey,=A02 and 3 is quite doable using do= mains, and 1st could be achievable with v8
    debugger.=A0
  • Alexey Guskov at Oct 14, 2012 at 5:29 pm
    Yep i see, timers and fds must be accessible via domain.members, but
    process.nextTick() tasks aren't there, am i wrong?

    I'll investigate what we can achive with debugger, thanks for the idea.

    суббота, 13 октября 2012 г., 17:09:58 UTC+4 пользователь Alex Kocharin
    написал:
    Hello Alexey,

    2 and 3 is quite doable using domains, and 1st could be achievable with v8
    debugger.

    It's a hack, but nevertheless it's possible, if all your processes follow
    some rules and don't mess with each other. But I'm afraid that an infinite
    loop in a service will screw the whole process anyway.
    --
    // alex


    13.10.2012, 13:29, "Alexey Guskov" <kvas...@gmail.com <javascript:>>:

    Hello everybody. I'm a huge fan of node.js and ve're actively using it in
    our project for creating various services.
    Recently we've faced a problem of significant ram overhead required to run
    each node.js instance (about 40Mb). This is an issue since we are running a
    number of different node.js services on one machine, and the memory is
    limited.
    So what we trying to use is run multiple services in single node.js
    process using vm.runInNewContext(), but compared this approach is missing
    some functionality:
    1) We cannot monitor amount of memory allocated by each subprocess.
    2) We cannot cancel all setTimeouts and process.nextTicks for given
    process to effectively stop its execution.
    3) We cannot get a list of open sockets and file descriptors for given
    process.
    With this functionality we could use node.js similar to erlang - run a
    number of processes inside a single vm - and use some erlang features (like
    'let it fail') that would certanly lead us to world domination ;)
    So, is there any projects targeted to solve similar problem?


    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines:
    https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nod...@googlegroups.com <javascript:>
    To unsubscribe from this group, send email to
    nodejs+un...@googlegroups.com <javascript:>
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Alexey Petrushin at Oct 15, 2012 at 5:50 am
    Not direct solution to a problems, but an interesting approach -
    node-fibers - it allow to use try/catch, it simplifies error handling.

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Alexey Guskov at Oct 15, 2012 at 6:44 pm
    By the way, i believe the way google chrome deals with multiple pages and
    browser addons is very similar to what i'm trying to achieve: isolated js
    vm instances inside a single process. If anyone knows chrome internals,
    please correct me.

    суббота, 13 октября 2012 г., 13:29:17 UTC+4 пользователь Alexey Guskov
    написал:
    Hello everybody. I'm a huge fan of node.js and ve're actively using it in
    our project for creating various services.

    Recently we've faced a problem of significant ram overhead required to run
    each node.js instance (about 40Mb). This is an issue since we are running a
    number of different node.js services on one machine, and the memory is
    limited.

    So what we trying to use is run multiple services in single node.js
    process using vm.runInNewContext(), but compared this approach is missing
    some functionality:

    1) We cannot monitor amount of memory allocated by each subprocess.
    2) We cannot cancel all setTimeouts and process.nextTicks for given
    process to effectively stop its execution.
    3) We cannot get a list of open sockets and file descriptors for given
    process.

    With this functionality we could use node.js similar to erlang - run a
    number of processes inside a single vm - and use some erlang features (like
    'let it fail') that would certanly lead us to world domination ;)

    So, is there any projects targeted to solve similar problem?
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Michal Kruk at Oct 16, 2012 at 2:06 pm
    Actually chrome has multiple processes running, i think its creating one
    for every tab you have open

    On Mon, Oct 15, 2012 at 8:44 PM, Alexey Guskov wrote:

    By the way, i believe the way google chrome deals with multiple pages and
    browser addons is very similar to what i'm trying to achieve: isolated js
    vm instances inside a single process. If anyone knows chrome internals,
    please correct me.

    суббота, 13 октября 2012 г., 13:29:17 UTC+4 пользователь Alexey Guskov
    написал:
    Hello everybody. I'm a huge fan of node.js and ve're actively using it in
    our project for creating various services.

    Recently we've faced a problem of significant ram overhead required to
    run each node.js instance (about 40Mb). This is an issue since we are
    running a number of different node.js services on one machine, and the
    memory is limited.

    So what we trying to use is run multiple services in single node.js
    process using vm.runInNewContext(), but compared this approach is missing
    some functionality:

    1) We cannot monitor amount of memory allocated by each subprocess.
    2) We cannot cancel all setTimeouts and process.nextTicks for given
    process to effectively stop its execution.
    3) We cannot get a list of open sockets and file descriptors for given
    process.

    With this functionality we could use node.js similar to erlang - run a
    number of processes inside a single vm - and use some erlang features (like
    'let it fail') that would certanly lead us to world domination ;)

    So, is there any projects targeted to solve similar problem?
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines:
    https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Alexey G at Oct 16, 2012 at 2:49 pm
    Thats' right, but it's doesn't always create a separate process, sometimes
    multiple tabs run in single process (when they're from same domain and from
    same origin). Besides, chrome addons certanly live in same process, but are
    somehow isolated from each other and js in pages. I suppose they're created
    by single vm, memory overhead would be significant otherwise.

    Anyway, it must be possible to create multiple vm's in v8 and isolates
    feature seems to be what i'm looking for. It's not implemented, however,
    and i don't know if it will be. So for now i'll take a look on what can be
    done with domains.

    2012/10/16 Michal Kruk <kruczy@gmail.com>
    Actually chrome has multiple processes running, i think its creating one
    for every tab you have open

    On Mon, Oct 15, 2012 at 8:44 PM, Alexey Guskov wrote:

    By the way, i believe the way google chrome deals with multiple pages and
    browser addons is very similar to what i'm trying to achieve: isolated js
    vm instances inside a single process. If anyone knows chrome internals,
    please correct me.

    суббота, 13 октября 2012 г., 13:29:17 UTC+4 пользователь Alexey Guskov
    написал:
    Hello everybody. I'm a huge fan of node.js and ve're actively using it
    in our project for creating various services.

    Recently we've faced a problem of significant ram overhead required to
    run each node.js instance (about 40Mb). This is an issue since we are
    running a number of different node.js services on one machine, and the
    memory is limited.

    So what we trying to use is run multiple services in single node.js
    process using vm.runInNewContext(), but compared this approach is missing
    some functionality:

    1) We cannot monitor amount of memory allocated by each subprocess.
    2) We cannot cancel all setTimeouts and process.nextTicks for given
    process to effectively stop its execution.
    3) We cannot get a list of open sockets and file descriptors for given
    process.

    With this functionality we could use node.js similar to erlang - run a
    number of processes inside a single vm - and use some erlang features (like
    'let it fail') that would certanly lead us to world domination ;)

    So, is there any projects targeted to solve similar problem?
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines:
    https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com

    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com

    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines:
    https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com

    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com

    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Ted Young at Oct 16, 2012 at 5:22 pm
    It was implemented, then removed. The instability and added complexity was determined to be not worth it at the time.

    Ted
    On Oct 16, 2012, at 7:20 AM, Alexey G wrote:

    Thats' right, but it's doesn't always create a separate process, sometimes multiple tabs run in single process (when they're from same domain and from same origin). Besides, chrome addons certanly live in same process, but are somehow isolated from each other and js in pages. I suppose they're created by single vm, memory overhead would be significant otherwise.

    Anyway, it must be possible to create multiple vm's in v8 and isolates feature seems to be what i'm looking for. It's not implemented, however, and i don't know if it will be. So for now i'll take a look on what can be done with domains.

    2012/10/16 Michal Kruk <kruczy@gmail.com>
    Actually chrome has multiple processes running, i think its creating one for every tab you have open


    On Mon, Oct 15, 2012 at 8:44 PM, Alexey Guskov wrote:
    By the way, i believe the way google chrome deals with multiple pages and browser addons is very similar to what i'm trying to achieve: isolated js vm instances inside a single process. If anyone knows chrome internals, please correct me.

    суббота, 13 октября 2012 г., 13:29:17 UTC+4 пользователь Alexey Guskov написал:
    Hello everybody. I'm a huge fan of node.js and ve're actively using it in our project for creating various services.

    Recently we've faced a problem of significant ram overhead required to run each node.js instance (about 40Mb). This is an issue since we are running a number of different node.js services on one machine, and the memory is limited.

    So what we trying to use is run multiple services in single node.js process using vm.runInNewContext(), but compared this approach is missing some functionality:

    1) We cannot monitor amount of memory allocated by each subprocess.
    2) We cannot cancel all setTimeouts and process.nextTicks for given process to effectively stop its execution.
    3) We cannot get a list of open sockets and file descriptors for given process.

    With this functionality we could use node.js similar to erlang - run a number of processes inside a single vm - and use some erlang features (like 'let it fail') that would certanly lead us to world domination ;)

    So, is there any projects targeted to solve similar problem?

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com

    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com

    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en


    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com

    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com

    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en


    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupnodejs @
categoriesnodejs
postedOct 13, '12 at 9:29a
activeOct 16, '12 at 5:22p
posts9
users6
websitenodejs.org
irc#node.js

People

Translate

site design / logo © 2022 Grokbase