FAQ
Hi all,

I've got a couple of long-running functions that use BigIntegers and the
like for some financial calculations. These take a few seconds to execute.
Now a few seconds of CPU on the main node thread means all other activity
grinds to a halt. What's the easiest way to call these functions
asynchronously? ie, on another thread?

Should I be making a module? Is it possible without a module? Should I be
using child_process?

Any pointers or tips appreciated.
Thank you.

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Search Discussions

  • Oleg Efimov (Sannis) at Apr 9, 2012 at 10:45 am
    Yes, you can execute them in child_process. Also where may be already
    existing modules, try look for maintained forks of "webworkers" or
    "backgrounder".
    Also you can use queues: kue or geaman-node

    понедельник, 9 апреля 2012 г., 14:34:05 UTC+4 пользователь Jeremy Rudd
    написал:
    Hi all,

    I've got a couple of long-running functions that use BigIntegers and the
    like for some financial calculations. These take a few seconds to execute.
    Now a few seconds of CPU on the main node thread means all other activity
    grinds to a halt. What's the easiest way to call these functions
    asynchronously? ie, on another thread?

    Should I be making a module? Is it possible without a module? Should I be
    using child_process?

    Any pointers or tips appreciated.
    Thank you.
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Jeremy Rudd at Apr 9, 2012 at 11:18 am
    I've gone through what you said, and here are my results:

    - Child_process is slow to startup (30 ms) and consumes 10 MB base since
    its a whole new V8 instance. Maybe if these are started and not stopped
    and used alongside a job queue it could work. But it cannot work like
    "async function calls" where you start one for each small calc you need
    to do.

    - WebWorkers seem to be an HTML5 integration API, and extra bloat is not
    needed

    - Cannot find anything like "backgrounder"

    - Gearman-node seems more suited to longer running jobs, and not 1 sec
    tasks.

    - Resque seems to be a reliable queue, should one be needed. --
    https://github.com/blog/542-introducing-resque

    Currently I'll just run my calc on the main CPU and see what happens.
    Any ideas anybody?

    - Should I use gearman?
    - Or should I go with child_process since I currently have the task code
    also written in NodeJS, and not C++.

    On 4/9/2012 4:15 PM, Oleg Efimov (Sannis) wrote:
    Yes, you can execute them in child_process. Also where may be already
    existing modules, try look for maintained forks of "webworkers" or
    "backgrounder".
    Also you can use queues: kue or geaman-node
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Oliver Leics at Apr 9, 2012 at 12:01 pm
    Maybe this works for you: https://github.com/xk/node-threads-a-gogo
    On Mon, Apr 9, 2012 at 1:18 PM, Jeremy Rudd wrote:
    I've gone through what you said, and here are my results:

    - Child_process is slow to startup (30 ms) and consumes 10 MB base since its
    a whole new V8 instance. Maybe if these are started and not stopped and used
    alongside a job queue it could work. But it cannot work like "async function
    calls" where you start one for each small calc you need to do.

    - WebWorkers seem to be an HTML5 integration API, and extra bloat is not
    needed

    - Cannot find anything like "backgrounder"

    - Gearman-node seems more suited to longer running jobs, and not 1 sec
    tasks.

    - Resque seems to be a reliable queue, should one be needed. --
    https://github.com/blog/542-introducing-resque

    Currently I'll just run my calc on the main CPU and see what happens. Any
    ideas anybody?

    - Should I use gearman?
    - Or should I go with child_process since I currently have the task code
    also written in NodeJS, and not C++.


    On 4/9/2012 4:15 PM, Oleg Efimov (Sannis) wrote:

    Yes, you can execute them in child_process. Also where may be already
    existing modules, try look for maintained forks of "webworkers" or
    "backgrounder".
    Also you can use queues: kue or geaman-node

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines:
    https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en


    --
    Oliver Leics @ G+
    https://plus.google.com/112912441146721682527

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Roly Fentanes at Apr 9, 2012 at 12:22 pm
    You could use just one child process that you spawn at the start of your
    app. Then every time you need to do the calculation, you tell it to. And it
    will get back to you when it's finished.

    Would only work there's little chance for many calculations to be needed at
    once. Or you could spawn a few more workers as needed.
    On Monday, April 9, 2012 3:34:05 AM UTC-7, Jeremy Rudd wrote:

    Hi all,

    I've got a couple of long-running functions that use BigIntegers and the
    like for some financial calculations. These take a few seconds to execute.
    Now a few seconds of CPU on the main node thread means all other activity
    grinds to a halt. What's the easiest way to call these functions
    asynchronously? ie, on another thread?

    Should I be making a module? Is it possible without a module? Should I be
    using child_process?

    Any pointers or tips appreciated.
    Thank you.
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Shin Suzuki at Apr 9, 2012 at 3:08 pm
    Hi,

    Here I wrote a simple module to make functions asynchronous.

    https://github.com/shinout/standalone


    With this, synchronous methods can be called by the same method
    asynchonously.
    Also this module can be available in browsers (using WebWorker API).

    Thanks.



    2012/4/9 Roly Fentanes <roly426@gmail.com>
    You could use just one child process that you spawn at the start of your
    app. Then every time you need to do the calculation, you tell it to. And it
    will get back to you when it's finished.

    Would only work there's little chance for many calculations to be needed
    at once. Or you could spawn a few more workers as needed.

    On Monday, April 9, 2012 3:34:05 AM UTC-7, Jeremy Rudd wrote:

    Hi all,

    I've got a couple of long-running functions that use BigIntegers and the
    like for some financial calculations. These take a few seconds to execute.
    Now a few seconds of CPU on the main node thread means all other activity
    grinds to a halt. What's the easiest way to call these functions
    asynchronously? ie, on another thread?

    Should I be making a module? Is it possible without a module? Should I be
    using child_process?

    Any pointers or tips appreciated.
    Thank you.
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines:
    https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Ilya Dmitrichenko at Apr 9, 2012 at 3:47 pm
    You should definitely also have a look at Fabric Engine:
    http://fabric-engine.com/

    It is specifically designed for the sort problems :)
    On 9 April 2012 16:08, Shin Suzuki wrote:
    Hi,

    Here I wrote a simple module to make functions asynchronous.

    https://github.com/shinout/standalone


    With this, synchronous methods can be called by the same method
    asynchonously.
    Also this module can be available in browsers (using WebWorker API).

    Thanks.



    2012/4/9 Roly Fentanes <roly426@gmail.com>
    You could use just one child process that you spawn at the start of your
    app. Then every time you need to do the calculation, you tell it to. And it
    will get back to you when it's finished.

    Would only work there's little chance for many calculations to be needed
    at once. Or you could spawn a few more workers as needed.

    On Monday, April 9, 2012 3:34:05 AM UTC-7, Jeremy Rudd wrote:

    Hi all,

    I've got a couple of long-running functions that use BigIntegers and the
    like for some financial calculations. These take a few seconds to execute.
    Now a few seconds of CPU on the main node thread means all other activity
    grinds to a halt. What's the easiest way to call these functions
    asynchronously? ie, on another thread?

    Should I be making a module? Is it possible without a module? Should I be
    using child_process?

    Any pointers or tips appreciated.
    Thank you.
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines:
    https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines:
    https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Oleg Efimov (Sannis) at Apr 9, 2012 at 4:24 pm
    Cool, I've save it for future use.

    понедельник, 9 апреля 2012 г., 19:08:06 UTC+4 пользователь shinout написал:
    Hi,

    Here I wrote a simple module to make functions asynchronous.

    https://github.com/shinout/standalone


    With this, synchronous methods can be called by the same method
    asynchonously.
    Also this module can be available in browsers (using WebWorker API).

    Thanks.



    2012/4/9 Roly Fentanes <roly426@gmail.com>
    You could use just one child process that you spawn at the start of your
    app. Then every time you need to do the calculation, you tell it to. And it
    will get back to you when it's finished.

    Would only work there's little chance for many calculations to be needed
    at once. Or you could spawn a few more workers as needed.

    On Monday, April 9, 2012 3:34:05 AM UTC-7, Jeremy Rudd wrote:

    Hi all,

    I've got a couple of long-running functions that use BigIntegers and the
    like for some financial calculations. These take a few seconds to execute.
    Now a few seconds of CPU on the main node thread means all other activity
    grinds to a halt. What's the easiest way to call these functions
    asynchronously? ie, on another thread?

    Should I be making a module? Is it possible without a module? Should I
    be using child_process?

    Any pointers or tips appreciated.
    Thank you.
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines:
    https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Bruno Jouhier at Apr 9, 2012 at 4:26 pm
    Child_process will do the job but threads_a_gogo will do it too, just faster and with less memory overhead because it uses threads rather than processes.

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Jorge at Apr 10, 2012 at 12:40 pm

    On Apr 9, 2012, at 12:34 PM, Jeremy Rudd wrote:

    Hi all,

    I've got a couple of long-running functions that use BigIntegers and the like for some financial calculations. These take a few seconds to execute. Now a few seconds of CPU on the main node thread means all other activity grinds to a halt. What's the easiest way to call these functions asynchronously? ie, on another thread?

    Should I be making a module? Is it possible without a module? Should I be using child_process?

    Any pointers or tips appreciated.
    Thank you.
    That's exactly what threads_a_gogo is for:

    <https://github.com/xk/node-threads-a-gogo>

    It lets you run these calculations in parallel, in javascript, in threads, and using all the available CPU cores. Then the main thread will receive the results in a callback or via an event.
    --
    Jorge.

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Jeremy Rudd at Apr 11, 2012 at 7:40 am
    Hi all,

    threads_a_gogo sounds cool, but what are the technical details of using it?

    1. What does it use in the back-end to schedule / manage tasks? child_process?
    2. child_process takes some 30 ms to spawn a child thread. How much time does GoGo take?
    3. How does it internally work? Are there constantly running threads + scheduler? Or does it spawn a new thread per task?
    4. Is it recommended for 1-10 ms tasks as well? Do you have any idea / stats to show performance with / without GoGo?

    Thanks,
    Jeremy Rudd
    On 4/10/2012 6:09 PM, Jorge wrote:
    That's exactly what threads_a_gogo is for:

    <https://github.com/xk/node-threads-a-gogo>

    It lets you run these calculations in parallel, in javascript, in threads, and using all the available CPU cores. Then the main thread will receive the results in a callback or via an event.
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Jorge at Apr 11, 2012 at 9:35 am

    On Apr 11, 2012, at 9:40 AM, Jeremy Rudd wrote:
    On 4/10/2012 6:09 PM, Jorge wrote:
    That's exactly what threads_a_gogo is for:

    <https://github.com/xk/node-threads-a-gogo>

    It lets you run these calculations in parallel, in javascript, in threads, and using all the available CPU cores. Then the main thread will receive the results in a callback or via an event.
    Hi all,
    Hi Jeremy,
    threads_a_gogo sounds cool, but what are the technical details of using it?

    1. What does it use in the back-end to schedule / manage tasks? child_process?
    It creates p-threads and runs v8 isolates in them.
    2. child_process takes some 30 ms to spawn a child thread. How much time does GoGo take?
    When I run this:

    $ time node -e "require('threads_a_gogo').createPool(100).destroy()"

    I get:

    real 0m0.387s
    user 0m0.390s
    sys 0m0.286s

    That's 3.87 ms per thread

    But note that these 0.387s include as well the time to start node, the time to require('threads_a_gogo'), and the time to destroy the 100 threads, so it must be a bit less than 3.87 ms per thread.
    3. How does it internally work? Are there constantly running threads + scheduler? Or does it spawn a new thread per task?
    You could create/destroy the threads on demand, but I would rather create a thread pool of twice or 3x the number of cores and install in them only once the functions that do the calculations, then reuse them again and again via a pool.any.eval(program, cb). Each thread takes only ~2MB, and when idle it uses exactly 0% cpu.
    4. Is it recommended for 1-10 ms tasks as well? Do you have any idea / stats to show performance with / without GoGo?
    Threads_a_gogo lets you

    -run these blocking calls in parallel so
    -they won't block node's main thread
    -you'll be exploiting all the cpu cores in the machine

    With an n cores machine, you'll be serving results n times faster, but even in a single core machine, you should use threads_a_gogo simply to avoid blocking node.

    WRT to performance, a fibonacci(40) in a thread_a_gogo runs about twice as fast as in node's main thread, see: <https://gist.github.com/2018811> it's also a good example because it's very similar to your use case, istm.
    Thanks,
    Jeremy Rudd
    Cheers,
    --
    Jorge.

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Jeremy Rudd at Apr 12, 2012 at 6:16 am
    Thanks Jorge!

    Fantastic information, it sounds like threads-a-gogo is the way to go.
    I'll use it and let you all know of my results.

    Thanks again!
    Jeremy
    On 4/11/2012 3:05 PM, Jorge wrote:
    On Apr 11, 2012, at 9:40 AM, Jeremy Rudd wrote:
    On 4/10/2012 6:09 PM, Jorge wrote:
    That's exactly what threads_a_gogo is for:

    <https://github.com/xk/node-threads-a-gogo>

    It lets you run these calculations in parallel, in javascript, in threads, and using all the available CPU cores. Then the main thread will receive the results in a callback or via an event.
    Hi all,
    Hi Jeremy,
    threads_a_gogo sounds cool, but what are the technical details of using it?

    1. What does it use in the back-end to schedule / manage tasks? child_process?
    It creates p-threads and runs v8 isolates in them.
    2. child_process takes some 30 ms to spawn a child thread. How much time does GoGo take?
    When I run this:

    $ time node -e "require('threads_a_gogo').createPool(100).destroy()"

    I get:

    real 0m0.387s
    user 0m0.390s
    sys 0m0.286s

    That's 3.87 ms per thread

    But note that these 0.387s include as well the time to start node, the time to require('threads_a_gogo'), and the time to destroy the 100 threads, so it must be a bit less than 3.87 ms per thread.
    3. How does it internally work? Are there constantly running threads + scheduler? Or does it spawn a new thread per task?
    You could create/destroy the threads on demand, but I would rather create a thread pool of twice or 3x the number of cores and install in them only once the functions that do the calculations, then reuse them again and again via a pool.any.eval(program, cb). Each thread takes only ~2MB, and when idle it uses exactly 0% cpu.
    4. Is it recommended for 1-10 ms tasks as well? Do you have any idea / stats to show performance with / without GoGo?
    Threads_a_gogo lets you

    -run these blocking calls in parallel so
    -they won't block node's main thread
    -you'll be exploiting all the cpu cores in the machine

    With an n cores machine, you'll be serving results n times faster, but even in a single core machine, you should use threads_a_gogo simply to avoid blocking node.

    WRT to performance, a fibonacci(40) in a thread_a_gogo runs about twice as fast as in node's main thread, see:<https://gist.github.com/2018811> it's also a good example because it's very similar to your use case, istm.
    Thanks,
    Jeremy Rudd
    Cheers,
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Harsh Dev at Aug 6, 2012 at 9:23 pm
    Hi Jeremy,
    I've been looking into TAGG as well so if possible, could you give us an
    overview of what your results are/were?

    Thanks,
    Harsh
    On Wednesday, April 11, 2012 11:15:50 PM UTC-7, Jeremy Rudd wrote:

    Thanks Jorge!

    Fantastic information, it sounds like threads-a-gogo is the way to go.
    I'll use it and let you all know of my results.

    Thanks again!
    Jeremy
    On 4/11/2012 3:05 PM, Jorge wrote:
    On Apr 11, 2012, at 9:40 AM, Jeremy Rudd wrote:
    On 4/10/2012 6:09 PM, Jorge wrote:
    That's exactly what threads_a_gogo is for:

    <https://github.com/xk/node-threads-a-gogo>

    It lets you run these calculations in parallel, in javascript, in
    threads, and using all the available CPU cores. Then the main thread will
    receive the results in a callback or via an event.
    Hi all,
    Hi Jeremy,
    threads_a_gogo sounds cool, but what are the technical details of using
    it?
    1. What does it use in the back-end to schedule / manage tasks?
    child_process?
    It creates p-threads and runs v8 isolates in them.
    2. child_process takes some 30 ms to spawn a child thread. How much
    time does GoGo take?
    When I run this:

    $ time node -e "require('threads_a_gogo').createPool(100).destroy()"

    I get:

    real 0m0.387s
    user 0m0.390s
    sys 0m0.286s

    That's 3.87 ms per thread

    But note that these 0.387s include as well the time to start node, the
    time to require('threads_a_gogo'), and the time to destroy the 100 threads,
    so it must be a bit less than 3.87 ms per thread.
    3. How does it internally work? Are there constantly running threads +
    scheduler? Or does it spawn a new thread per task?
    You could create/destroy the threads on demand, but I would rather
    create a thread pool of twice or 3x the number of cores and install in them
    only once the functions that do the calculations, then reuse them again and
    again via a pool.any.eval(program, cb). Each thread takes only ~2MB, and
    when idle it uses exactly 0% cpu.
    4. Is it recommended for 1-10 ms tasks as well? Do you have any idea /
    stats to show performance with / without GoGo?
    Threads_a_gogo lets you

    -run these blocking calls in parallel so
    -they won't block node's main thread
    -you'll be exploiting all the cpu cores in the machine

    With an n cores machine, you'll be serving results n times faster, but
    even in a single core machine, you should use threads_a_gogo simply to
    avoid blocking node.
    WRT to performance, a fibonacci(40) in a thread_a_gogo runs about twice
    as fast as in node's main thread, see:<https://gist.github.com/2018811>
    it's also a good example because it's very similar to your use case, istm.
    Thanks,
    Jeremy Rudd
    Cheers,
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Jeremy Rudd at Aug 7, 2012 at 9:28 am
    Hi Harsh,

    I did not have the time/need to try that one out. "threads_a_gogo"
    sounded the most useful though.

    Jeremy
    On 8/7/2012 2:49 AM, Harsh Dev wrote:
    Hi Jeremy,
    I've been looking into TAGG as well so if possible, could you give us
    an overview of what your results are/were?

    Thanks,
    Harsh

    On Wednesday, April 11, 2012 11:15:50 PM UTC-7, Jeremy Rudd wrote:

    Thanks Jorge!

    Fantastic information, it sounds like threads-a-gogo is the way to
    go.
    I'll use it and let you all know of my results.

    Thanks again!
    Jeremy
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Greelgorke at Aug 7, 2012 at 12:30 pm
    If you can split your calculation to partials, like recursive approach, you
    can use process.nextTick to execute the partial asynchronously. Check out
    this example https://gist.github.com/3284767



    Am Dienstag, 7. August 2012 11:28:23 UTC+2 schrieb Jeremy Rudd:
    Hi Harsh,

    I did not have the time/need to try that one out. "threads_a_gogo" sounded
    the most useful though.

    Jeremy

    On 8/7/2012 2:49 AM, Harsh Dev wrote:

    Hi Jeremy,
    I've been looking into TAGG as well so if possible, could you give us an
    overview of what your results are/were?

    Thanks,
    Harsh
    On Wednesday, April 11, 2012 11:15:50 PM UTC-7, Jeremy Rudd wrote:

    Thanks Jorge!

    Fantastic information, it sounds like threads-a-gogo is the way to go.
    I'll use it and let you all know of my results.

    Thanks again!
    Jeremy
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • George Stagas at Aug 7, 2012 at 1:04 pm
    https://github.com/stagas/bitch allows you to define a function that
    is forked away in a new process and allows you to transparently call
    it whenever you want and kill it when you don't need it any more.

    2012/8/7 greelgorke <g.elke@wowbiz.de>:
    If you can split your calculation to partials, like recursive approach, you
    can use process.nextTick to execute the partial asynchronously. Check out
    this example https://gist.github.com/3284767



    Am Dienstag, 7. August 2012 11:28:23 UTC+2 schrieb Jeremy Rudd:
    Hi Harsh,

    I did not have the time/need to try that one out. "threads_a_gogo" sounded
    the most useful though.

    Jeremy

    On 8/7/2012 2:49 AM, Harsh Dev wrote:

    Hi Jeremy,
    I've been looking into TAGG as well so if possible, could you give us an
    overview of what your results are/were?

    Thanks,
    Harsh
    On Wednesday, April 11, 2012 11:15:50 PM UTC-7, Jeremy Rudd wrote:

    Thanks Jorge!

    Fantastic information, it sounds like threads-a-gogo is the way to go.
    I'll use it and let you all know of my results.

    Thanks again!
    Jeremy
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines:
    https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupnodejs @
categoriesnodejs
postedApr 9, '12 at 10:34a
activeAug 7, '12 at 1:04p
posts17
users11
websitenodejs.org
irc#node.js

People

Translate

site design / logo © 2022 Grokbase