FAQ
hi, we have some problems with http requests in node. We switched from the
module restler to request but the error still occurs and it seems to be a
node problem, not a problem of the module.

code:
var options = {
method: "GET",
uri: "http://192.168.71.30:8080/api",
json: true
};
var timeout = setTimeout(function() {
console.log("slow request");
}, 5000);
var begin = new Date();
request(options, function(error, response, body) {
var end = new Date();
clearTimeout(timeout);
if (error) {
console.log("error in request");
} else {
console.log("fast request");
}
});

The phenomena:

A healthy request looks like this in strace -e trace=network -v -p [pid]:
connect(26, {sa_family=AF_INET, sin_port=htons(8080),
sin_addr=inet_addr("192.168.71.30")}, 16) = -1 EINPROGRESS (Operation now
in progress)
getsockopt(26, SOL_SOCKET, SO_ERROR, [175508375352639488], [4]) = 0

A boken request does not appear at all in strace and a "slow request" is
logged.


So my question: Why does node do not open a socket and does not throw an
error?

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Search Discussions

  • Ben Noordhuis at Oct 20, 2012 at 12:33 pm

    On Fri, Oct 19, 2012 at 2:07 PM, Michael wrote:
    hi, we have some problems with http requests in node. We switched from the
    module restler to request but the error still occurs and it seems to be a
    node problem, not a problem of the module.

    code:
    var options = {
    method: "GET",
    uri: "http://192.168.71.30:8080/api",
    json: true
    };
    var timeout = setTimeout(function() {
    console.log("slow request");
    }, 5000);
    var begin = new Date();
    request(options, function(error, response, body) {
    var end = new Date();
    clearTimeout(timeout);
    if (error) {
    console.log("error in request");
    } else {
    console.log("fast request");
    }
    });

    The phenomena:

    A healthy request looks like this in strace -e trace=network -v -p [pid]:
    connect(26, {sa_family=AF_INET, sin_port=htons(8080),
    sin_addr=inet_addr("192.168.71.30")}, 16) = -1 EINPROGRESS (Operation now in
    progress)
    getsockopt(26, SOL_SOCKET, SO_ERROR, [175508375352639488], [4]) = 0

    A boken request does not appear at all in strace and a "slow request" is
    logged.


    So my question: Why does node do not open a socket
    It did, fd 26.
    and does not throw an error?
    EINPROGRESS is not a real error, it just means that the connection
    hasn't been fully established yet.

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Michael at Oct 21, 2012 at 7:32 am
    sry. this seems to me a misunderstanding. the strace output shows a working request.

    if the request hangs, there is nothing on strace. situatins appears by chance.

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Ben Noordhuis at Oct 21, 2012 at 8:49 pm

    On Sun, Oct 21, 2012 at 9:24 AM, Michael wrote:
    sry. this seems to me a misunderstanding. the strace output shows a working request.

    if the request hangs, there is nothing on strace. situatins appears by chance.
    Right. Do you see the same issue if you use the built-in http module?
    If so, can you post the full strace output, i.e. without the
    trace=network filter?

    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Michael at Oct 22, 2012 at 10:58 am
    ok. I will trace all if the problem happens again. We can not reproduce the
    failure at the moment, so we have top wait until it happens...

    Am Sonntag, 21. Oktober 2012 22:41:43 UTC+2 schrieb Ben Noordhuis:
    On Sun, Oct 21, 2012 at 9:24 AM, Michael wrote:
    sry. this seems to me a misunderstanding. the strace output shows a
    working request.
    if the request hangs, there is nothing on strace. situatins appears by
    chance.

    Right. Do you see the same issue if you use the built-in http module?
    If so, can you post the full strace output, i.e. without the
    trace=network filter?
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Aaron Seet at Dec 27, 2012 at 1:54 pm
    Unfortunately as a "me too" post, we have been seeing this random stalling
    behaviour for our Node server as well in recent weeks. The issue is
    described in the Restler issue log, since we use Restler on top of Node.

    https://github.com/danwrong/restler/issues/102

    In summary, we use Node (Windows) as a web socket server via Sockjs. The
    front-end part of accepting client connections and responding works fine.
    But backend HTTP requests it is supposed to make would cease to happen
    after a random period of time (couple of days to couple of hours) for some
    unknown reason. As in, absolutely no TCP socket opened on the network
    adapter to connect to backend server. Only workaround is to restart the
    node.exe process.

    Would appreciate advice on how to diagnose this problem; the unreliability
    is affecting us significantly. Happens be it v0.6 or v0.8.


    Thanks,
    Aaron

    On Monday, 22 October 2012 18:58:04 UTC+8, Michael wrote:

    ok. I will trace all if the problem happens again. We can not reproduce
    the failure at the moment, so we have top wait until it happens...
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Michael Wittig at Dec 27, 2012 at 2:01 pm
    Hi Aaron,

    I haven't found the solution yet. For the moment my system runs with http.globalAgent.maxSockets = 10000
    I haven't had the problem since then...:)

    Maybe this could help you?

    Am 27.12.2012 um 13:17 schrieb Aaron Seet <icelava@gmail.com>:
    Unfortunately as a "me too" post, we have been seeing this random stalling behaviour for our Node server as well in recent weeks. The issue is described in the Restler issue log, since we use Restler on top of Node.

    https://github.com/danwrong/restler/issues/102

    In summary, we use Node (Windows) as a web socket server via Sockjs. The front-end part of accepting client connections and responding works fine. But backend HTTP requests it is supposed to make would cease to happen after a random period of time (couple of days to couple of hours) for some unknown reason. As in, absolutely no TCP socket opened on the network adapter to connect to backend server. Only workaround is to restart the node.exe process.

    Would appreciate advice on how to diagnose this problem; the unreliability is affecting us significantly. Happens be it v0.6 or v0.8.


    Thanks,
    Aaron

    On Monday, 22 October 2012 18:58:04 UTC+8, Michael wrote:

    ok. I will trace all if the problem happens again. We can not reproduce the failure at the moment, so we have top wait until it happens...
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Aaron Seet at Dec 27, 2012 at 3:54 pm
    Thanks for the suggestion; i will try that tomorrow.

    However, netstat would not show any existing TCP connection to the backend
    server, so I'm not sure if providing a high number would somehow
    "encourage" Node to continue opening new connections. Do you notice growing
    memory leak with such a setting?


    thanks,
    Aaron
    On Thursday, 27 December 2012 22:01:24 UTC+8, Michael wrote:

    Hi Aaron,

    I haven't found the solution yet. For the moment my system runs with
    http.globalAgent.maxSockets = 10000
    I haven't had the problem since then...:)

    Maybe this could help you?
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Michael Wittig at Dec 28, 2012 at 8:14 am
    I can check memory comsumption after the holidays. in the meantime you could try to print the numbere of open sockets using http.globalAgent.sockets.length and http.globalAgent.requests.length to see if there is a growing number of requests.

    Am 27.12.2012 um 16:54 schrieb Aaron Seet <icelava@gmail.com>:
    Thanks for the suggestion; i will try that tomorrow.

    However, netstat would not show any existing TCP connection to the backend server, so I'm not sure if providing a high number would somehow "encourage" Node to continue opening new connections. Do you notice growing memory leak with such a setting?


    thanks,
    Aaron
    On Thursday, 27 December 2012 22:01:24 UTC+8, Michael wrote:

    Hi Aaron,

    I haven't found the solution yet. For the moment my system runs with http.globalAgent.maxSockets = 10000
    I haven't had the problem since then...:)

    Maybe this could help you?
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Aaron Seet at Dec 31, 2012 at 1:32 am
    Hmmm i tried to log http.globalAgent.sockets.length
    and http.globalAgent.requests.length and they came out as undefined.


    thanks,
    Aaron
    On Fri, Dec 28, 2012 at 4:14 PM, Michael Wittig wrote:

    I can check memory comsumption after the holidays. in the meantime you
    could try to print the numbere of open sockets using
    http.globalAgent.sockets.length and http.globalAgent.requests.length to see
    if there is a growing number of requests.
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Mikeal Rogers at Dec 31, 2012 at 1:35 am
    an agent holds sockets for a bunch of domains, it's not an array, it's an object, so that length check will never pass.
    On Dec 30, 2012, at December 30, 20125:32 PM, Aaron Seet wrote:

    Hmmm i tried to log http.globalAgent.sockets.length and http.globalAgent.requests.length and they came out as undefined.


    thanks,
    Aaron

    On Fri, Dec 28, 2012 at 4:14 PM, Michael Wittig wrote:
    I can check memory comsumption after the holidays. in the meantime you could try to print the numbere of open sockets using http.globalAgent.sockets.length and http.globalAgent.requests.length to see if there is a growing number of requests.


    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Aaron Seet at Dec 31, 2012 at 7:05 am
    Ah i used Object.keys(http.globalAgent.requests).length and it is just
    regularly 0 or 1 count.


    On Mon, Dec 31, 2012 at 9:32 AM, Aaron Seet wrote:

    Hmmm i tried to log http.globalAgent.sockets.length
    and http.globalAgent.requests.length and they came out as undefined.


    thanks,
    Aaron
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Aaron Seet at Dec 31, 2012 at 8:15 am
    Btw if I open up multiple tab windows and load up our web app + unit test
    pages to form say four separate clients, and have them continuously hit the
    socket server with data messages that require backend HTTP requests, I seem
    to able to rather quickly (in terms of less than an hour) cause the Node
    process to quit making new backend TCP connections.

    All these while the globalAgent's report is 1 socket and 0-1 requests,
    which doesn't seem like some socket leak. Also between testing the x86 and
    x64 versions, the memory consumption at point of stalling is approximately
    13MB and 25MB respectively.

    Wonder what else would cause ClientRequests to stop making connections
    without error?


    thanks,
    Aaron

    On Mon, Dec 31, 2012 at 3:05 PM, Aaron Seet wrote:

    Ah i used Object.keys(http.globalAgent.requests).length and it is just
    regularly 0 or 1 count.
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Aaron Seet at Jan 7, 2013 at 2:02 am
    In monitoring the socket server on my local workstation, i notice that
    despite the logging always showing only 1 socket in use, netstat reveals it
    quickly maxes out at 5 socket ESTABLISHED connections to the backend server
    (sometimes 6). This is odd because when i originally investigated this on
    the actual server itself, i saw absolutely no TCP sockets open (not even
    CLOSE_WAIT).

    But nonetheless, setting globalAgent's maxSocket to 10000 does allow the
    process to open heck a lot more connections to the backend. My colleagues
    have been using the socket server for the past week
    without re-occurrence of the problem.


    Much thanks,
    Aaron

    On Mon, Dec 31, 2012 at 4:15 PM, Aaron Seet wrote:

    Btw if I open up multiple tab windows and load up our web app + unit test
    pages to form say four separate clients, and have them continuously hit the
    socket server with data messages that require backend HTTP requests, I seem
    to able to rather quickly (in terms of less than an hour) cause the Node
    process to quit making new backend TCP connections.

    All these while the globalAgent's report is 1 socket and 0-1 requests,
    which doesn't seem like some socket leak. Also between testing the x86 and
    x64 versions, the memory consumption at point of stalling is approximately
    13MB and 25MB respectively.

    Wonder what else would cause ClientRequests to stop making connections
    without error?


    thanks,
    Aaron
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Murvin Lai at Jan 10, 2013 at 7:36 pm
    In linux, you may also need to increase the openFile number. Not sure for
    windows.
    On Sun, Jan 6, 2013 at 6:01 PM, Aaron Seet wrote:

    In monitoring the socket server on my local workstation, i notice that
    despite the logging always showing only 1 socket in use, netstat reveals it
    quickly maxes out at 5 socket ESTABLISHED connections to the backend server
    (sometimes 6). This is odd because when i originally investigated this on
    the actual server itself, i saw absolutely no TCP sockets open (not even
    CLOSE_WAIT).

    But nonetheless, setting globalAgent's maxSocket to 10000 does allow the
    process to open heck a lot more connections to the backend. My colleagues
    have been using the socket server for the past week
    without re-occurrence of the problem.


    Much thanks,
    Aaron

    On Mon, Dec 31, 2012 at 4:15 PM, Aaron Seet wrote:

    Btw if I open up multiple tab windows and load up our web app + unit test
    pages to form say four separate clients, and have them continuously hit the
    socket server with data messages that require backend HTTP requests, I seem
    to able to rather quickly (in terms of less than an hour) cause the Node
    process to quit making new backend TCP connections.

    All these while the globalAgent's report is 1 socket and 0-1 requests,
    which doesn't seem like some socket leak. Also between testing the x86 and
    x64 versions, the memory consumption at point of stalling is approximately
    13MB and 25MB respectively.

    Wonder what else would cause ClientRequests to stop making connections
    without error?


    thanks,
    Aaron
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines:
    https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en
  • Claude at Mar 6, 2013 at 12:46 pm
    Just wondering. Are you using node domains? After having the same issue,
    and a lot of digging, I found a problem with node domains and http (note:
    domains are still in Experimental state, so I guess that serves me right
    for trying to use them :( )
    On Friday, October 19, 2012 2:07:42 PM UTC+2, Michael wrote:

    hi, we have some problems with http requests in node. We switched from the
    module restler to request but the error still occurs and it seems to be a
    node problem, not a problem of the module.

    code:
    var options = {
    method: "GET",
    uri: "http://192.168.71.30:8080/api",
    json: true
    };
    var timeout = setTimeout(function() {
    console.log("slow request");
    }, 5000);
    var begin = new Date();
    request(options, function(error, response, body) {
    var end = new Date();
    clearTimeout(timeout);
    if (error) {
    console.log("error in request");
    } else {
    console.log("fast request");
    }
    });

    The phenomena:

    A healthy request looks like this in strace -e trace=network -v -p [pid]:
    connect(26, {sa_family=AF_INET, sin_port=htons(8080),
    sin_addr=inet_addr("192.168.71.30")}, 16) = -1 EINPROGRESS (Operation now
    in progress)
    getsockopt(26, SOL_SOCKET, SO_ERROR, [175508375352639488], [4]) = 0

    A boken request does not appear at all in strace and a "slow request" is
    logged.


    So my question: Why does node do not open a socket and does not throw an
    error?
    --
    --
    Job Board: http://jobs.nodejs.org/
    Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
    You received this message because you are subscribed to the Google
    Groups "nodejs" group.
    To post to this group, send email to nodejs@googlegroups.com
    To unsubscribe from this group, send email to
    nodejs+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/nodejs?hl=en?hl=en

    ---
    You received this message because you are subscribed to the Google Groups "nodejs" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupnodejs @
categoriesnodejs
postedOct 19, '12 at 12:07p
activeMar 6, '13 at 12:46p
posts16
users6
websitenodejs.org
irc#node.js

People

Translate

site design / logo © 2022 Grokbase