I am using rabbitmq-java-client-bin-2.7.1/rabbitmq-client.jar with
RabbitMQ 2.7.1 Server.

Just run this code

public static void main(String[] argv) throws IOException {
ConnectionFactory f = new ConnectionFactory();
Connection c = f.newConnection();
for(int i = 0; i < 20000; ++i) {
Channel ch = c.createChannel();
ch.close();
}
System.gc();
while(true) {}
}

While program is running, observe the memory usage, it keeps on
increasing
Get process id and do following
jmap -dump:format=b,file=yo.bin <pid>
jhat jhat -J-mx768m -stack false yo.bin
Once jhat brings up http server, go here
http://localhost:7000/showInstanceCounts/

20001 instances of class com.rabbitmq.client.impl.AMQCommand
20001 instances of class com.rabbitmq.client.impl.CommandAssembler
20000 instances of class com.rabbitmq.client.ShutdownSignalException
20000 instances of class com.rabbitmq.client.impl.AMQImpl$Channel
$Close
20000 instances of class com.rabbitmq.client.impl.ChannelN
20000 instances of class com.rabbitmq.client.impl.ConsumerDispatcher

In-spite of closing all the channels, not only channels but associated
objects (ShutdownSignalException, ChannelClose) continue to consume
memory.
For sure, knowing this now, I will use channelCreate call sparingly.

Is there a way to remedy this situation?
What is also worrying is instances of AMQCommand, so would this be the
case for each and every call of client library?

regards, Yogesh

Search Discussions

  • Steve Powell at Feb 1, 2012 at 10:42 am
    Yogesh,

    Thank you for this problem report. First point: each connection defines a
    (hidden) channel zero, so that explains why there are 20001 of some objects and
    20000 of others.

    As to why these are not garbage collected, we must be holding a reference
    somewhere (probably in the Connection).

    We will investigate. I have raised a bug (24723).

    Steve Powell (a happy bunny)
    ----------some more definitions from the SPD----------
    vermin (v.) Treating the dachshund for roundworm.
    chinchilla (n.) Cooling device for the lower jaw.
    socialcast (n.) Someone to whom everyone is speaking but nobody likes.
    On 31 Jan 2012, at 17:49, Yogesh Ketkar wrote:

    I am using rabbitmq-java-client-bin-2.7.1/rabbitmq-client.jar with
    RabbitMQ 2.7.1 Server.

    Just run this code

    public static void main(String[] argv) throws IOException {
    ConnectionFactory f = new ConnectionFactory();
    Connection c = f.newConnection();
    for(int i = 0; i < 20000; ++i) {
    Channel ch = c.createChannel();
    ch.close();
    }
    System.gc();
    while(true) {}
    }

    While program is running, observe the memory usage, it keeps on
    increasing
    Get process id and do following
    jmap -dump:format=b,file=yo.bin <pid>
    jhat jhat -J-mx768m -stack false yo.bin
    Once jhat brings up http server, go here
    http://localhost:7000/showInstanceCounts/

    20001 instances of class com.rabbitmq.client.impl.AMQCommand
    20001 instances of class com.rabbitmq.client.impl.CommandAssembler
    20000 instances of class com.rabbitmq.client.ShutdownSignalException
    20000 instances of class com.rabbitmq.client.impl.AMQImpl$Channel
    $Close
    20000 instances of class com.rabbitmq.client.impl.ChannelN
    20000 instances of class com.rabbitmq.client.impl.ConsumerDispatcher

    In-spite of closing all the channels, not only channels but associated
    objects (ShutdownSignalException, ChannelClose) continue to consume
    memory.
    For sure, knowing this now, I will use channelCreate call sparingly.

    Is there a way to remedy this situation?
    What is also worrying is instances of AMQCommand, so would this be the
    case for each and every call of client library?

    regards, Yogesh

    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Yogesh Ketkar at Feb 1, 2012 at 11:21 am
    Thanks Steve.
    Can you give me some idea about date of next Java Client release (with
    this bug-fix)?

    regards, Yogesh
    On Feb 1, 3:42?pm, Steve Powell wrote:
    Yogesh,

    Thank you for this problem report. First point: each connection defines a
    (hidden) channel zero, so that explains why there are 20001 of some objects and
    20000 of others.

    As to why these are not garbage collected, we must be holding a reference
    somewhere (probably in the Connection).

    We will investigate. I have raised a bug (24723).

    Steve Powell ?(a happy bunny)
    ----------some more definitions from the SPD----------
    vermin (v.) Treating the dachshund for roundworm.
    chinchilla (n.) Cooling device for the lower jaw.
    socialcast (n.) Someone to whom everyone is speaking but nobody likes.

    On 31 Jan 2012, at 17:49, Yogesh Ketkar wrote:








    I am using rabbitmq-java-client-bin-2.7.1/rabbitmq-client.jar with
    RabbitMQ 2.7.1 Server.
    Just run this code
    public static void main(String[] argv) throws IOException {
    ? ?ConnectionFactory f = new ConnectionFactory();
    ? ?Connection c = f.newConnection();
    ? ?for(int i = 0; i < 20000; ++i) {
    ? ? ? ?Channel ch = c.createChannel();
    ? ? ? ?ch.close();
    ? ?}
    ? ?System.gc();
    ? ?while(true) {}
    }
    While program is running, observe the memory usage, it keeps on
    increasing
    Get process id and do following
    jmap -dump:format=b,file=yo.bin ?<pid>
    jhat jhat -J-mx768m -stack false yo.bin
    Once jhat brings up http server, go here
    http://localhost:7000/showInstanceCounts/
    20001 instances of class com.rabbitmq.client.impl.AMQCommand
    20001 instances of class com.rabbitmq.client.impl.CommandAssembler
    20000 instances of class com.rabbitmq.client.ShutdownSignalException
    20000 instances of class com.rabbitmq.client.impl.AMQImpl$Channel
    $Close
    20000 instances of class com.rabbitmq.client.impl.ChannelN
    20000 instances of class com.rabbitmq.client.impl.ConsumerDispatcher
    In-spite of closing all the channels, not only channels but associated
    objects (ShutdownSignalException, ChannelClose) continue to consume
    memory.
    For sure, knowing this now, I will use channelCreate call sparingly.
    Is there a way to remedy this situation?
    What is also worrying is instances of AMQCommand, so would this be the
    case for each and every call of client library?
    regards, Yogesh
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Steve Powell at Feb 1, 2012 at 5:37 pm
    The fix (which we have coded) will be in the next release,
    but I don't know precisely when that will be.

    Steve Powell (a happy bunny)
    ----------some more definitions from the SPD----------
    vermin (v.) Treating the dachshund for roundworm.
    chinchilla (n.) Cooling device for the lower jaw.
    socialcast (n.) Someone to whom everyone is speaking but nobody likes.
    On 1 Feb 2012, at 11:21, Yogesh Ketkar wrote:

    Thanks Steve.
    Can you give me some idea about date of next Java Client release (with
    this bug-fix)?

    regards, Yogesh
    On Feb 1, 3:42 pm, Steve Powell wrote:
    Yogesh,

    Thank you for this problem report. First point: each connection defines a
    (hidden) channel zero, so that explains why there are 20001 of some objects and
    20000 of others.

    As to why these are not garbage collected, we must be holding a reference
    somewhere (probably in the Connection).

    We will investigate. I have raised a bug (24723).

    Steve Powell (a happy bunny)
    ----------some more definitions from the SPD----------
    vermin (v.) Treating the dachshund for roundworm.
    chinchilla (n.) Cooling device for the lower jaw.
    socialcast (n.) Someone to whom everyone is speaking but nobody likes.

    On 31 Jan 2012, at 17:49, Yogesh Ketkar wrote:








    I am using rabbitmq-java-client-bin-2.7.1/rabbitmq-client.jar with
    RabbitMQ 2.7.1 Server.
    Just run this code
    public static void main(String[] argv) throws IOException {
    ConnectionFactory f = new ConnectionFactory();
    Connection c = f.newConnection();
    for(int i = 0; i < 20000; ++i) {
    Channel ch = c.createChannel();
    ch.close();
    }
    System.gc();
    while(true) {}
    }
    While program is running, observe the memory usage, it keeps on
    increasing
    Get process id and do following
    jmap -dump:format=b,file=yo.bin <pid>
    jhat jhat -J-mx768m -stack false yo.bin
    Once jhat brings up http server, go here
    http://localhost:7000/showInstanceCounts/
    20001 instances of class com.rabbitmq.client.impl.AMQCommand
    20001 instances of class com.rabbitmq.client.impl.CommandAssembler
    20000 instances of class com.rabbitmq.client.ShutdownSignalException
    20000 instances of class com.rabbitmq.client.impl.AMQImpl$Channel
    $Close
    20000 instances of class com.rabbitmq.client.impl.ChannelN
    20000 instances of class com.rabbitmq.client.impl.ConsumerDispatcher
    In-spite of closing all the channels, not only channels but associated
    objects (ShutdownSignalException, ChannelClose) continue to consume
    memory.
    For sure, knowing this now, I will use channelCreate call sparingly.
    Is there a way to remedy this situation?
    What is also worrying is instances of AMQCommand, so would this be the
    case for each and every call of client library?
    regards, Yogesh
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Yogesh Ketkar at Feb 2, 2012 at 5:32 pm
    Hi Steve,

    Now that I am reusing the channels, memory problem with channels is
    not an issue, but there seems to be some more problem with memory
    leak.
    I had a program which pumped 100000 messages on a queue.

    Then this program kicked in and created certain queues on the fly and
    distributed these messages across them and
    eventually all the messages got consumed.

    Now when all queues in the system are empty, here is the object dump
    and I again see a lot of memory used by
    these objects.

    96449 instances of class com.rabbitmq.client.AMQP$BasicProperties
    96428 instances of class com.rabbitmq.client.Envelope
    96399 instances of class com.rabbitmq.client.impl.ConsumerDispatcher
    $4

    I created BasicProperties object like this, while routing the messages
    to appropriate queue.
    BasicProperties bp = new BasicProperties();
    bp.setTimestamp(Calendar.getInstance().getTime());
    bp.setType(MessageProperties.PERSISTENT_TEXT_PLAIN.toString());
    This was then used in basicPublish method.

    regards, Yogesh
    On Feb 1, 10:37?pm, Steve Powell wrote:
    The fix (which we have coded) will be in the next release,
    but I don't know precisely when that will be.

    Steve Powell ?(a happy bunny)
    ----------some more definitions from the SPD----------
    vermin (v.) Treating the dachshund for roundworm.
    chinchilla (n.) Cooling device for the lower jaw.
    socialcast (n.) Someone to whom everyone is speaking but nobody likes.

    On 1 Feb 2012, at 11:21, Yogesh Ketkar wrote:








    Thanks Steve.
    Can you give me some idea about date of next Java Client release (with
    this bug-fix)?
    regards, Yogesh
    On Feb 1, 3:42 pm, Steve Powell wrote:
    Yogesh,
    Thank you for this problem report. First point: each connection defines a
    (hidden) channel zero, so that explains why there are 20001 of some objects and
    20000 of others.
    As to why these are not garbage collected, we must be holding a reference
    somewhere (probably in the Connection).
    We will investigate. I have raised a bug (24723).
    Steve Powell ?(a happy bunny)
    ----------some more definitions from the SPD----------
    vermin (v.) Treating the dachshund for roundworm.
    chinchilla (n.) Cooling device for the lower jaw.
    socialcast (n.) Someone to whom everyone is speaking but nobody likes.
    On 31 Jan 2012, at 17:49, Yogesh Ketkar wrote:

    I am using rabbitmq-java-client-bin-2.7.1/rabbitmq-client.jar with
    RabbitMQ 2.7.1 Server.
    Just run this code
    public static void main(String[] argv) throws IOException {
    ? ?ConnectionFactory f = new ConnectionFactory();
    ? ?Connection c = f.newConnection();
    ? ?for(int i = 0; i < 20000; ++i) {
    ? ? ? ?Channel ch = c.createChannel();
    ? ? ? ?ch.close();
    ? ?}
    ? ?System.gc();
    ? ?while(true) {}
    }
    While program is running, observe the memory usage, it keeps on
    increasing
    Get process id and do following
    jmap -dump:format=b,file=yo.bin ?<pid>
    jhat jhat -J-mx768m -stack false yo.bin
    Once jhat brings up http server, go here
    http://localhost:7000/showInstanceCounts/
    20001 instances of class com.rabbitmq.client.impl.AMQCommand
    20001 instances of class com.rabbitmq.client.impl.CommandAssembler
    20000 instances of class com.rabbitmq.client.ShutdownSignalException
    20000 instances of class com.rabbitmq.client.impl.AMQImpl$Channel
    $Close
    20000 instances of class com.rabbitmq.client.impl.ChannelN
    20000 instances of class com.rabbitmq.client.impl.ConsumerDispatcher
    In-spite of closing all the channels, not only channels but associated
    objects (ShutdownSignalException, ChannelClose) continue to consume
    memory.
    For sure, knowing this now, I will use channelCreate call sparingly.
    Is there a way to remedy this situation?
    What is also worrying is instances of AMQCommand, so would this be the
    case for each and every call of client library?
    regards, Yogesh
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Steve Powell at Feb 13, 2012 at 3:47 pm
    Yogesh,
    Sorry not to get back to you sooner.

    I'm not at all sure what is causing your leak this time.

    Are you using QueueingConsumer? How are your messages being consumed?

    From the list of objects I am particularly interested in the ConsumerDispatcher
    objects.

    These ought to be created one per Channel, and can only be created when calling
    the createChannel() method on the Connection interface. So please can you check
    that you are not creating lots of channels.

    If it is possible, can you construct a small program to demonstrate the leak,
    like last time?

    Steve Powell (a happy bunny)
    ----------some more definitions from the SPD----------
    vermin (v.) Treating the dachshund for roundworm.
    chinchilla (n.) Cooling device for the lower jaw.
    socialcast (n.) Someone to whom everyone is speaking but nobody likes.
    On 2 Feb 2012, at 17:32, Yogesh Ketkar wrote:

    Hi Steve,

    Now that I am reusing the channels, memory problem with channels is
    not an issue, but there seems to be some more problem with memory
    leak.
    I had a program which pumped 100000 messages on a queue.

    Then this program kicked in and created certain queues on the fly and
    distributed these messages across them and
    eventually all the messages got consumed.

    Now when all queues in the system are empty, here is the object dump
    and I again see a lot of memory used by
    these objects.

    96449 instances of class com.rabbitmq.client.AMQP$BasicProperties
    96428 instances of class com.rabbitmq.client.Envelope
    96399 instances of class com.rabbitmq.client.impl.ConsumerDispatcher
    $4

    I created BasicProperties object like this, while routing the messages
    to appropriate queue.
    BasicProperties bp = new BasicProperties();
    bp.setTimestamp(Calendar.getInstance().getTime());
    bp.setType(MessageProperties.PERSISTENT_TEXT_PLAIN.toString());
    This was then used in basicPublish method.

    regards, Yogesh
    On Feb 1, 10:37 pm, Steve Powell wrote:
    The fix (which we have coded) will be in the next release,
    but I don't know precisely when that will be.

    Steve Powell (a happy bunny)
    ----------some more definitions from the SPD----------
    vermin (v.) Treating the dachshund for roundworm.
    chinchilla (n.) Cooling device for the lower jaw.
    socialcast (n.) Someone to whom everyone is speaking but nobody likes.

    On 1 Feb 2012, at 11:21, Yogesh Ketkar wrote:








    Thanks Steve.
    Can you give me some idea about date of next Java Client release (with
    this bug-fix)?
    regards, Yogesh
    On Feb 1, 3:42 pm, Steve Powell wrote:
    Yogesh,
    Thank you for this problem report. First point: each connection defines a
    (hidden) channel zero, so that explains why there are 20001 of some objects and
    20000 of others.
    As to why these are not garbage collected, we must be holding a reference
    somewhere (probably in the Connection).
    We will investigate. I have raised a bug (24723).
    Steve Powell (a happy bunny)
    ----------some more definitions from the SPD----------
    vermin (v.) Treating the dachshund for roundworm.
    chinchilla (n.) Cooling device for the lower jaw.
    socialcast (n.) Someone to whom everyone is speaking but nobody likes.
    On 31 Jan 2012, at 17:49, Yogesh Ketkar wrote:

    I am using rabbitmq-java-client-bin-2.7.1/rabbitmq-client.jar with
    RabbitMQ 2.7.1 Server.
    Just run this code
    public static void main(String[] argv) throws IOException {
    ConnectionFactory f = new ConnectionFactory();
    Connection c = f.newConnection();
    for(int i = 0; i < 20000; ++i) {
    Channel ch = c.createChannel();
    ch.close();
    }
    System.gc();
    while(true) {}
    }
    While program is running, observe the memory usage, it keeps on
    increasing
    Get process id and do following
    jmap -dump:format=b,file=yo.bin <pid>
    jhat jhat -J-mx768m -stack false yo.bin
    Once jhat brings up http server, go here
    http://localhost:7000/showInstanceCounts/
    20001 instances of class com.rabbitmq.client.impl.AMQCommand
    20001 instances of class com.rabbitmq.client.impl.CommandAssembler
    20000 instances of class com.rabbitmq.client.ShutdownSignalException
    20000 instances of class com.rabbitmq.client.impl.AMQImpl$Channel
    $Close
    20000 instances of class com.rabbitmq.client.impl.ChannelN
    20000 instances of class com.rabbitmq.client.impl.ConsumerDispatcher
    In-spite of closing all the channels, not only channels but associated
    objects (ShutdownSignalException, ChannelClose) continue to consume
    memory.
    For sure, knowing this now, I will use channelCreate call sparingly.
    Is there a way to remedy this situation?
    What is also worrying is instances of AMQCommand, so would this be the
    case for each and every call of client library?
    regards, Yogesh
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedJan 31, '12 at 5:49p
activeFeb 13, '12 at 3:47p
posts6
users2
websiterabbitmq.com
irc#rabbitmq

2 users in discussion

Steve Powell: 3 posts Yogesh Ketkar: 3 posts

People

Translate

site design / logo © 2022 Grokbase