FAQ
Howdy,

we are running Redis 2.6.7 with maxmemory of 19G on several nodes. Recently
there was on two nodes a gain within 30 minutes from the used 9G up to
closely 19G.

It was not caused by more data that we stored. When 19G where nearly
reached redis started to free memory again. After another 30 minutes the
used memory was back to the normal 9G.

There were no log entries and redis was still answering but it was very
slow during the peak.

At the same time there was a networking problem. So during the peak some
clients could randomly not reach the two redis nodes or would get broken
connections.

How can this behaviorial be explained? Is there a known bug?

david

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
To post to this group, send email to redis-db@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Salvatore Sanfilippo at May 28, 2013 at 1:19 pm
    Hello, probably the additional memory was used by some misbehaving
    client or slave? To verify this you should check with CLIENT LIST if
    there are clients using ways too much memory. Next time it happens
    please save output of:

    INFO all
    CLIENT list

    investigating both we should be able to understand more.

    Thanks,
    Salvatore
    On Tue, May 28, 2013 at 2:34 PM, David wrote:
    Howdy,

    we are running Redis 2.6.7 with maxmemory of 19G on several nodes. Recently
    there was on two nodes a gain within 30 minutes from the used 9G up to
    closely 19G.

    It was not caused by more data that we stored. When 19G where nearly reached
    redis started to free memory again. After another 30 minutes the used memory
    was back to the normal 9G.

    There were no log entries and redis was still answering but it was very slow
    during the peak.

    At the same time there was a networking problem. So during the peak some
    clients could randomly not reach the two redis nodes or would get broken
    connections.

    How can this behaviorial be explained? Is there a known bug?

    david

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.


    --
    Salvatore 'antirez' Sanfilippo
    open source developer - GoPivotal
    http://invece.org

    Beauty is more important in computing than anywhere else in technology
    because software is so complicated. Beauty is the ultimate defence
    against complexity.
            — David Gelernter

    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Josiah Carlson at May 28, 2013 at 8:55 pm
    I would also point out that running a 'ps aux' will also let you take a
    quick snapshot of the actual memory use of the machine, as it is not
    impossible for your machine to have been in the process of a fork+sync
    operation. That could have caused memory doubling, and disk cache use to
    spike.

    Also... the 9G vs 19G, was that reported by some sort of monitoring
    solution for the platform (munin, nginx, etc.), or was it something that
    explicitly checks Redis' self-reported memory use?

      - Josiah

    On Tue, May 28, 2013 at 6:19 AM, Salvatore Sanfilippo wrote:

    Hello, probably the additional memory was used by some misbehaving
    client or slave? To verify this you should check with CLIENT LIST if
    there are clients using ways too much memory. Next time it happens
    please save output of:

    INFO all
    CLIENT list

    investigating both we should be able to understand more.

    Thanks,
    Salvatore
    On Tue, May 28, 2013 at 2:34 PM, David wrote:
    Howdy,

    we are running Redis 2.6.7 with maxmemory of 19G on several nodes. Recently
    there was on two nodes a gain within 30 minutes from the used 9G up to
    closely 19G.

    It was not caused by more data that we stored. When 19G where nearly reached
    redis started to free memory again. After another 30 minutes the used memory
    was back to the normal 9G.

    There were no log entries and redis was still answering but it was very slow
    during the peak.

    At the same time there was a networking problem. So during the peak some
    clients could randomly not reach the two redis nodes or would get broken
    connections.

    How can this behaviorial be explained? Is there a known bug?

    david

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.


    --
    Salvatore 'antirez' Sanfilippo
    open source developer - GoPivotal
    http://invece.org

    Beauty is more important in computing than anywhere else in technology
    because software is so complicated. Beauty is the ultimate defence
    against complexity.
    — David Gelernter

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • David at May 29, 2013 at 1:30 pm
    We saw the problems in munin, which checks the self reported memory
    statistics. Also checked manually the output of redis-cli info.

    Just had a similar problem on another instance running redis 2.4.12. Memory
    was rising for ~10 hours until it dropped down to ~93M within 1 second.
    Memory usage around 100M is expected.

    A slave with redis 2.6.10 had ~60M memory usage in the last 24 hours.


    2013/5/28 Josiah Carlson <josiah.carlson@gmail.com>
    I would also point out that running a 'ps aux' will also let you take a
    quick snapshot of the actual memory use of the machine, as it is not
    impossible for your machine to have been in the process of a fork+sync
    operation. That could have caused memory doubling, and disk cache use to
    spike.

    Also... the 9G vs 19G, was that reported by some sort of monitoring
    solution for the platform (munin, nginx, etc.), or was it something that
    explicitly checks Redis' self-reported memory use?

    - Josiah

    On Tue, May 28, 2013 at 6:19 AM, Salvatore Sanfilippo wrote:

    Hello, probably the additional memory was used by some misbehaving
    client or slave? To verify this you should check with CLIENT LIST if
    there are clients using ways too much memory. Next time it happens
    please save output of:

    INFO all
    CLIENT list

    investigating both we should be able to understand more.

    Thanks,
    Salvatore
    On Tue, May 28, 2013 at 2:34 PM, David wrote:
    Howdy,

    we are running Redis 2.6.7 with maxmemory of 19G on several nodes. Recently
    there was on two nodes a gain within 30 minutes from the used 9G up to
    closely 19G.

    It was not caused by more data that we stored. When 19G where nearly reached
    redis started to free memory again. After another 30 minutes the used memory
    was back to the normal 9G.

    There were no log entries and redis was still answering but it was very slow
    during the peak.

    At the same time there was a networking problem. So during the peak some
    clients could randomly not reach the two redis nodes or would get broken
    connections.

    How can this behaviorial be explained? Is there a known bug?

    david

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.


    --
    Salvatore 'antirez' Sanfilippo
    open source developer - GoPivotal
    http://invece.org

    Beauty is more important in computing than anywhere else in technology
    because software is so complicated. Beauty is the ultimate defence
    against complexity.
    — David Gelernter

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • David at May 29, 2013 at 3:16 pm
    Additionally in the same second when the memory was freed the total command
    count went up to 30k requests per second. Normally there are only 6-7k
    req/s.

    Is it possibly linked to a queue at a moment when there are many updates on
    the same key?

    david


    2013/5/29 David <edlerd@gmail.com>
    We saw the problems in munin, which checks the self reported memory
    statistics. Also checked manually the output of redis-cli info.

    Just had a similar problem on another instance running redis 2.4.12.
    Memory was rising for ~10 hours until it dropped down to ~93M within 1
    second. Memory usage around 100M is expected.

    A slave with redis 2.6.10 had ~60M memory usage in the last 24 hours.


    2013/5/28 Josiah Carlson <josiah.carlson@gmail.com>
    I would also point out that running a 'ps aux' will also let you take a
    quick snapshot of the actual memory use of the machine, as it is not
    impossible for your machine to have been in the process of a fork+sync
    operation. That could have caused memory doubling, and disk cache use to
    spike.

    Also... the 9G vs 19G, was that reported by some sort of monitoring
    solution for the platform (munin, nginx, etc.), or was it something that
    explicitly checks Redis' self-reported memory use?

    - Josiah

    On Tue, May 28, 2013 at 6:19 AM, Salvatore Sanfilippo wrote:

    Hello, probably the additional memory was used by some misbehaving
    client or slave? To verify this you should check with CLIENT LIST if
    there are clients using ways too much memory. Next time it happens
    please save output of:

    INFO all
    CLIENT list

    investigating both we should be able to understand more.

    Thanks,
    Salvatore
    On Tue, May 28, 2013 at 2:34 PM, David wrote:
    Howdy,

    we are running Redis 2.6.7 with maxmemory of 19G on several nodes. Recently
    there was on two nodes a gain within 30 minutes from the used 9G up to
    closely 19G.

    It was not caused by more data that we stored. When 19G where nearly reached
    redis started to free memory again. After another 30 minutes the used memory
    was back to the normal 9G.

    There were no log entries and redis was still answering but it was very slow
    during the peak.

    At the same time there was a networking problem. So during the peak some
    clients could randomly not reach the two redis nodes or would get broken
    connections.

    How can this behaviorial be explained? Is there a known bug?

    david

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.


    --
    Salvatore 'antirez' Sanfilippo
    open source developer - GoPivotal
    http://invece.org

    Beauty is more important in computing than anywhere else in technology
    because software is so complicated. Beauty is the ultimate defence
    against complexity.
    — David Gelernter

    --
    You received this message because you are subscribed to the Google
    Groups "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Josiah Carlson at May 29, 2013 at 3:30 pm
    How big is the data you are pushing onto your queues? Entry sizes? Number
    of elements? How many queue clients? Might it be possible that those
    clients were stopped for some reason? How long does it take to process
    queue items?

      - Josiah

    On Wed, May 29, 2013 at 8:16 AM, David wrote:

    Additionally in the same second when the memory was freed the total
    command count went up to 30k requests per second. Normally there are only
    6-7k req/s.

    Is it possibly linked to a queue at a moment when there are many updates
    on the same key?

    david


    2013/5/29 David <edlerd@gmail.com>
    We saw the problems in munin, which checks the self reported memory
    statistics. Also checked manually the output of redis-cli info.

    Just had a similar problem on another instance running redis 2.4.12.
    Memory was rising for ~10 hours until it dropped down to ~93M within 1
    second. Memory usage around 100M is expected.

    A slave with redis 2.6.10 had ~60M memory usage in the last 24 hours.


    2013/5/28 Josiah Carlson <josiah.carlson@gmail.com>
    I would also point out that running a 'ps aux' will also let you take a
    quick snapshot of the actual memory use of the machine, as it is not
    impossible for your machine to have been in the process of a fork+sync
    operation. That could have caused memory doubling, and disk cache use to
    spike.

    Also... the 9G vs 19G, was that reported by some sort of monitoring
    solution for the platform (munin, nginx, etc.), or was it something that
    explicitly checks Redis' self-reported memory use?

    - Josiah


    On Tue, May 28, 2013 at 6:19 AM, Salvatore Sanfilippo <antirez@gmail.com
    wrote:
    Hello, probably the additional memory was used by some misbehaving
    client or slave? To verify this you should check with CLIENT LIST if
    there are clients using ways too much memory. Next time it happens
    please save output of:

    INFO all
    CLIENT list

    investigating both we should be able to understand more.

    Thanks,
    Salvatore
    On Tue, May 28, 2013 at 2:34 PM, David wrote:
    Howdy,

    we are running Redis 2.6.7 with maxmemory of 19G on several nodes. Recently
    there was on two nodes a gain within 30 minutes from the used 9G up to
    closely 19G.

    It was not caused by more data that we stored. When 19G where nearly reached
    redis started to free memory again. After another 30 minutes the used memory
    was back to the normal 9G.

    There were no log entries and redis was still answering but it was very slow
    during the peak.

    At the same time there was a networking problem. So during the peak some
    clients could randomly not reach the two redis nodes or would get broken
    connections.

    How can this behaviorial be explained? Is there a known bug?

    david

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.


    --
    Salvatore 'antirez' Sanfilippo
    open source developer - GoPivotal
    http://invece.org

    Beauty is more important in computing than anywhere else in technology
    because software is so complicated. Beauty is the ultimate defence
    against complexity.
    — David Gelernter

    --
    You received this message because you are subscribed to the Google
    Groups "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google
    Groups "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • David at May 31, 2013 at 2:26 pm
    We will update the old redis instance to 2.6 soon, that is not so easy as
    just to restart it.

    The instance has
    ~ 300.000 Keys
    ~ 3000 clients
    Clients are stopped or restarted approximately every 4 hours.
    Data and Keys are usually under 50 characters long. There are some hashes
    with ~7000 entires which are very concurrently accessed.

    In the client list output all the entries have obl=0 and oll=0. Are there
    any other fields which could cause a memory consumption?


    2013/5/29 Josiah Carlson <josiah.carlson@gmail.com>
    How big is the data you are pushing onto your queues? Entry sizes? Number
    of elements? How many queue clients? Might it be possible that those
    clients were stopped for some reason? How long does it take to process
    queue items?

    - Josiah

    On Wed, May 29, 2013 at 8:16 AM, David wrote:

    Additionally in the same second when the memory was freed the total
    command count went up to 30k requests per second. Normally there are only
    6-7k req/s.

    Is it possibly linked to a queue at a moment when there are many updates
    on the same key?

    david


    2013/5/29 David <edlerd@gmail.com>
    We saw the problems in munin, which checks the self reported memory
    statistics. Also checked manually the output of redis-cli info.

    Just had a similar problem on another instance running redis 2.4.12.
    Memory was rising for ~10 hours until it dropped down to ~93M within 1
    second. Memory usage around 100M is expected.

    A slave with redis 2.6.10 had ~60M memory usage in the last 24 hours.


    2013/5/28 Josiah Carlson <josiah.carlson@gmail.com>
    I would also point out that running a 'ps aux' will also let you take a
    quick snapshot of the actual memory use of the machine, as it is not
    impossible for your machine to have been in the process of a fork+sync
    operation. That could have caused memory doubling, and disk cache use to
    spike.

    Also... the 9G vs 19G, was that reported by some sort of monitoring
    solution for the platform (munin, nginx, etc.), or was it something that
    explicitly checks Redis' self-reported memory use?

    - Josiah


    On Tue, May 28, 2013 at 6:19 AM, Salvatore Sanfilippo <
    antirez@gmail.com> wrote:
    Hello, probably the additional memory was used by some misbehaving
    client or slave? To verify this you should check with CLIENT LIST if
    there are clients using ways too much memory. Next time it happens
    please save output of:

    INFO all
    CLIENT list

    investigating both we should be able to understand more.

    Thanks,
    Salvatore
    On Tue, May 28, 2013 at 2:34 PM, David wrote:
    Howdy,

    we are running Redis 2.6.7 with maxmemory of 19G on several nodes. Recently
    there was on two nodes a gain within 30 minutes from the used 9G up to
    closely 19G.

    It was not caused by more data that we stored. When 19G where nearly reached
    redis started to free memory again. After another 30 minutes the
    used memory
    was back to the normal 9G.

    There were no log entries and redis was still answering but it was very slow
    during the peak.

    At the same time there was a networking problem. So during the peak some
    clients could randomly not reach the two redis nodes or would get broken
    connections.

    How can this behaviorial be explained? Is there a known bug?

    david

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.


    --
    Salvatore 'antirez' Sanfilippo
    open source developer - GoPivotal
    http://invece.org

    Beauty is more important in computing than anywhere else in technology
    because software is so complicated. Beauty is the ultimate defence
    against complexity.
    — David Gelernter

    --
    You received this message because you are subscribed to the Google
    Groups "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google
    Groups "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Josiah Carlson at May 29, 2013 at 3:28 pm
    Redis 2.4.12 doesn't have settings for maximum client buffer sizes. I've
    run into outgoing buffer size issues in production when network connections
    were not the best. Upgrade that 2.4.12 machine to 2.6 and update the slave
    buffer sizes. Also, you should add monitors for max client buffer size, and
    if possible, parse the output of "client list" to plot a few of the worst
    connections over time.

      - Josiah

    On Wed, May 29, 2013 at 6:30 AM, David wrote:

    We saw the problems in munin, which checks the self reported memory
    statistics. Also checked manually the output of redis-cli info.

    Just had a similar problem on another instance running redis 2.4.12.
    Memory was rising for ~10 hours until it dropped down to ~93M within 1
    second. Memory usage around 100M is expected.

    A slave with redis 2.6.10 had ~60M memory usage in the last 24 hours.


    2013/5/28 Josiah Carlson <josiah.carlson@gmail.com>
    I would also point out that running a 'ps aux' will also let you take a
    quick snapshot of the actual memory use of the machine, as it is not
    impossible for your machine to have been in the process of a fork+sync
    operation. That could have caused memory doubling, and disk cache use to
    spike.

    Also... the 9G vs 19G, was that reported by some sort of monitoring
    solution for the platform (munin, nginx, etc.), or was it something that
    explicitly checks Redis' self-reported memory use?

    - Josiah

    On Tue, May 28, 2013 at 6:19 AM, Salvatore Sanfilippo wrote:

    Hello, probably the additional memory was used by some misbehaving
    client or slave? To verify this you should check with CLIENT LIST if
    there are clients using ways too much memory. Next time it happens
    please save output of:

    INFO all
    CLIENT list

    investigating both we should be able to understand more.

    Thanks,
    Salvatore
    On Tue, May 28, 2013 at 2:34 PM, David wrote:
    Howdy,

    we are running Redis 2.6.7 with maxmemory of 19G on several nodes. Recently
    there was on two nodes a gain within 30 minutes from the used 9G up to
    closely 19G.

    It was not caused by more data that we stored. When 19G where nearly reached
    redis started to free memory again. After another 30 minutes the used memory
    was back to the normal 9G.

    There were no log entries and redis was still answering but it was very slow
    during the peak.

    At the same time there was a networking problem. So during the peak some
    clients could randomly not reach the two redis nodes or would get broken
    connections.

    How can this behaviorial be explained? Is there a known bug?

    david

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.


    --
    Salvatore 'antirez' Sanfilippo
    open source developer - GoPivotal
    http://invece.org

    Beauty is more important in computing than anywhere else in technology
    because software is so complicated. Beauty is the ultimate defence
    against complexity.
    — David Gelernter

    --
    You received this message because you are subscribed to the Google
    Groups "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupredis-db @
categoriesredis
postedMay 28, '13 at 12:51p
activeMay 31, '13 at 2:26p
posts8
users3
websiteredis.io
irc#redis

People

Translate

site design / logo © 2022 Grokbase