Grokbase › Groups › Redis › redis-db › May 2012
FAQ
Question about master-Slave -

After Sync was completed, Master had 144,925,679 keys and slave had
127,124,218 keys. I wonder why didn’t SLAVE sync all the keys?


Thank you

Pawan Raina

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To view this discussion on the web visit https://groups.google.com/d/msg/redis-db/-/yLRwIvt4KckJ.
To post to this group, send email to redis-db@googlegroups.com.
To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.

Search Discussions

  • Josiah Carlson at May 11, 2012 at 8:16 pm ⇧
    That seems strange. Which version of Redis are you using? Do you have
    mismatched master/slave revisions? Can you give us more information?

    Regards,
    - Josiah
    On Thu, May 3, 2012 at 6:40 PM, Pawan Raina wrote:
    Question about master-Slave -

    After Sync was completed, Master had 144,925,679 keys and slave had
    127,124,218 keys. I wonder why didn’t SLAVE sync all the keys?


    Thank you

    Pawan Raina

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To view this discussion on the web visit
    https://groups.google.com/d/msg/redis-db/-/yLRwIvt4KckJ.
    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to
    redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/redis-db?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
  • Jeremy Zawodny at May 11, 2012 at 8:25 pm ⇧
    What about expired keys? The slave will discard already expired keys.

    Jeremy
    On Thu, May 3, 2012 at 6:40 PM, Pawan Raina wrote:

    Question about master-Slave -

    After Sync was completed, Master had 144,925,679 keys and slave had
    127,124,218 keys. I wonder why didn’t SLAVE sync all the keys? ****


    Thank you

    Pawan Raina

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To view this discussion on the web visit
    https://groups.google.com/d/msg/redis-db/-/yLRwIvt4KckJ.
    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to
    redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/redis-db?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
  • Salvatore Sanfilippo at May 11, 2012 at 8:36 pm ⇧

    On Fri, May 11, 2012 at 10:25 PM, Jeremy Zawodny wrote:

    What about expired keys?  The slave will discard already expired keys.
    I put my money on this :)

    Salvatore

    --
    Salvatore 'antirez' Sanfilippo
    open source developer - VMware
    http://invece.org

    Beauty is more important in computing than anywhere else in technology
    because software is so complicated. Beauty is the ultimate defence
    against complexity.
    — David Gelernter

    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
  • Jay A. Kreibich at May 11, 2012 at 8:51 pm ⇧

    On Fri, May 11, 2012 at 10:36:14PM +0200, Salvatore Sanfilippo scratched on the wall:
    On Fri, May 11, 2012 at 10:25 PM, Jeremy Zawodny wrote:

    What about expired keys? ?The slave will discard already expired keys.
    I put my money on this :)
    Wouldn't the master, upon reading the keys for replication, also
    delete them?

    -j

    --
    Jay A. Kreibich < J A Y @ K R E I B I.C H >

    "Intelligence is like underwear: it is important that you have it,
    but showing it to the wrong people has the tendency to make them
    feel uncomfortable." -- Angela Johnson

    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
  • Salvatore Sanfilippo at May 11, 2012 at 9:21 pm ⇧

    On Fri, May 11, 2012 at 10:50 PM, Jay A. Kreibich wrote:

    Wouldn't the master, upon reading the keys for replication, also
    delete them?
    Master -> Slave SYNC is performed via RDBs.
    RDBs are generated by child process that should touch as little pages
    as possible, and can't change parent's memory anyway, so there is no
    expiration process: simply keys that are found already expired are not
    persisted at all, of course.

    So a Master can have N keys in memory that are already expired but are
    still to be "collected", but the slave wil not have these keys as the
    RDB was purged.

    Salvatore

    --
    Salvatore 'antirez' Sanfilippo
    open source developer - VMware
    http://invece.org

    Beauty is more important in computing than anywhere else in technology
    because software is so complicated. Beauty is the ultimate defence
    against complexity.
    — David Gelernter

    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
  • Jay A. Kreibich at May 12, 2012 at 3:43 am ⇧

    On Fri, May 11, 2012 at 11:20:42PM +0200, Salvatore Sanfilippo scratched on the wall:
    On Fri, May 11, 2012 at 10:50 PM, Jay A. Kreibich wrote:

    Wouldn't the master, upon reading the keys for replication, also
    delete them?
    Master -> Slave SYNC is performed via RDBs.
    Oh, right... the initial sync is done from a forked copy. Got it.

    -j

    --
    Jay A. Kreibich < J A Y @ K R E I B I.C H >

    "Intelligence is like underwear: it is important that you have it,
    but showing it to the wrong people has the tendency to make them
    feel uncomfortable." -- Angela Johnson

    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
  • Pawan Raina at May 14, 2012 at 1:21 pm ⇧
    Thank you all for responding.



    I am using 2.2.12 on both master and slave. Even today I have 32 Million
    keys is Master and 28 Million in Slave. I don’t think I have 4 million that
    are expired.



    How can find number of already expired keys? Is there a command (or a
    combination of commands) that I can use.


    Pawan
    On Thursday, May 3, 2012 9:40:33 PM UTC-4, Pawan Raina wrote:

    Question about master-Slave -

    After Sync was completed, Master had 144,925,679 keys and slave had
    127,124,218 keys. I wonder why didn’t SLAVE sync all the keys?


    Thank you

    Pawan Raina
    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/redis-db/-/aHidA6sFKaEJ.
    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
  • Josiah Carlson at May 14, 2012 at 3:32 pm ⇧
    INFO will give you a list of information. One of them is the number of
    keys and the number of volatile keys per database. It will look
    something like: db0:keys=15,expires=1 .

    Regards,
    - Josiah
    On Mon, May 14, 2012 at 6:21 AM, Pawan Raina wrote:
    Thank you all for responding.



    I am using 2.2.12 on both master and slave. Even today I have 32 Million
    keys is Master and 28 Million in Slave. I don’t think I have 4 million that
    are expired.



    How can find number of already expired keys? Is there a command (or a
    combination of commands) that I can use.


    Pawan

    On Thursday, May 3, 2012 9:40:33 PM UTC-4, Pawan Raina wrote:

    Question about master-Slave -

    After Sync was completed, Master had 144,925,679 keys and slave had
    127,124,218 keys. I wonder why didn’t SLAVE sync all the keys?


    Thank you

    Pawan Raina
    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To view this discussion on the web visit
    https://groups.google.com/d/msg/redis-db/-/aHidA6sFKaEJ.

    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to
    redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/redis-db?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
  • Salvatore Sanfilippo at May 14, 2012 at 3:42 pm ⇧
    You may try the following:

    use RANDOMKEYS in the master, than check if the key exists or not in the slave.
    If you write a simple scripting doing this work you may easily find
    what the problem could be.

    Another note: You are using a pretty old version of Redis, I suggest
    upgrading ASAP to latest 2.4.x because there may be bug fixes or
    improvements that are only available in latest stable version.

    Cheers,
    Salvatore
    On Mon, May 14, 2012 at 3:21 PM, Pawan Raina wrote:
    Thank you all for responding.



    I am using 2.2.12 on both master and slave. Even today I have 32 Million
    keys is Master and 28 Million in Slave. I don’t think I have 4 million that
    are expired.



    How can find number of already expired keys? Is there a command (or a
    combination of commands) that I can use.


    Pawan

    On Thursday, May 3, 2012 9:40:33 PM UTC-4, Pawan Raina wrote:

    Question about master-Slave -

    After Sync was completed, Master had 144,925,679 keys and slave had
    127,124,218 keys. I wonder why didn’t SLAVE sync all the keys?


    Thank you

    Pawan Raina
    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To view this discussion on the web visit
    https://groups.google.com/d/msg/redis-db/-/aHidA6sFKaEJ.

    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to
    redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/redis-db?hl=en.


    --
    Salvatore 'antirez' Sanfilippo
    open source developer - VMware
    http://invece.org

    Beauty is more important in computing than anywhere else in technology
    because software is so complicated. Beauty is the ultimate defence
    against complexity.
    — David Gelernter

    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
  • Sripathi Krishnan at May 14, 2012 at 3:53 pm ⇧

    Use RANDOMKEYS in the master, than check if the key exists or not in the
    slave. If you write a simple scripting doing this work you may easily
    find what the problem could be.

    You can also use
    redis-rdb-tools<https://github.com/sripathikrishnan/redis-rdb-tools>
    to
    compare the dump.rdb files from master and slave.

    First, use the --command diff option on the dump files from master and
    slave, and pipe the output to standard sort utility

    ./rdb --command diff master/dump.rdb | sort > master.dump.txt
    ./rdb --command diff slave/dump.rdb | sort > slave.dump.txt

    Then, run your favourite diff program

    kdiff3 master.dump.txt slave.dump.txt

    To limit the size of the files, you can filter on keys using the
    --key=regex option
    --Sri

    On 14 May 2012 21:11, Salvatore Sanfilippo wrote:

    You may try the following:

    use RANDOMKEYS in the master, than check if the key exists or not in the
    slave.
    If you write a simple scripting doing this work you may easily find
    what the problem could be.

    Another note: You are using a pretty old version of Redis, I suggest
    upgrading ASAP to latest 2.4.x because there may be bug fixes or
    improvements that are only available in latest stable version.

    Cheers,
    Salvatore
    On Mon, May 14, 2012 at 3:21 PM, Pawan Raina wrote:
    Thank you all for responding.



    I am using 2.2.12 on both master and slave. Even today I have 32 Million
    keys is Master and 28 Million in Slave. I don’t think I have 4 million that
    are expired.



    How can find number of already expired keys? Is there a command (or a
    combination of commands) that I can use.


    Pawan

    On Thursday, May 3, 2012 9:40:33 PM UTC-4, Pawan Raina wrote:

    Question about master-Slave -

    After Sync was completed, Master had 144,925,679 keys and slave had
    127,124,218 keys. I wonder why didn’t SLAVE sync all the keys?


    Thank you

    Pawan Raina
    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To view this discussion on the web visit
    https://groups.google.com/d/msg/redis-db/-/aHidA6sFKaEJ.

    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to
    redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/redis-db?hl=en.


    --
    Salvatore 'antirez' Sanfilippo
    open source developer - VMware
    http://invece.org

    Beauty is more important in computing than anywhere else in technology
    because software is so complicated. Beauty is the ultimate defence
    against complexity.
    — David Gelernter

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to
    redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/redis-db?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
  • Yiftach Shoolman at May 14, 2012 at 4:00 pm ⇧
    AFAIK (but maybe I'm wrong) there is also a replication buff in the master
    for those keys that haven't yet forwarded to the slave. Maybe this buffer
    is just huge in your case due to master-slave link issues. Have you managed
    to look at it ?
    On Mon, May 14, 2012 at 6:52 PM, Sripathi Krishnan wrote:

    Use RANDOMKEYS in the master, than check if the key exists or not in the
    slave. If you write a simple scripting doing this work you may easily
    find what the problem could be.

    You can also use redis-rdb-tools<https://github.com/sripathikrishnan/redis-rdb-tools> to
    compare the dump.rdb files from master and slave.

    First, use the --command diff option on the dump files from master and
    slave, and pipe the output to standard sort utility

    ./rdb --command diff master/dump.rdb | sort > master.dump.txt
    ./rdb --command diff slave/dump.rdb | sort > slave.dump.txt

    Then, run your favourite diff program

    kdiff3 master.dump.txt slave.dump.txt

    To limit the size of the files, you can filter on keys using the
    --key=regex option
    --Sri


    On 14 May 2012 21:11, Salvatore Sanfilippo wrote:

    You may try the following:

    use RANDOMKEYS in the master, than check if the key exists or not in the
    slave.
    If you write a simple scripting doing this work you may easily find
    what the problem could be.

    Another note: You are using a pretty old version of Redis, I suggest
    upgrading ASAP to latest 2.4.x because there may be bug fixes or
    improvements that are only available in latest stable version.

    Cheers,
    Salvatore

    On Mon, May 14, 2012 at 3:21 PM, Pawan Raina <rainapawan@gmail.com>
    wrote:
    Thank you all for responding.



    I am using 2.2.12 on both master and slave. Even today I have 32 Million
    keys is Master and 28 Million in Slave. I don’t think I have 4 million that
    are expired.



    How can find number of already expired keys? Is there a command (or a
    combination of commands) that I can use.


    Pawan

    On Thursday, May 3, 2012 9:40:33 PM UTC-4, Pawan Raina wrote:

    Question about master-Slave -

    After Sync was completed, Master had 144,925,679 keys and slave had
    127,124,218 keys. I wonder why didn’t SLAVE sync all the keys?


    Thank you

    Pawan Raina
    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To view this discussion on the web visit
    https://groups.google.com/d/msg/redis-db/-/aHidA6sFKaEJ.

    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to
    redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/redis-db?hl=en.


    --
    Salvatore 'antirez' Sanfilippo
    open source developer - VMware
    http://invece.org

    Beauty is more important in computing than anywhere else in technology
    because software is so complicated. Beauty is the ultimate defence
    against complexity.
    — David Gelernter

    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to
    redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/redis-db?hl=en.
    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to
    redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/redis-db?hl=en.


    --
    Yiftach Shoolman
    +972-54-7634621

    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To post to this group, send email to redis-db@googlegroups.com.
    To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupredis-db @
categoriesredis
postedMay 4, '12 at 1:40a
activeMay 14, '12 at 4:00p
posts12
users7
websiteredis.io
irc#redis

7 users in discussion

Salvatore Sanfilippo: 3 posts Josiah Carlson: 2 posts Jay A. Kreibich: 2 posts Pawan Raina: 2 posts Jeremy Zawodny: 1 post Yiftach Shoolman: 1 post Sripathi Krishnan: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase