FAQ
Redis looks like a great solution for server-side web session data. Here's
what I'd like to do: store everything in memory and expire it with LRU (or
if that's not possible, with a TTL) and *when it expires*, put the content
into a database. I don't want to write data to the database until it
expires in Redis. If someone later tries to retrieve one of these
now-expired (and thus no longer in memory) keys, they can go to the
database, retrieve the value and invoke `setnx` and now their session is in
memory and recently used.

How can I get this "expire to database" functionality? If I have to do it
manually by finding old keys, what's the best way to find them? The first
approach I thought of is labor-intensive: on every session update, update
the score of a sorted set, then periodically grab keys N through the end
and atomically (somehow -- how is unclear because they're two separate
programs -- the DB and Redis) unset them and store their values in the
database. Not great. Any better ideas?



--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
To post to this group, send email to redis-db@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Josiah Carlson at May 13, 2013 at 4:57 am
    There is a method of subscribing to know about when a key is expiring in
    2.8, but it doesn't give you the data, it just tells you "this data is now
    gone".

    Using a ZSET is the only way you can know when data is about to expire so
    that you can manually pull it out.

    In terms of getting data out of Redis and into your database, you can use a
    Lua script to check your ZSET, pull your data from another structure, then
    remove it from Redis - returning it for database update.


    When you know you need to read the data from your database, acquire a lock
    for that specific data (in Redis or another distributed platform for such
    things), then block your clients waiting for the data to be read from your
    DB and written into Redis.

    - Josiah

    On Sat, May 11, 2013 at 6:21 AM, Charlie Brown wrote:

    Redis looks like a great solution for server-side web session data. Here's
    what I'd like to do: store everything in memory and expire it with LRU (or
    if that's not possible, with a TTL) and *when it expires*, put the content
    into a database. I don't want to write data to the database until it
    expires in Redis. If someone later tries to retrieve one of these
    now-expired (and thus no longer in memory) keys, they can go to the
    database, retrieve the value and invoke `setnx` and now their session is in
    memory and recently used.

    How can I get this "expire to database" functionality? If I have to do it
    manually by finding old keys, what's the best way to find them? The first
    approach I thought of is labor-intensive: on every session update, update
    the score of a sorted set, then periodically grab keys N through the end
    and atomically (somehow -- how is unclear because they're two separate
    programs -- the DB and Redis) unset them and store their values in the
    database. Not great. Any better ideas?



    --
    You received this message because you are subscribed to the Google Groups
    "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Matt Palmer at May 13, 2013 at 2:17 pm

    On Sat, May 11, 2013 at 06:21:51AM -0700, Charlie Brown wrote:
    Redis looks like a great solution for server-side web session data. Here's
    what I'd like to do: store everything in memory and expire it with LRU (or
    if that's not possible, with a TTL) and *when it expires*, put the content
    into a database. I don't want to write data to the database until it
    expires in Redis. If someone later tries to retrieve one of these
    now-expired (and thus no longer in memory) keys, they can go to the
    database, retrieve the value and invoke `setnx` and now their session is in
    memory and recently used.

    How can I get this "expire to database" functionality? If I have to do it
    manually by finding old keys, what's the best way to find them? The first
    approach I thought of is labor-intensive: on every session update, update
    the score of a sorted set, then periodically grab keys N through the end
    and atomically (somehow -- how is unclear because they're two separate
    programs -- the DB and Redis) unset them and store their values in the
    database. Not great. Any better ideas?
    Without modifying Redis, I don't think there's a way to do this that doesn't
    involve some sort of brute force (sets, etc). I'm working on a modification
    to Redis that keeps all keys backed to disk, with the "hot set" of keys in
    memory for speedy access. It's viciously experimental, but it works pretty
    much exactly as you describe (where "the database" == "an on-disk key/value
    store"). $DAYJOB's tech writer put up an introductory blog post about it:

    http://www.anchor.com.au/blog/2013/04/redis-rethought-exciting-extremes-with-larger-than-memory-datasets/

    The code's available on Github, although it's running a bit behind reality
    at the moment -- I'm refactoring it to use MDB rather than KyotoCabinet,
    because KC has some really perverse disk space consumption behaviours with
    NDS' use case. The Github code should be back up-to-date in 24 hours.

    - Matt

    --
    Well-designed presentation software would, when the user chooses any
    transition other than "dissolve", put up a dialog box saying "You are a
    jerk. Press OK to continue."
       -- Simon Cozens, in a place that doesn't exist

    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Didier Spezia at May 13, 2013 at 5:25 pm
    You may want to have a look at my answer on SO:
    http://stackoverflow.com/questions/11810020/how-to-handle-session-expire-basing-redis/11815594#11815594

    Regards,
    Didier.

    --
    You received this message because you are subscribed to the Google Groups "Redis DB" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
    To post to this group, send email to redis-db@googlegroups.com.
    Visit this group at http://groups.google.com/group/redis-db?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupredis-db @
categoriesredis
postedMay 12, '13 at 10:08p
activeMay 13, '13 at 5:25p
posts4
users4
websiteredis.io
irc#redis

People

Translate

site design / logo © 2022 Grokbase