[Redis] [ANN] Scarlet 0.6.0 - An HTTP interface for Redis

On Thu, Nov 1, 2012 at 2:21 PM, Nick Saika wrote:

In my spare time, I have been working on an HTTP frontend for Redis,
written in Go: Scarlet <http://nesv.github.com/Scarlet/>.

Version 0.6.0 of Scarlet is the first version to be "fully-functional";
you can create, read, update and delete keys. Some of my future plans for
Scarlet include:

- Providing clustering support
- Automatic failover for replication setups
- A nice, web-based monitoring interface for your clustering and
replication setups
- Act as a load-balancer in front of a cluster, including forwarding
all write operations to masters, and all read operations to slaves

If anyone is interested in trying it out, I would welcome any feedback!

--
You received this message because you are subscribed to the Google Groups
"Redis DB" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/redis-db/-/8eyz24uB5OAJ.
To post to this group, send email to redis-db@googlegroups.com.
To unsubscribe from this group, send email to
redis-db+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/redis-db?hl=en.

On Thursday, 1 November 2012 17:57:46 UTC-4, Michael Gorsuch wrote:

Nick - thank you! I love the idea of a Go frontend largely due to ease of
deployment (don't need to make sure a supporting runtime is installed).

When you speak of replication setups, are you considering leveraging
sentinel here?

Best,

Michael Gorsuch


On Thu, Nov 1, 2012 at 2:21 PM, Nick Saika <nick...@gmail.com<javascript:>

To be perfectly honest, Michael, I am taking the approach of assuming
there is no Sentinel; that being said though, I have to admit I haven't
looked too much into Redis Sentinel, and I will definitely do that over my
vacation, and see if I can leverage any of the feature(s) provided by
Sentinel.

My initial plan for assisting with Redis' master-slave replication, was to
build up a list of nodes in the chain via data gleaned from the INFO
command. Should a master fail out, for any reason, Scarlet would elect one
of that master's slaves to become the new master, and then update any of
the old master's slaves to point to the new master. I haven't thought
too-too far into this yet, but my initial thought would be to have the old
master (once it comes alive again), to become a slave of the new master.
You received this message because you are subscribed to the Google Groups
"Redis DB" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/redis-db/-/qVMMqzfprtwJ.

To post to this group, send email to redis-db@googlegroups.com.
To unsubscribe from this group, send email to
redis-db+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/redis-db?hl=en.

On Thu, Nov 1, 2012 at 9:21 PM, Nick Saika wrote:
To be perfectly honest, Michael, I am taking the approach of assuming there
is no Sentinel; that being said though, I have to admit I haven't looked too
much into Redis Sentinel, and I will definitely do that over my vacation,
and see if I can leverage any of the feature(s) provided by Sentinel.

My initial plan for assisting with Redis' master-slave replication, was to
build up a list of nodes in the chain via data gleaned from the INFO
command. Should a master fail out, for any reason, Scarlet would elect one
of that master's slaves to become the new master, and then update any of the
old master's slaves to point to the new master. I haven't thought too-too
far into this yet, but my initial thought would be to have the old master
(once it comes alive again), to become a slave of the new master.
--
You received this message because you are subscribed to the Google Groups
"Redis DB" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/redis-db/-/qVMMqzfprtwJ.

To post to this group, send email to redis-db@googlegroups.com.
To unsubscribe from this group, send email to
redis-db+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/redis-db?hl=en.

On Fri, Nov 2, 2012 at 5:43 AM, Josiah Carlson wrote:

Considering that you are describing essentially what Redis Sentinel
does, only Redis Sentinel does a lot more work to make sure it isn't
doing bad things, you really should work off of the assumption that
Redis Sentinel exists.

On Fri, Nov 2, 2012 at 6:43 AM, Josiah Carlson wrote:
Considering that you are describing essentially what Redis Sentinel
does, only Redis Sentinel does a lot more work to make sure it isn't
doing bad things, you really should work off of the assumption that
Redis Sentinel exists.

Regards,
- Josiah
--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To post to this group, send email to redis-db@googlegroups.com.
To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.

On Friday, 2 November 2012 07:58:49 UTC-4, dvirsky wrote:
This. Sentinel is not finished yet, but in my company we're in the
process of integrating it into production, while hacking it a bit. So
my guess is that it will mature in the coming months.

What we're missing and added a patch for, was to enable some sort of
orchestration via sentinel -that is, a slave goes up, it doesn't need
to know who is his master, just the addresses of the sentinels, and
its "master name" or as I call it, "role". It connects to the
sentinel, asks for the master for that role, and connects to it. It
makes auto scaling redis master/slave clusters very very easy.

I hope our patches will be seen favorably by the powers that be, hint hint
;)

there
looked too
vacation,
to
one
of the
too-too
master
ease of
installed).
"fully-functional";
for
all
feedback!
Groups
Groups


--
Dvir Volk
Chief Architect, Everything.me
http://everything.me

On Fri, Nov 2, 2012 at 7:10 AM, Nick Saika wrote:
Additionally, I was initially approaching this project as a simple web
frontend to Redis, to make it a worthy contender with Riak (mind you,
Scarlet + Redis is still two, moving parts versus one), because truth be
told, I do enjoy Riak's HTTP interface, but I would much rather use Redis
because of its wonderful types (sets, lists, sorted sets, and hashes).

As I started writing Scarlet, it seemed to have the potential to act as a
clustering/replication management solution, should Redis Sentinel not be
available to you (think of slow-moving enterprise-y setups, where everything
must be standardized). That being said though, clustering is not my "next
task". After I switch out the underlying driver (which is almost done), my
next work item is going to be master-slave discovery insofar as sending all
write-op commands to master(s), and load balancing all read-op commands
between slaves.
--
You received this message because you are subscribed to the Google Groups
"Redis DB" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/redis-db/-/Hfgv0mMRDHMJ.

To post to this group, send email to redis-db@googlegroups.com.
To unsubscribe from this group, send email to
redis-db+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/redis-db?hl=en.

On Fri, Nov 2, 2012 at 11:14 AM, Josiah Carlson wrote:

Have you benchmarked your http interface yet? No offense intended, but
I would be surprised if the parsing overhead of http, along with the
overhead of sending and receiving traffic (especially the data
copy/buffering) wouldn't significantly *reduce* the performance of the
stack.

In particular, HAProxy is the fastest known software HTTP proxy
(though it got beta SSL support in the most recent development release
:D ), and it peaks at 100k requests/second on a single machine,
without passing connections through. When using passthrough, it sees
40k requests/second. But Redis itself can handle 100k requests/second
easily across persistent connections, and with pipelining, I see
300k-400k/second without issue. With a bit of processor affinity
tweaking, I can see over 700k pipelined requests/second on a 2.4 ghz
core2 duo.

While it's awesome to want to build an http interface to Redis (thus
simplifying a variety of use-cases), note that you will likely be
slowing down Redis, and may never see a payoff in your load-balancing
proxy, unless each of your requests is expensive (large sort, union,
intersection, or difference operations).

Regards,
- Josiah
hint
was
elect
Redis,
plans
--
You received this message because you are subscribed to the Google Groups
"Redis DB" group.
To post to this group, send email to redis-db@googlegroups.com.
To unsubscribe from this group, send email to
redis-db+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/redis-db?hl=en.

On Friday, 2 November 2012 00:43:51 UTC-4, Josiah Carlson wrote:

Considering that you are describing essentially what Redis Sentinel
does, only Redis Sentinel does a lot more work to make sure it isn't
doing bad things, you really should work off of the assumption that
Redis Sentinel exists.

Regards,
- Josiah
of
"fully-functional";
for
feedback!
Groups