FAQ
If you raid EBS volumes you're pretty much protected from data loss as an
added benefit. So yeah if a broker goes down, you don't have the data until
you bring it back up, but you CAN bring it back up, even if the box dies.

On Mon, Nov 19, 2012 at 3:34 PM, Matt Jones wrote:

We have pretty much the same setup as this, running on m1.large instances.

On Mon, Nov 19, 2012 at 11:44 AM, Matthew Rathbone
wrote:
We RAID-0 4 EBS disks, we find that to be most performant, although it does
leave you more vulnerable to EBS network errors and outages. Ideally, if
you're spread across AZ's you could do some clever routing to
geographically local brokers, whilst keeping the others as a backup in case
of failure.

In the same vein, we have N+1 brokers, where N is how many we reasonably
think we need, this way we can hopefully survive outages.

On Mon, Nov 19, 2012 at 1:30 PM, David Arthur wrote:

I'd only consider m1.xlarge and higher for Kafka. The m1.xlarge have
"high" I/O performance according to Amazon. This is disk I/O and
network
I/O performance. Of course you need to use EBS volumes if you want your
Kafka brokers to survive reboots - you can expect reboots on AWS. Some
people have reported I/O improvements by RAIDing EBS volumes (
http://alestic.com/2009/06/ec2-ebs-raid). Deploying in the same region as
your application will also improve performance.

On Nov 16, 2012, at 8:04 PM, Senthilvel Rangaswamy wrote:

Have folks implemented large installations on Kafka on Amazon EC2. I
am
looking for best practices. Like the kind of nodes, EBS vs Instance
store
etc.,

--
..Senthil

"If there's anything more important than my ego around, I want it
caught and shot now."
- Douglas Adams.

--
Matthew Rathbone
Foursquare | Software Engineer | Server Engineering Team
matthew@foursquare.com | @rathboma <http://twitter.com/rathboma> |
4sq<http://foursquare.com/rathboma>


--
Matthew Rathbone
Foursquare | Software Engineer | Server Engineering Team
matthew@foursquare.com | @rathboma <http://twitter.com/rathboma> |
4sq<http://foursquare.com/rathboma>

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 10 of 13 | next ›
Discussion Overview
groupusers @
categorieskafka
postedNov 17, '12 at 1:05a
activeNov 20, '12 at 5:13p
posts13
users9
websitekafka.apache.org
irc#kafka

People

Translate

site design / logo © 2021 Grokbase