FAQ
Hi,

I was trying to set-up hadoop/hbase cluster on ec2 which took me few hours
to set-up from scratch on bundled image from s3. I am curious to know, what
is the best way to setting hadoop/hbase cluster on amazon ec2? How do we do
it fast?

Thanks in advance!

regards,
Shah

Search Discussions

  • Harsh J at Sep 7, 2011 at 11:29 am
    You are looking for the Apache Whirr project: http://whirr.apache.org/

    Here's a great article at Phil Whelan's site that covers getting HBase
    up in a jiffy on ec2:
    http://www.philwhln.com/run-the-latest-whirr-and-deploy-hbase-in-minutes
    On Wed, Sep 7, 2011 at 4:48 PM, Shahnawaz Saifi wrote:
    Hi,

    I was trying to set-up hadoop/hbase cluster on ec2 which took me few hours
    to set-up from scratch on bundled image from s3. I am curious to know, what
    is the best way to setting hadoop/hbase cluster on amazon ec2? How do we do
    it fast?

    Thanks in advance!

    regards,
    Shah


    --
    Harsh J
  • John Conwell at Sep 7, 2011 at 2:59 pm
    I second that. Whirr is an invaluable resource for automagically spinning
    up resources on EC2
    On Wed, Sep 7, 2011 at 4:28 AM, Harsh J wrote:

    You are looking for the Apache Whirr project: http://whirr.apache.org/

    Here's a great article at Phil Whelan's site that covers getting HBase
    up in a jiffy on ec2:
    http://www.philwhln.com/run-the-latest-whirr-and-deploy-hbase-in-minutes
    On Wed, Sep 7, 2011 at 4:48 PM, Shahnawaz Saifi wrote:
    Hi,

    I was trying to set-up hadoop/hbase cluster on ec2 which took me few hours
    to set-up from scratch on bundled image from s3. I am curious to know, what
    is the best way to setting hadoop/hbase cluster on amazon ec2? How do we do
    it fast?

    Thanks in advance!

    regards,
    Shah


    --
    Harsh J


    --

    Thanks,
    John C
  • Shi Yu at Sep 7, 2011 at 4:41 pm
    Interested in this topic. We have experienced plenty of difficulties
    running hadoop in Eucalyptus based virtual instance clusters. Typical
    issues like

    java.net.SocketTimeoutException: 69000 millis timeout while waiting for
    channel to be ready for read. ch : java.nio.channels.SocketChannel

    kill the whole job. The IO of HDFS based on network storage is very
    slow. I am wondering whether Apache Whirr has made any significant
    improvement for hadoop implementation in virtual instances like Ec2.

    On 9/7/2011 9:58 AM, John Conwell wrote:
    I second that. Whirr is an invaluable resource for automagically spinning
    up resources on EC2

    On Wed, Sep 7, 2011 at 4:28 AM, Harsh Jwrote:
    You are looking for the Apache Whirr project: http://whirr.apache.org/

    Here's a great article at Phil Whelan's site that covers getting HBase
    up in a jiffy on ec2:
    http://www.philwhln.com/run-the-latest-whirr-and-deploy-hbase-in-minutes

    On Wed, Sep 7, 2011 at 4:48 PM, Shahnawaz Saifi<shahsaifi@gmail.com>
    wrote:
    Hi,

    I was trying to set-up hadoop/hbase cluster on ec2 which took me few hours
    to set-up from scratch on bundled image from s3. I am curious to know, what
    is the best way to setting hadoop/hbase cluster on amazon ec2? How do we do
    it fast?

    Thanks in advance!

    regards,
    Shah

    --
    Harsh J
  • Shahnawaz Saifi at Sep 7, 2011 at 5:04 pm
    Thanks a lot, I will definitely try this. But there are so many blogs about
    configuring hadoop/hbase and bundling images to s3 bucket. Whirr is faster
    or smoother than this concept?

    regards,
    Shah
    On Wed, Sep 7, 2011 at 8:28 PM, John Conwell wrote:

    I second that. Whirr is an invaluable resource for automagically spinning
    up resources on EC2
    On Wed, Sep 7, 2011 at 4:28 AM, Harsh J wrote:

    You are looking for the Apache Whirr project: http://whirr.apache.org/

    Here's a great article at Phil Whelan's site that covers getting HBase
    up in a jiffy on ec2:
    http://www.philwhln.com/run-the-latest-whirr-and-deploy-hbase-in-minutes

    On Wed, Sep 7, 2011 at 4:48 PM, Shahnawaz Saifi <shahsaifi@gmail.com>
    wrote:
    Hi,

    I was trying to set-up hadoop/hbase cluster on ec2 which took me few hours
    to set-up from scratch on bundled image from s3. I am curious to know, what
    is the best way to setting hadoop/hbase cluster on amazon ec2? How do
    we
    do
    it fast?

    Thanks in advance!

    regards,
    Shah


    --
    Harsh J


    --

    Thanks,
    John C


    --
    Thanks,
    Shah
  • John Conwell at Sep 7, 2011 at 9:11 pm
    I dont know about configuring hbase, but whirr definitely makes spinning up
    a hadoop cluster very very easy. Checkout the quickstart guide. As long as
    you have your EC2 creds, its very simple to ramp up a hadoop cluster.
    http://whirr.apache.org/docs/0.6.0/quick-start-guide.html
    On Wed, Sep 7, 2011 at 10:03 AM, Shahnawaz Saifi wrote:

    Thanks a lot, I will definitely try this. But there are so many blogs about
    configuring hadoop/hbase and bundling images to s3 bucket. Whirr is faster
    or smoother than this concept?

    regards,
    Shah
    On Wed, Sep 7, 2011 at 8:28 PM, John Conwell wrote:

    I second that. Whirr is an invaluable resource for automagically spinning
    up resources on EC2
    On Wed, Sep 7, 2011 at 4:28 AM, Harsh J wrote:

    You are looking for the Apache Whirr project: http://whirr.apache.org/

    Here's a great article at Phil Whelan's site that covers getting HBase
    up in a jiffy on ec2:
    http://www.philwhln.com/run-the-latest-whirr-and-deploy-hbase-in-minutes
    On Wed, Sep 7, 2011 at 4:48 PM, Shahnawaz Saifi <shahsaifi@gmail.com>
    wrote:
    Hi,

    I was trying to set-up hadoop/hbase cluster on ec2 which took me few hours
    to set-up from scratch on bundled image from s3. I am curious to
    know,
    what
    is the best way to setting hadoop/hbase cluster on amazon ec2? How do
    we
    do
    it fast?

    Thanks in advance!

    regards,
    Shah


    --
    Harsh J


    --

    Thanks,
    John C


    --
    Thanks,
    Shah


    --

    Thanks,
    John C

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedSep 7, '11 at 11:19a
activeSep 7, '11 at 9:11p
posts6
users4
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2021 Grokbase