FAQ
Hi All

We plan to setup a Impala cluster in AWS with 5 to 10 machines to evaluate
it. It suggests the following hardware requirements in Impala installation
guide:

Memory - 128 GB or more recommended, ideally 256 GB or more.
Storage - DataNodes with 12 or more disks each

So we may have to chose the following instance types:

    1. m2.4xlarge
    2. cr1.8xlarge
    3. hs1.8xlarge

But we cannot make the final decision, guys could you please share your
ideas on this, thanks.

Cheers
Ramon

To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.

Search Discussions

  • Amandeep Khurana at Jan 26, 2014 at 7:09 am
    Hi Ramon

    I'd recommend trying the hs1.8xlarge or cc2.8xlarge instances, depending on
    your storage requirements. m2.4xlarge has relatively lower I/O compared to
    the these two and cr1.8xlarge has relatively lower amount of storage
    capacity. You'll end up paying a higher price for the SSD storage on the
    cr1.8xlarge but won't get a good bang for your buck because of lower total
    storage and the fact that HDFS doesn't leverage SSD performance
    characteristics to justify the cost.

    I'd recommend taking a look at the reference architecture
    here<http://www.cloudera.com/content/dam/cloudera/Resources/PDF/whitepaper/AWS_Reference_Architecture_Whitepaper.pdf>.
    It's for production deployments of the Hadoop stack on AWS and might be a
    useful read if you are looking at a deployment where performance and
    security is important.

    Hope this helps.

    -Amandeep


    ---
    Amandeep Khurana
    Cloudera Inc
    Twitter: @amansk

    On Sat, Jan 25, 2014 at 11:02 PM, wrote:

    Hi All

    We plan to setup a Impala cluster in AWS with 5 to 10 machines to evaluate
    it. It suggests the following hardware requirements in Impala installation
    guide:

    Memory - 128 GB or more recommended, ideally 256 GB or more.
    Storage - DataNodes with 12 or more disks each

    So we may have to chose the following instance types:

    1. m2.4xlarge
    2. cr1.8xlarge
    3. hs1.8xlarge

    But we cannot make the final decision, guys could you please share your
    ideas on this, thanks.

    Cheers
    Ramon

    To unsubscribe from this group and stop receiving emails from it, send an
    email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Ramon Wang at Jan 26, 2014 at 8:16 am
    Hi Amandeep

    Thanks for the quick reply, that's really helpful. Although they(hs1.8xlarge
    and cc2.8xlarge) don't meet the memory requirement(128GB or 256GB), we will
    try both of them.

    Cheers
    Ramon


    On Sun, Jan 26, 2014 at 3:08 PM, Amandeep Khurana wrote:

    Hi Ramon

    I'd recommend trying the hs1.8xlarge or cc2.8xlarge instances, depending
    on your storage requirements. m2.4xlarge has relatively lower I/O compared
    to the these two and cr1.8xlarge has relatively lower amount of storage
    capacity. You'll end up paying a higher price for the SSD storage on the
    cr1.8xlarge but won't get a good bang for your buck because of lower total
    storage and the fact that HDFS doesn't leverage SSD performance
    characteristics to justify the cost.

    I'd recommend taking a look at the reference architecture here<http://www.cloudera.com/content/dam/cloudera/Resources/PDF/whitepaper/AWS_Reference_Architecture_Whitepaper.pdf>.
    It's for production deployments of the Hadoop stack on AWS and might be a
    useful read if you are looking at a deployment where performance and
    security is important.

    Hope this helps.

    -Amandeep


    ---
    Amandeep Khurana
    Cloudera Inc
    Twitter: @amansk

    On Sat, Jan 25, 2014 at 11:02 PM, wrote:

    Hi All

    We plan to setup a Impala cluster in AWS with 5 to 10 machines to
    evaluate it. It suggests the following hardware requirements in Impala
    installation guide:

    Memory - 128 GB or more recommended, ideally 256 GB or more.
    Storage - DataNodes with 12 or more disks each

    So we may have to chose the following instance types:

    1. m2.4xlarge
    2. cr1.8xlarge
    3. hs1.8xlarge

    But we cannot make the final decision, guys could you please share your
    ideas on this, thanks.

    Cheers
    Ramon

    To unsubscribe from this group and stop receiving emails from it, send an
    email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupimpala-user @
categorieshadoop
postedJan 26, '14 at 7:02a
activeJan 26, '14 at 8:16a
posts3
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Ramon Wang: 2 posts Amandeep Khurana: 1 post

People

Translate

site design / logo © 2022 Grokbase