FAQ
Hi all,

I have a 4 node test cluster on Amazon EC2 using EBS volumes for data
storage that I would like to bring up / down when not in use.

I have assigned 4 Elastic IP's to each node and resolved those Elastic IP's
to host names:

namenode1.domain.com -> Elastic IP 1
datanode1.domain.com -> Elastic IP 2
datanode2.domain.com -> Elastic IP 3
datanode3.domain.com -> Elastic IP 4 etc...

When I use Cloudera manager to add hosts as "namenode1.domain.com"
everything works correctly and installs but,
the node/instance that is configured in Cloudera manager is then configured
with an internal IP :

ip-10-70-x-x.ec2.internal & 10.70.x.x

Seeing as I will be bringing this cluster down / back up each day/week I
will need a more dynamic solution.

Is there anyway to force Cloudera manager to use my hostnames / elastic
IP's and not detect the private IP / hostname?

Any help would be greatly appreciated!

Thanks,

Evan

Search Discussions

  • Philip Langdale at Jan 16, 2013 at 7:58 pm
    Hi Evan,

    See this previous discussion about what you need to consider.

    https://groups.google.com/a/cloudera.org/d/msg/scm-users/AHiKnmXOh2w/hW8UwkKLh0AJ

    You will not be able to achieve anything completely seamless, such as
    Hadoop actually using the elastic
    IP addresses for all communication, simply because it's impossible to
    discover the elastic IP without
    making ec2-specific calls. You'll also have to manage the re-association of
    elastic IPs to instances when
    you restart them.

    --phil

    On 16 January 2013 11:11, 3con wrote:

    Hi all,

    I have a 4 node test cluster on Amazon EC2 using EBS volumes for data
    storage that I would like to bring up / down when not in use.

    I have assigned 4 Elastic IP's to each node and resolved those Elastic
    IP's to host names:

    namenode1.domain.com -> Elastic IP 1
    datanode1.domain.com -> Elastic IP 2
    datanode2.domain.com -> Elastic IP 3
    datanode3.domain.com -> Elastic IP 4 etc...

    When I use Cloudera manager to add hosts as "namenode1.domain.com"
    everything works correctly and installs but,
    the node/instance that is configured in Cloudera manager is then
    configured with an internal IP :

    ip-10-70-x-x.ec2.internal & 10.70.x.x

    Seeing as I will be bringing this cluster down / back up each day/week I
    will need a more dynamic solution.

    Is there anyway to force Cloudera manager to use my hostnames / elastic
    IP's and not detect the private IP / hostname?

    Any help would be greatly appreciated!

    Thanks,

    Evan

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedJan 16, '13 at 7:11p
activeJan 16, '13 at 7:58p
posts2
users2
websitecloudera.com
irc#hadoop

2 users in discussion

3con: 1 post Philip Langdale: 1 post

People

Translate

site design / logo © 2022 Grokbase