FAQ
I'm a rank beginner with clusters, but am determined to move into
them, starting with Hadoop. I have a habuntu machine under VMware on
my MacBook Pro for starters (got it on a DVD when visiting at the
Googleplex).

Now I've just received an Apple Xserve and four Mac Minis to set up a cluster.

Though I know little, I did see a suggestion somewhere that I set up a
small Mac OS X partition on each, under VMware Fusion, for general
admin via the Apple Server, for Mac firmware updates, monitoring, etc.

Then I'd set up a larger partition under VMware on each of the five
machines with linux images, to do the heavy lifting.

I'm looking for any and all suggestions from whomever. You may well
be able to skip all sorts of details and just point me to
answers/examples out there, people, search terms, etc. Then I can
start doing my homework. I'm happy to experiment, setting up stuff,
stripping it clean, setting up another approach, etc., as part of
learning what works.

Looking forward to the standard mix of confusion, frustration,
success, failure, elation, and lots of messy work.

Oh, what do I do? Want to map out image analysis and text analysis
work to the nodes. I analyze, extract from, Biomedical research
papers.

- Bob

Robert P. Futrelle
Associate Professor
Biological Knowledge Laboratory
College of Computer and Information Science
Northeastern University MS WVH202
360 Huntington Ave.
Boston, MA 02115

Office: 617-373-4239
Fax: 617-373-5121
http://www.ccs.neu.edu/home/futrelle
http://www.bionlp.org
http://www.diagrams.org

Search Discussions

  • Colin Evans at Oct 12, 2007 at 4:58 am
    I don't think that you need to run Linux - Hadoop should run just fine
    under OS X. You'll need to enable ssh, and you might have to fiddle
    around with the native libraries if you want to compile them, but
    everything else should work great.



    Bob Futrelle wrote:
    I'm a rank beginner with clusters, but am determined to move into
    them, starting with Hadoop. I have a habuntu machine under VMware on
    my MacBook Pro for starters (got it on a DVD when visiting at the
    Googleplex).

    Now I've just received an Apple Xserve and four Mac Minis to set up a cluster.

    Though I know little, I did see a suggestion somewhere that I set up a
    small Mac OS X partition on each, under VMware Fusion, for general
    admin via the Apple Server, for Mac firmware updates, monitoring, etc.

    Then I'd set up a larger partition under VMware on each of the five
    machines with linux images, to do the heavy lifting.

    I'm looking for any and all suggestions from whomever. You may well
    be able to skip all sorts of details and just point me to
    answers/examples out there, people, search terms, etc. Then I can
    start doing my homework. I'm happy to experiment, setting up stuff,
    stripping it clean, setting up another approach, etc., as part of
    learning what works.

    Looking forward to the standard mix of confusion, frustration,
    success, failure, elation, and lots of messy work.

    Oh, what do I do? Want to map out image analysis and text analysis
    work to the nodes. I analyze, extract from, Biomedical research
    papers.

    - Bob

    Robert P. Futrelle
    Associate Professor
    Biological Knowledge Laboratory
    College of Computer and Information Science
    Northeastern University MS WVH202
    360 Huntington Ave.
    Boston, MA 02115

    Office: 617-373-4239
    Fax: 617-373-5121
    http://www.ccs.neu.edu/home/futrelle
    http://www.bionlp.org
    http://www.diagrams.org
  • Ross Boucher at Oct 12, 2007 at 5:08 pm
    I can tell you from experience that Hadoop does run fine under OS X,
    and would second the recommendation not to bother with Linux. The
    process of setting up the cluster is also just as simple under OS X.

    Partitioning the internal drives may not be a bad idea, so you can
    keep the DFS data on its own partition, you may also want to look
    into getting external drives for the mac minis, since the stock
    drives are 5400 RPM. Then again, if you are not IO bound, it may not
    matter much.

    I used the following link to get through basic setup tasks, it has
    corresponding multi node cluster instructions as well.

    http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_%
    28Single-Node_Cluster%29

    I'd be happy to share anything else I've learned regarding my xserve
    cluster, so feel free to ask.

    Ross Boucher
    boucher@apple.com
    On Oct 11, 2007, at 9:54 PM, Colin Evans wrote:

    I don't think that you need to run Linux - Hadoop should run just
    fine under OS X. You'll need to enable ssh, and you might have to
    fiddle around with the native libraries if you want to compile
    them, but everything else should work great.



    Bob Futrelle wrote:
    I'm a rank beginner with clusters, but am determined to move into
    them, starting with Hadoop. I have a habuntu machine under VMware on
    my MacBook Pro for starters (got it on a DVD when visiting at the
    Googleplex).

    Now I've just received an Apple Xserve and four Mac Minis to set
    up a cluster.

    Though I know little, I did see a suggestion somewhere that I set
    up a
    small Mac OS X partition on each, under VMware Fusion, for general
    admin via the Apple Server, for Mac firmware updates, monitoring,
    etc.

    Then I'd set up a larger partition under VMware on each of the five
    machines with linux images, to do the heavy lifting.

    I'm looking for any and all suggestions from whomever. You may well
    be able to skip all sorts of details and just point me to
    answers/examples out there, people, search terms, etc. Then I can
    start doing my homework. I'm happy to experiment, setting up stuff,
    stripping it clean, setting up another approach, etc., as part of
    learning what works.

    Looking forward to the standard mix of confusion, frustration,
    success, failure, elation, and lots of messy work.

    Oh, what do I do? Want to map out image analysis and text analysis
    work to the nodes. I analyze, extract from, Biomedical research
    papers.

    - Bob

    Robert P. Futrelle
    Associate Professor
    Biological Knowledge Laboratory
    College of Computer and Information Science
    Northeastern University MS WVH202
    360 Huntington Ave.
    Boston, MA 02115

    Office: 617-373-4239
    Fax: 617-373-5121
    http://www.ccs.neu.edu/home/futrelle
    http://www.bionlp.org
    http://www.diagrams.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedOct 12, '07 at 3:29a
activeOct 12, '07 at 5:08p
posts3
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase