[Hadoop-common-user] Hadoop - Solaris

That's my cue! :)

We have a few grids that are running Solaris. It mostly works out of the box as long as you are aware of three things:

- There are some settings in hadoop-env.sh and in the path that need to be dealt with. Rather than re-quote, these were added to the Hadoop FAQ a week or two ago so definitely take a look at that.

- The native compression libraries will need to be compiled. Depending upon what you are doing/how performant the machines are, this may or may not make a big difference. Compiling under Solaris with gcc will work fine (but it is gcc... ugh!). Only a few minor changes are required to compile it with SUNWspro. [I have patches laying around here somewhere if anyone wants to play with them.]

- The Solaris JRE is a mixed-mode implementation. So keep in mind that -d32 and -d64 have meaning and do work as advertised. You'll likely want to pick a bitsize and use that for all your Hadoop daemons and tasks, especially if you plan on using any JNI like the compression libraries.

I have no recent experiences with Solaris nor with Hadoop on Solaris,
but there is an blog: http://blogs.sun.com/george/category/Hadoop with
three postings dedicated to Hadoop (although maybe a bit outdated). In
2007 or 2008 GNU and Solaris did not play together very well as things
as tar, gcc are concerned. But these issues should have gone now.

Sun Grid Engine might be a starting point to look for as Hadoop is built in.



2010/10/16 Allen Wittenauer <awittenauer@linkedin.com>:

Only if you didn't know to look in /usr/sfw or were using such an ancient version of Solaris that one be embarrassed.

Of course it is simple if you know about it. From an Linux and Java
point of view the Solaris way is not that obvious.

My 2 cents

2010/10/17 Allen Wittenauer <awittenauer@linkedin.com>:

Java has nothing to do with GNU tar's extensions or gcc.

Forward compatibility is definitely not the Linux way, so it is easy to understand the confusion.

I didn't want to jump in and say thanks for each post but thanks for
all. Owen, you probably don't remember me, but we met at that first
Hadoop meetup in Sunnyvale a couple years ago.
Bruce Williams
-----------------------------------------------------------------------------------------------------------------------------





On Sun, Oct 17, 2010 at 5:44 PM, Allen Wittenauer
wrote:

LinkedIn is currently running Hadoop on Solaris. Hopefully, Allen
Wittenauer can get back to you on some hints.

-- Owen