Hadoop > common-user > March 2006

jobtracker.jsp shows the list of TaskTrackers over the course of a job -- in a HADOOP-16 -like manner -- shrinking. No errors show in the jobdetails.jsp to explain the disappeared. I see this ...

Trying Hadoop out with a proof of concept I came across the following problem. My input looks conceptually like this: textid1, number1, number2, number3 textid2, number2, number2, number3 textid1, ...

Hello Team, I created a backup of my DFS database: # bin/hadoop dfs -copyToLocal /user/root/crawl /mylocaldir I now want to restore from the backup using: # bin/hadoop dfs -copyFromLocal ...

Thanks Andrzej, Works a treat, which is more than I can say for my thoroughly broken DFS :( I used: # bin/hadoop org.apache.hadoop.dfs.DFSck /user/root/crawl and got! Status: CORRUPT Total size: ...

hi. i'm trying to get hadoop running in the pseudo distributed configuration and have checked out the latest code and used the instructions located in the API: ...

Why would a lightly loaded nameserver w/ no other emissions on a seemingly healthy machine have trouble allocating blocks in a job that is almost done? From the nameserver log: 060323 173126 Server ...

Does Hadoop run multiple threads on a single slave? In my current experiment, I'm running with three slaves, each of which has four processors. Hadoop only appears to be using one processor on each ...

Hello, Hadoop wiki could use some configuration ;) Jeff.

Hi, I have some problem with classloading in hadoop. I had written a set of mapper and reducers with a set of custom writable implementations. Also my classes require nutch writables. I'm able to get ...

Dear Hadoop users and developers, I have been evaluating Hadoop with a view to using it as a distributed computing platform for performing fast operations on locally stored data. I have found several ...

Whoever added the names field to the general administration webpage of the jobtracker thank you. It is a very big help to see what the jobs are and the order in which they run. Dennis

Hi all Anyone subscribed to this yet? ;-) Anyhoo, I'm fairly new to hadoop and was wondering about the redundancy aspects of it. If I have a few servers running for nutch, one being a name and data ...

So, I've just worked my way through the GFS whitepaper (mapreduce is next), and was wondering what the major design/architecture differences are compared with Hadoop's DFS. The JavaDoc points out ...

Below are some exceptions gleaned from tasktracker logs. I was wondering if others are seeing similar and if suggestions on how to avoid/fix them. First, the general setting is a rack of 35 ...

Anyone else seeing these on startup of a job in fresh update from this morning (Revision: 382439)? 060302 170405 Server handler 0 on 50050 call error: java.io.IOException: ...

Hello Team, I have installed hadoop-2006-03-01.tar.gz, and configured hadoop-site.xml. The initial sanity tests from the tutorials worked fine until I start jobtracker the feedback looks as if it ...

Currently Even with the local option set, I am able to see many submit_file and job.jar and job.xml Is'nt this supposed to be used by the task tracker? So having a local option should not be creating ...

Hi I currently run hadoop with local option. Does this mean that still map reduce will work on the local system? If that is so, then hadoop incorporates two things 1)distributed file system 2) map ...

Hi, I updated to the latest source updated my installation and restarted the system with start-all.sh. I note that tasktrackers need up to several minutes to connect to the jobtracker. I saw in the ...

Hi, I've written several MapReduce jobs. However, I noticed the jobs took a long time to finish due to sort and reduce. I run tasks with 1 reducer. Is there a guideline on how many reduce task? If ...

Hello Team, I am running nutch with mapreduce/DFS using: - nutch-0.8-dev (rev="387659" of 2006-03-23) with - hadoop-0.1-dev (rev="387664") built and merged into <nutchdir /lib/hadoop - A cluster of 6 ...

Hi, Is there a way to make sure the job is actually got distributed to different nodes? Is there a command to check the task status associated with each box? On the slave box's log, I see ...Lost ...

I want to setup two servers 68.14.240.166 and 68.14.240.168 to run example program. I put the following into hadoop-site.xml and run bin/start-all.sh on both nodes. However, when I try to run ...

Yesterday I wrote of a hung rack. In that case -- after consultation w/ DC -- it looked like the map output files had been removed prematurely for some unknown reason. Subsequently, the rack is stuck ...

Hi, I'm curious if anyone has tried to run Hadoop on the Sun Grid. http://www.sun.com/service/sungrid/overview.jsp If so, would you consider posting any experiences, frustrations, or other ...

So, I was sick for a week, before which I'd been a week or so since I'd touched the hadoop work I've been doing. I left the namenode/datanodes running during that time, so that disk failures and ...