Hadoop > mapreduce-user > November 2009

Hi, I use cloudera's distribution for hadoop. What I see is that a small fraction of maps get stuck at 100%. They show up as 100% but continue running. After a lot of delay, they succeed finally but ...

hi, I'm try to write a hadoop streaming job by perl. But i'm complately confused by the key/value separator. I found lots of separators I can set ... # -jobconf stream.map.output.field.separator=A \ ...

Hey all, I am trying to figure out the best way to approach some joining/merging computation in a map-reduce / hbase framework. I have the following large timeseries datasets (key/value pairs keyed ...

I wrote a simple code in my eclipse as Text t = new Text("hadoop"); System.out.println((char)t.charAt(2)); when I try to run this I got Exception in thread "main" java.lang.NoClassDefFoundError: ...

Hi, briefly, I'm writing my dissertation on Distributed computation, and then in detail at the various interfaces atop of Hadoop, including Pig, Hive, JAQL etc... One thing I have noticed in early ...

hi all, We are using hadoop-0.19.1 on about 200 nodes. We find there are lots of slaves keep Child process even the job is done. Here is an example, the process is running since "AUGEST 09"! jstack ...

I want to know if I can "wire" mapper and reducer classes. By "wire" I mean to use inversion of control (or dependency injection). For example, I see in a lot of books and on the Map/Reduce tutorial ...

Dear hadoop fellows, We have been using Hadoop-0.20.1 MapReduce to crawl some web data. In this case, we only have mappers to crawl data and save data into HDFS in a distributed way. No reducers is ...

Hi all, 1 - I'm trying to compile hadoop-0.20.1 but I'm facing several errors. Hadoop includes several subprojects, but these subprojects are all compiled in one file called hadoop-0.20.1-core.jar. ...

Hi All! How much overhead using Hadoop Streming vs. native Java steps does add? How can I estimate performance increase I can get by rewriting my streaming job to Java? What parameters the overhead ...

Hey all, I am currenty trying to write a simple map job that dumps a file to the hdfs. I see that hadoop creates 3 instances of my mapper class (I expected 3, so this is fine :) but then things get a ...

Hi all, I have 2 KVP files of 200million+ rows, and plan to do a reduce side join (my first...). Input 1 ---------- KEY TC_ID Input 2 ---------- KEY OCC_ID I aim to produce an output of: Output ...

(moving to mapreduce-user@hadoop.apache.org) Use one of the several ways to pass the queue name as a config variable: JobConf j = new JobConf(); j.setQueueName("myqueue"); or cmd line: hadoop jar ...

hello I am using hadoop wordcount examples and trying to use jobcleint API to get the status of map and reduce as i am trying to get my own status bar for the map reduce instead using JobTracker.jsp ...

Hello, I need to know how to configure a job implemented in java (in Hadoop Training Cloudera VM) to run only one Reduce Task and it begins only when all Map Tasks have completed their execution. I ...

Dear Hadoop fellows, We have been using Hadoop-0.18.3 to run MapReduce in web application. We have tried to run the application in both Jetty 6.1 and Tomcat 6.0. At first, we directly call the ...