Search Discussions
-
Hi Everyone, I make a Project with Hadoop-MapRedeuce for my master-Thesis. I have a strange problem on my System. First of all, I use Hadoop-0.20.2 on Windows XP Pro with Eclipse Plug-In. When I ...
Baran cakici
Mar 16, 2011 at 4:50 pm
Oct 6, 2011 at 4:29 pm -
Hi, Is there a good tutorial for writing custom InputFormat classes? Any help would be greatly appreciated. Thanks, Souri
Souri datta
Mar 4, 2011 at 5:58 pm
Mar 6, 2011 at 7:51 am -
I am not sure if this is the right listserv, forgive me if it is not. My goal is this: monitor HDFS until a file is create, and then kick off a job. Ideally I'd want to do this continuously, but the ...
Jonathan Coveney
Mar 24, 2011 at 5:10 pm
May 15, 2011 at 10:10 pm -
Hi mapreduce gurus - Today while looking into a few TaskTracker's with full disks I came across the following directory using 207GB: ...
Travis Crawford
Mar 31, 2011 at 6:45 pm
Apr 1, 2011 at 9:12 pm -
Hi, I'm trying to increase the job-tracker heap by adding Xmx definition to hadoop-env.sh (adding it to HADOOP_TASKTRACKER_OPTS). I keep getting this error while starting hadoop: Invalid maximum heap ...
Lior Schachter
Mar 17, 2011 at 4:38 pm
Mar 17, 2011 at 6:19 pm -
Hi, I am working on a problem to compare two different datasets, and rank each record of the first with respect to the other, in terms of how similar they are. The records are dimensional, but do not ...
Sonal Goyal
Mar 7, 2011 at 7:07 pm
Mar 10, 2011 at 6:39 am -
Hi, I downloaded CDH3 VM for hadoop but if I want to use something like: import org.apache.hadoop.conf.Configuration; in my java code, what else do I need to do ? Do i need to download hadoop from ...
Mapred Learn
Mar 8, 2011 at 12:22 am
Mar 9, 2011 at 3:50 am -
Re: Mapper and Reducer of Terasort examplehi, i inadvertently got onto this list and it's not all that relevant for me, can anyone suggest how i get off it? Many thanks ----- Original Message ----- ...
David Heneghan
Mar 31, 2011 at 1:56 pm
Apr 1, 2011 at 8:23 am -
Hi, I can't seem to be able to find either jira or implementation of MultipleOutputFormat in new api in either 0.21 or 0.22 branches. Are there any plans to port that to new api as well? thanks in ...
Dmitriy Lyubimov
Mar 30, 2011 at 3:40 am
Mar 30, 2011 at 4:13 am -
Hi, is it possible to configure the total number of slots that a TaskTracker has, to run the map and reduce tasks? Thanks, -- Pedro
Pedro Costa
Mar 25, 2011 at 6:26 pm
Mar 25, 2011 at 7:25 pm -
Hi all , I am a beginner to hadoop . I am passing a FileName as the Key and the Contents of he FIle as the Value I want to write the contents of the FIle to a text file with Key as the fileName, from ...
Dinesh shan
Mar 24, 2011 at 12:19 pm
Mar 24, 2011 at 4:03 pm -
Hi What's the purpose of the parameter "mapred.min.split.size"? Thanks, -- Pedro
Pedro Costa
Mar 18, 2011 at 7:55 pm
Mar 18, 2011 at 10:59 pm -
Hi, I want to know that io.sort.mb and io.sort.factor are properties that set for the whole cluster, or can be set differently per node? -- Best regards!
Debbie Fu
Mar 14, 2011 at 8:27 am
Mar 14, 2011 at 9:38 am -
Hi all I have a hadoop cluster with a namenode and 3 datanodes, I've installed it by normal process. everything's fine, but it couldn't run wordcount map reduce job. Follow are output logs ...
Yorgo Sun
Mar 14, 2011 at 4:12 am
Mar 14, 2011 at 6:41 am -
We see a lot of IOExceptions coming from HDFS during a job that does nothing but untar 100 files (1 per Mapper, sizes vary between 5GB and 80GB) that are in HDFS, to HDFS. DataNodes are also showing ...
Evert Lammerts
Mar 9, 2011 at 11:30 am
Mar 9, 2011 at 4:57 pm -
Hi everyone, I'm experiencing a chain of runtime exceptions which I believe starts with a ClassNotFoundException for a Trove library class I'm using in my custom jar. Here's a copy of the error ...
John Therrell
Mar 31, 2011 at 10:58 pm
Apr 9, 2011 at 8:33 am -
I tried to run hadoop on multi-node cluster. Each node installs hadoop on different path. But not success Is it possible to run hadoop on multi-node cluster and these nodes install hadoop on ...
Zhengjun chen
Mar 3, 2011 at 2:36 pm
Apr 7, 2011 at 2:56 am -
I'm looking to get acquainted with the new API in 0.20.2 but all the online documentation I've found uses the old API. I need to understand how to chain two mapreduce jobs together efficiently that ...
John Therrell
Mar 30, 2011 at 6:09 pm
Apr 6, 2011 at 8:28 pm -
Hi, We recently did some experiment on mapreduce job scheduling and found that sometimes there were 2 jobs running on the same machine and each of them ran very slowly. We used to think that 2nd job ...
Wisteria_longyan=126 Com
Mar 29, 2011 at 2:21 pm
Mar 29, 2011 at 10:00 pm -
Hi, We have a map/reduce task that insert to hbase (in the reduce phase). Our problem is that some reduce jobs finish early and then the framework use them in-order to "backup" running reducers (As ...
Lior Schachter
Mar 28, 2011 at 3:02 pm
Mar 28, 2011 at 8:38 pm -
Hi, 1 - A Task is composed by several phases: STARTING, MAP/REDUCE, SHUFFLE, SORT, CLEANUP. A JobTracker and a TaskTracker also has phases? 2 - It exists also the following phases RUNNING, SUCCEEDED, ...
Pedro Costa
Mar 24, 2011 at 6:38 pm
Mar 24, 2011 at 6:47 pm -
Hello, I have a hadoop cluster that has 10 nodes. I use both of the hdfs/mapred(0.20.2) and the hbase(0.20.6). Every node is both of a datanode and a tasktracker. But I just configured 9 nodes as the ...
Jameson Li
Mar 21, 2011 at 11:31 am
Mar 22, 2011 at 2:46 am -
Hi, I would like define the number of map tasks to use in the GridMix2. For example, I would like to run the GridMixMonsterQuery at GridMix2 with 5 maps, another with 10 and another with 20 maps. How ...
Pedro Costa
Mar 18, 2011 at 5:05 pm
Mar 19, 2011 at 2:02 am -
Hi , I am trying to run a job which does not require reducer, I commented out the reducer configuration from the job, but now its throwing following error java.lang.ClassCastException: class ...
Sagar Kohli
Mar 11, 2011 at 5:53 am
Mar 11, 2011 at 7:32 am -
Hi I am new to Hadoop, so maybe I am missing something obvious. I have written a small map reduce program that runs two jobs. I want the output of the first job to serve as the input to the second ...
John Sanda
Mar 3, 2011 at 2:22 am
Mar 3, 2011 at 4:48 am -
Dear all, I am working on a project involving the implementation of different parallel programming patterns on Hadoop. Right now I am trying to find what other patterns might be useful for Hadoop ...
Dan Serban Taras
Mar 1, 2011 at 11:57 am
Mar 2, 2011 at 6:14 pm -
Hello all, I'm having trouble getting a large mapping job to complete. Several of thousands of mappers are failing with this error: java.io.FileNotFoundException: File does not exist: ...
Yuval Oren
Mar 22, 2011 at 5:45 pm
Apr 9, 2011 at 9:20 am -
I'd like to get some idea on how the task scheduler relies on RecordReader.getProgress() with version 0.20.2. There are times when I don't have an accurate count of the total records to be processed, ...
Jane Chen
Mar 28, 2011 at 11:11 pm
Apr 9, 2011 at 9:12 am -
Looking under http://svn.apache.org/repos/asf/hadoop/mapreduce/branches/ it does not seem to be present. "pointers" to correct location appreciated.
Stephen Boesch
Mar 28, 2011 at 11:00 pm
Mar 28, 2011 at 11:17 pm -
Hi, in this MR example, it exists the field "Reduce input groups" and "Reduce input records". What's the difference between these 2 fields? $ hadoop jar cloud9.jar ...
Pedro Costa
Mar 25, 2011 at 4:24 pm
Mar 25, 2011 at 9:50 pm -
Hi, during the setup phase and the cleanup phase of the tasks, the Hadoop MR uses map tasks to do it. These tasks appears in the counters shown at the end of an example? For example, the counter ...
Pedro Costa
Mar 25, 2011 at 5:09 pm
Mar 25, 2011 at 9:05 pm -
Hi, when I'm running the Gridmix2 examples, during the execution the tests halt and the following error is displayed: [code] 11/03/23 12:52:06 WARN mapred.JobClient:544 Use GenericOptionsParser for ...
Pedro Costa
Mar 23, 2011 at 12:03 pm
Mar 23, 2011 at 12:08 pm -
HI, How many tasks would assign per TT heartbeat? Is there a parameter for it to configure in hadoop? -- Best regards!
Debbie Fu
Mar 13, 2011 at 2:41 am
Mar 13, 2011 at 4:40 am -
Hi, I'm running hadoop map-reduce in clustering, and I've a Reduce Task that it remains in the state COMMIT_PENDING, and it doesn't finish. This is happening because I've made some changes to the ...
Pedro Costa
Mar 9, 2011 at 9:47 am
Mar 12, 2011 at 6:34 am -
Hi all, I encounter a wired problem while running a job on a cluster of 8 machines(1 master,7 slaves). I am under hadoop 0.21. What I am trying to do is storing the values from the map side into a ...
Exception
Mar 11, 2011 at 7:48 am
Mar 11, 2011 at 8:51 am -
Hello experts, I am recently testing a set of logs that I converted to avro format in hadoop. I am notice really really slow performance when compare to raw logs. The map logs showing below seems to ...
Felix gao
Mar 2, 2011 at 6:26 pm
Mar 2, 2011 at 8:11 pm -
Hi, I am running a mapReduce job that uses up all the processor perfectly, however only 20% of available memory is being used. Is there a way to better utilize all the memory to speed up the ...
Ondřej Nevělík
Mar 1, 2011 at 10:23 am
Mar 1, 2011 at 12:05 pm -
Hi, is there a config param that prevents m/r from deleting the job directory after the job is finished/failed? I need to look into the jar file with which the job is executed. Best regards, PŁ
Paweł Łoziński
Mar 31, 2011 at 1:47 pm
Mar 31, 2011 at 1:47 pm -
Hi all, I need to generate random data with random writer. Can somebody tell me how I can generate less than 10 GB of data in total ? Which options should I configure in the configuration file ? Many ...
Robert Grandl
Mar 25, 2011 at 9:43 am
Mar 25, 2011 at 9:43 am -
Jon Lederman
Mar 21, 2011 at 9:34 pm
Mar 21, 2011 at 9:34 pm -
Hi, In the datanode logs I see an error about Duplicate MetricsName and Error register getProtocolVersion. This may be related to an error in repiicatiion that I encountered earlier. Does anyone have ...
Jon Lederman
Mar 21, 2011 at 4:11 pm
Mar 21, 2011 at 4:11 pm -
Hi, Upon trying a simple copyFromLocal I am receiving an error message "could only be replicated to 0 nodes instead of 1" as indicated below. All of my configuration appears ok and all daemons are ...
Jon Lederman
Mar 21, 2011 at 3:56 pm
Mar 21, 2011 at 3:56 pm -
Hi, I don't know what the examples of the Gridmix do. Where can I find an explanation of that? Thank -- Pedro
Pedro Costa
Mar 18, 2011 at 5:20 pm
Mar 18, 2011 at 5:20 pm -
Hi all, I want to modify WordCount application in order to delay the execution of maps. I have tried to put a sleep in map function but even a 1 ns sleep and for 128 MB blocks it took almost 30 ...
Robert Grandl
Mar 18, 2011 at 12:16 pm
Mar 18, 2011 at 12:16 pm -
I sent this to the HBase mailing list, but thought I would also send this here in case anyone has any idea what might be going on. Thanks ---------- Forwarded message ---------- From: Nichole ...
Nichole Treadway
Mar 17, 2011 at 8:25 pm
Mar 17, 2011 at 8:25 pm -
Hello Fellow Hadoopists, We are meeting at 7:15 PM April 21st at the University Heights Community Center 5031 University Way NE Seattle WA 98105 Room #110 The meetings are informal and highly ...
Sean jensen-grey
Mar 17, 2011 at 3:39 am
Mar 17, 2011 at 3:39 am -
Hi, I was trying to start up a single-node Hadoop cluster using 0.20.2. The namenode, datanode and jobtracker all started fine. The task tracker failed with the following error: 2011-03-16 ...
Jane Chen
Mar 16, 2011 at 9:29 pm
Mar 16, 2011 at 9:29 pm -
[ Moving this to mapreduce-user@ - since it isn't a common-development related question, but a mapreduce specific one || bcc-ing common-dev@ ] Could we additionally know what scheduler is in effect ...
Harsh J
Mar 15, 2011 at 4:43 pm
Mar 15, 2011 at 4:43 pm -
What are we supposed to use instead to submit jobs? I spent all day going through tutorials and documentations and could not find anything. Thanks in advance. Steven.
S Steven Kang
Mar 15, 2011 at 4:13 am
Mar 15, 2011 at 4:13 am -
Hi, I have some types of data that I have to upload on HDFS as Sequence Files. Initially, I had thought of creating a .jr file at runtime depending on the type of schema and use rcc DDL tool by ...
Mapred Learn
Mar 11, 2011 at 11:51 pm
Mar 11, 2011 at 11:51 pm
Group Overview
| group | mapreduce-user |
| categories | hadoop |
| discussions | 57 |
| posts | 146 |
| users | 57 |
| website | hadoop.apache.org... |
| irc | #hadoop |
57 users for March 2011
Archives
- February 2013 (251)
- January 2013 (868)
- December 2012 (621)
- November 2012 (742)
- October 2012 (868)
- September 2012 (733)
- August 2012 (1,082)
- July 2012 (226)
- June 2012 (135)
- May 2012 (102)
- April 2012 (180)
- March 2012 (164)
- February 2012 (167)
- January 2012 (284)
- December 2011 (249)
- November 2011 (201)
- October 2011 (130)
- September 2011 (310)
- August 2011 (168)
- July 2011 (207)
- June 2011 (241)
- May 2011 (225)
- April 2011 (157)
- March 2011 (146)
- February 2011 (174)
- January 2011 (226)
- December 2010 (166)
- November 2010 (135)
- October 2010 (126)
- September 2010 (145)
- August 2010 (128)
- July 2010 (121)
- June 2010 (136)
- May 2010 (82)
- April 2010 (108)
- March 2010 (62)
- February 2010 (59)
- January 2010 (95)
- December 2009 (46)
- November 2009 (45)
- October 2009 (75)
- September 2009 (24)
- August 2009 (30)
- July 2009 (15)