Search Discussions
-
Hello, I am using the Hadoop MapReduce version 0.20.2 and soon 0.21. I wanted to use the JobClient class to circumvent the use of the command line interface. I am noticed that JobClient still uses ...
Martin Becker
Sep 22, 2010 at 9:38 am
Sep 27, 2010 at 3:55 pm -
When running map reduce tasks in Hadoop I run into classpath issues. Contrary to previous posts, my problem is not that I am missing classes on the Task's class path (we have a perfect solution for ...
Henning Blohm
Sep 17, 2010 at 11:56 am
Oct 8, 2010 at 7:53 am -
Hello, I am having a problem that might be expected behaviour. I am using a cloud with Hadoop remotely through ssh. I have a program that runs for about a minute, it processes a 200 MB file using ...
Mario M
Sep 25, 2010 at 11:56 am
Sep 27, 2010 at 8:19 am -
Hi, We deployed Hadoop on several datacenters in EC2. We configed mapred.job.tracker, fs.default.name and slaves by external hostname(DNS)s. We can start the hadoop service, but the ...
Tao You
Sep 13, 2010 at 3:37 pm
Sep 15, 2010 at 5:55 pm -
Hello, I am trying to move to Hadoop MapReduce 0.21.0. The corresponding tutorial still uses Tool and ToolRunner. Yet both are deprecated. What would be the correct way to implement, configure and ...
Martin Becker
Sep 22, 2010 at 1:53 pm
Sep 22, 2010 at 4:12 pm -
I have a mapper class (extended from mapreduce.Mapper), where setup reports to an outside resource. I want to make sure that most of the time when the task fails or is killed, a specific chunk of ...
Chase Bradford
Sep 12, 2010 at 6:39 am
Sep 13, 2010 at 8:57 pm -
Hi all, I have a task which involves loading a large amount of data from a database and then using that data to process a large number of small files. I'm trying to split up the file processing via ...
Angus Helm
Sep 10, 2010 at 6:19 am
Sep 13, 2010 at 8:43 pm -
Hi all, I still have the same problem -- So far I've added some log messages to WordCount.java and JobClient.java with "info" level, but none of them shows in any of the logs (JobTracker, ...
Rita Liu
Sep 7, 2010 at 5:11 pm
Sep 8, 2010 at 2:10 pm -
Hi All, What would be the right approach to solve this problem: 1. I need to output an object as the value from my map method. The object's class should have two mambers: an ArrayList<String and ...
Bhaskar Ghosh
Sep 19, 2010 at 10:29 am
Sep 22, 2010 at 12:02 pm -
Hi, Can someone please point me to the Maven repo for 0.21 release? Thanks. Thanks and Regards, Sonal www.meghsoft.com http://in.linkedin.com/in/sonalgoyal
Sonal Goyal
Sep 10, 2010 at 8:34 pm
Sep 13, 2010 at 10:12 am -
Hi guys, I want to distribute an xml file to the tasks of a MR job and then be able to retrieve it using code like this: DatabaseDescriptor.class.getClassLoader().getResource("storage-conf.xml"); ...
Cristi Cioriia
Sep 7, 2010 at 1:03 pm
Sep 8, 2010 at 7:39 am -
Hi! :) I add some Log4j loggers into the mapper, the reducer, and the main method of WordCount.java. However, after I run this application on the cluster, I couldn't find any of my log messages from ...
Rita Liu
Sep 6, 2010 at 4:19 am
Sep 7, 2010 at 5:06 pm -
Dear Map Reduce users, ------------------------------------------------------------------------ The 11th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing (CCGrid 2011) May ...
Viraj Bhat
Sep 26, 2010 at 10:20 pm
Sep 28, 2010 at 4:29 pm -
Hello, what does job tracker's status reduce copy (xxx of yyy at 4.55 MB/s) mean? Thanks in Advance, Vitaliy S
Vitaliy Semochkin
Sep 24, 2010 at 11:31 am
Sep 27, 2010 at 10:46 am -
Hello, I'm currently trying to write testcases for my mapreduce application and I've read TestMapReduceLocal as an example, so I tried to use MiniMRCluster as well, but it's not included in the ...
Johannes.Lichtenberger
Sep 26, 2010 at 9:24 pm
Sep 26, 2010 at 9:40 pm -
Hi, I am using Hadoop MapReduce 0.21.0. The usual process of starting Hadoop/HDFS/MapReduce was to use the "start-all.sh" script. Now when calling that script, it tell me that its usage is deprecated ...
Martin Becker
Sep 22, 2010 at 3:17 pm
Sep 22, 2010 at 4:56 pm -
I'm trying to figure out how to achieve the following from a Java client, 1. My app (which is a web server) starts up 2. As part of startup my jar file, which includes my map reduce classes are ...
Pete Tyler
Sep 15, 2010 at 2:11 am
Sep 20, 2010 at 3:07 am -
It seems likely that you are only running one (single-threaded) map or reduce operation per worker node. Do you know whether you are in fact running multiple operations? This also sounds like it may ...
Christopher Shain
Sep 15, 2010 at 2:01 pm
Sep 15, 2010 at 5:40 pm -
Does anyone know the difference between the Hadoop counter TOTAL_LAUNCHED_MAPS and the "mapred.map.tasks" parameter available in the JobConf? We're seeing some situations where these two don't match ...
Elton Pinto
Sep 9, 2010 at 6:43 pm
Sep 10, 2010 at 6:37 am -
I'm writing my first m/r program, and seem to be having problems describing the types of my key-value pairs. I have this mapper public static class Map extends Mapper<LongWritable, Text, Text, ...
Clarence Gardner
Sep 8, 2010 at 4:51 am
Sep 8, 2010 at 3:32 pm -
Hello, I apologize if this topic has already been brought up, but I was unable to find it by searching around. We recently discovered in issue in one of our jobs where the output of one job does not ...
Elton Pinto
Sep 2, 2010 at 5:35 pm
Sep 2, 2010 at 6:56 pm -
Hi, I'm trying to run the Cloudera hadoop distribution, but it seems it always fails. The log of DataNode: 2010-09-27 15:49:07,081 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG: ...
Johannes.Lichtenberger
Sep 27, 2010 at 1:58 pm
Sep 28, 2010 at 10:11 am -
I'm getting an exception when running Hadoop: 10/09/27 14:31:34 ERROR conf.Configuration: Failed to set setXIncludeAware(true) for parser ...
Johannes.Lichtenberger
Sep 27, 2010 at 12:46 pm
Sep 27, 2010 at 12:52 pm -
Hi, I have a question regarding the log files generated when running some simple hadoop examples in the jar file I was given. The log files contained the time it took to run the map and reduce tasks, ...
Han Dong
Sep 25, 2010 at 6:11 pm
Sep 25, 2010 at 6:34 pm -
Hi, I've a problem using the eclipse plugin... I've started Hadoop/MapReduce 0.20.2 with `bin/start-mapred.sh` or `bin/start-all.sh` and then executed the main class within Eclipse "Run on hadoop", ...
Johannes.Lichtenberger
Sep 25, 2010 at 12:52 pm
Sep 25, 2010 at 12:55 pm -
Moving mapreduce-user@, bcc common-user@. Please use the appropriate project lists for discussions. This shouldn't happen in the CS. Maybe you are hitting 'slowstart' for the reduces of Job1? From ...
Arun C Murthy
Sep 13, 2010 at 8:16 pm
Sep 15, 2010 at 10:55 am -
Hello, As far as I know, the number of map tasks for "scan-based" mapreduce job is equal (not more than) number of underlying regions (for scan). Of course, if the max map task capacity is big ...
Alex Baranau
Sep 14, 2010 at 5:10 pm
Sep 14, 2010 at 5:16 pm -
Hi All, Is there an instruction on how to use rumen? I tried the following command: java -cp ...
Harold Lim
Sep 13, 2010 at 8:31 pm
Sep 14, 2010 at 10:11 am -
Hi :) In the current hadoop-common trunk, bin/start-all.sh has been deprecated. In order to start the cluster, we have to go to hadoop-hdfs trunk and hadoop-mapred trunk and run bin/start-dfs.sh and ...
Rita Liu
Sep 6, 2010 at 4:34 am
Sep 6, 2010 at 4:40 am -
Hi, I am trying to run Hadoop pipes C++ example. I have successfully compiled it with gcc using includes and libs under hadoop-0.21.0/c++/Linux-amd64-64. But when I try to run it, it hangs. Here is ...
David Haykazyan
Sep 29, 2010 at 3:46 pm
Sep 29, 2010 at 3:46 pm -
(adding mapreduce-user@ and re-scoping title) Can you jstack the client while it's waiting 20 seconds? Is it still waiting for the job to come back or it's something else? Is the job itself done ...
Jean-Daniel Cryans
Sep 27, 2010 at 7:18 pm
Sep 27, 2010 at 7:18 pm -
Possible beginner's question here but I can't find an obvious answer in the docs. Is there a way to configure a job such that it imposes a cap on the number of records each reduce process receives at ...
George P. Stathis
Sep 27, 2010 at 6:28 pm
Sep 27, 2010 at 6:28 pm -
I'm currently implementing JUnit testcases and setup a Cluster like it's done in TestMapReduceLocal: @Test public void testWithLocal() throws Exception { MiniMRCluster mr = null; try { mr = new ...
Johannes.Lichtenberger
Sep 26, 2010 at 10:40 pm
Sep 26, 2010 at 10:40 pm -
I want to know if it is possible to write to multiple outputs from a hadoop pipes job. If it possible, any links to relevant document is appreciated. Most of our jobs are in C++ and we use hadoop ...
George Kola
Sep 24, 2010 at 10:43 pm
Sep 24, 2010 at 10:43 pm -
Can anyone please help me in unsubscribing from the mailing lists?i send mails to the unsubscribe ids of different groups in response to which i get a confirmation mail, but the addresses on ...
Kunal Gupta
Sep 24, 2010 at 7:15 am
Sep 24, 2010 at 7:15 am -
Hi, I continuously run a series of batch job using Hadoop Map Reduce. I also have a managing daemon that moves data around on the hdfs making way for more jobs to be run. I use capacity scheduler to ...
Aniket ray
Sep 23, 2010 at 4:53 am
Sep 23, 2010 at 4:53 am -
ROOM CHANGE TO 211 (one floor up from usual) Hello Fellow Hadoopists, We are meeting at 7:15 pm on September 16th at the University Heights Community Center 5031 University Way NE Seattle WA 98105 ...
Sean jensen-grey
Sep 15, 2010 at 1:02 am
Sep 15, 2010 at 1:02 am -
Hi I am using Pig jobs to run on Hadoop but always it runs 4 mappers simultaneously. How can I increase the number of simultaneous mappers to run ? What config do I have to change ? Thanks, Rahul
Rahul Malviya
Sep 14, 2010 at 5:20 pm
Sep 14, 2010 at 5:20 pm -
Hi, Sometimes when I run mumak, I'll get the following exception. I would rerun it with exactly same trace files and eventually it will succeed without any exceptions. Any idea why I'm getting this ...
Harold Lim
Sep 14, 2010 at 12:40 am
Sep 14, 2010 at 12:40 am -
Moving mapreduce-user@, bcc common-user@. Please use the appropriate project lists for discussions. ---- The default scheduler tries to get all tasks of a single job done before moving onto the next ...
Arun C Murthy
Sep 13, 2010 at 10:40 pm
Sep 13, 2010 at 10:40 pm -
According to the description of this property, directories that do not exist are ignored. <property <name mapreduce.cluster.local.dir</name <value ${hadoop.tmp.dir}/mapred/local</value <description ...
Dan Adkins
Sep 8, 2010 at 7:21 pm
Sep 8, 2010 at 7:21 pm -
I restarted my jobtracker and got the following errors for _all_ active jobs at the time, after which the job history was subsequently deleted. 10/09/08 00:59:58 INFO mapred.JobTracker: Cleaning up ...
Dan Adkins
Sep 8, 2010 at 6:28 pm
Sep 8, 2010 at 6:28 pm -
Hi, The last few weeks we built an application using Hadoop. Because we're implementing against special logfiles (line oriented, textual and gzipped) and we wanted to extract specific fields from ...
Niels Basjes
Sep 7, 2010 at 10:02 pm
Sep 7, 2010 at 10:02 pm
Group Overview
| group | mapreduce-user |
| categories | hadoop |
| discussions | 43 |
| posts | 145 |
| users | 55 |
| website | hadoop.apache.org... |
| irc | #hadoop |
55 users for September 2010
Archives
- February 2013 (251)
- January 2013 (868)
- December 2012 (621)
- November 2012 (742)
- October 2012 (868)
- September 2012 (733)
- August 2012 (1,082)
- July 2012 (226)
- June 2012 (135)
- May 2012 (102)
- April 2012 (180)
- March 2012 (164)
- February 2012 (167)
- January 2012 (284)
- December 2011 (249)
- November 2011 (201)
- October 2011 (130)
- September 2011 (310)
- August 2011 (168)
- July 2011 (207)
- June 2011 (241)
- May 2011 (225)
- April 2011 (157)
- March 2011 (146)
- February 2011 (174)
- January 2011 (226)
- December 2010 (166)
- November 2010 (135)
- October 2010 (126)
- September 2010 (145)
- August 2010 (128)
- July 2010 (121)
- June 2010 (136)
- May 2010 (82)
- April 2010 (108)
- March 2010 (62)
- February 2010 (59)
- January 2010 (95)
- December 2009 (46)
- November 2009 (45)
- October 2009 (75)
- September 2009 (24)
- August 2009 (30)
- July 2009 (15)