Search Discussions

43 discussions - 145 posts

  • Hello, I am using the Hadoop MapReduce version 0.20.2 and soon 0.21. I wanted to use the JobClient class to circumvent the use of the command line interface. I am noticed that JobClient still uses ...
    Martin BeckerMartin Becker
    Sep 22, 2010 at 9:38 am
    Sep 27, 2010 at 3:55 pm
  • When running map reduce tasks in Hadoop I run into classpath issues. Contrary to previous posts, my problem is not that I am missing classes on the Task's class path (we have a perfect solution for ...
    Henning BlohmHenning Blohm
    Sep 17, 2010 at 11:56 am
    Oct 8, 2010 at 7:53 am
  • Hello, I am having a problem that might be expected behaviour. I am using a cloud with Hadoop remotely through ssh. I have a program that runs for about a minute, it processes a 200 MB file using ...
    Mario MMario M
    Sep 25, 2010 at 11:56 am
    Sep 27, 2010 at 8:19 am
  • Hi, We deployed Hadoop on several datacenters in EC2. We configed mapred.job.tracker, fs.default.name and slaves by external hostname(DNS)s. We can start the hadoop service, but the ...
    Tao YouTao You
    Sep 13, 2010 at 3:37 pm
    Sep 15, 2010 at 5:55 pm
  • Hello, I am trying to move to Hadoop MapReduce 0.21.0. The corresponding tutorial still uses Tool and ToolRunner. Yet both are deprecated. What would be the correct way to implement, configure and ...
    Martin BeckerMartin Becker
    Sep 22, 2010 at 1:53 pm
    Sep 22, 2010 at 4:12 pm
  • I have a mapper class (extended from mapreduce.Mapper), where setup reports to an outside resource. I want to make sure that most of the time when the task fails or is killed, a specific chunk of ...
    Chase BradfordChase Bradford
    Sep 12, 2010 at 6:39 am
    Sep 13, 2010 at 8:57 pm
  • Hi all, I have a task which involves loading a large amount of data from a database and then using that data to process a large number of small files. I'm trying to split up the file processing via ...
    Angus HelmAngus Helm
    Sep 10, 2010 at 6:19 am
    Sep 13, 2010 at 8:43 pm
  • Hi all, I still have the same problem -- So far I've added some log messages to WordCount.java and JobClient.java with "info" level, but none of them shows in any of the logs (JobTracker, ...
    Rita LiuRita Liu
    Sep 7, 2010 at 5:11 pm
    Sep 8, 2010 at 2:10 pm
  • Hi All, What would be the right approach to solve this problem: 1. I need to output an object as the value from my map method. The object's class should have two mambers: an ArrayList<String and ...
    Bhaskar GhoshBhaskar Ghosh
    Sep 19, 2010 at 10:29 am
    Sep 22, 2010 at 12:02 pm
  • Hi, Can someone please point me to the Maven repo for 0.21 release? Thanks. Thanks and Regards, Sonal www.meghsoft.com http://in.linkedin.com/in/sonalgoyal
    Sonal GoyalSonal Goyal
    Sep 10, 2010 at 8:34 pm
    Sep 13, 2010 at 10:12 am
  • Hi guys, I want to distribute an xml file to the tasks of a MR job and then be able to retrieve it using code like this: DatabaseDescriptor.class.getClassLoader().getResource("storage-conf.xml"); ...
    Cristi CioriiaCristi Cioriia
    Sep 7, 2010 at 1:03 pm
    Sep 8, 2010 at 7:39 am
  • Hi! :) I add some Log4j loggers into the mapper, the reducer, and the main method of WordCount.java. However, after I run this application on the cluster, I couldn't find any of my log messages from ...
    Rita LiuRita Liu
    Sep 6, 2010 at 4:19 am
    Sep 7, 2010 at 5:06 pm
  • Dear Map Reduce users, ------------------------------------------------------------------------ The 11th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing (CCGrid 2011) May ...
    Viraj BhatViraj Bhat
    Sep 26, 2010 at 10:20 pm
    Sep 28, 2010 at 4:29 pm
  • Hello, what does job tracker's status reduce copy (xxx of yyy at 4.55 MB/s) mean? Thanks in Advance, Vitaliy S
    Vitaliy SemochkinVitaliy Semochkin
    Sep 24, 2010 at 11:31 am
    Sep 27, 2010 at 10:46 am
  • Hello, I'm currently trying to write testcases for my mapreduce application and I've read TestMapReduceLocal as an example, so I tried to use MiniMRCluster as well, but it's not included in the ...
    Sep 26, 2010 at 9:24 pm
    Sep 26, 2010 at 9:40 pm
  • Hi, I am using Hadoop MapReduce 0.21.0. The usual process of starting Hadoop/HDFS/MapReduce was to use the "start-all.sh" script. Now when calling that script, it tell me that its usage is deprecated ...
    Martin BeckerMartin Becker
    Sep 22, 2010 at 3:17 pm
    Sep 22, 2010 at 4:56 pm
  • I'm trying to figure out how to achieve the following from a Java client, 1. My app (which is a web server) starts up 2. As part of startup my jar file, which includes my map reduce classes are ...
    Pete TylerPete Tyler
    Sep 15, 2010 at 2:11 am
    Sep 20, 2010 at 3:07 am
  • It seems likely that you are only running one (single-threaded) map or reduce operation per worker node. Do you know whether you are in fact running multiple operations? This also sounds like it may ...
    Christopher ShainChristopher Shain
    Sep 15, 2010 at 2:01 pm
    Sep 15, 2010 at 5:40 pm
  • Does anyone know the difference between the Hadoop counter TOTAL_LAUNCHED_MAPS and the "mapred.map.tasks" parameter available in the JobConf? We're seeing some situations where these two don't match ...
    Elton PintoElton Pinto
    Sep 9, 2010 at 6:43 pm
    Sep 10, 2010 at 6:37 am
  • I'm writing my first m/r program, and seem to be having problems describing the types of my key-value pairs. I have this mapper public static class Map extends Mapper<LongWritable, Text, Text, ...
    Clarence GardnerClarence Gardner
    Sep 8, 2010 at 4:51 am
    Sep 8, 2010 at 3:32 pm
  • Hello, I apologize if this topic has already been brought up, but I was unable to find it by searching around. We recently discovered in issue in one of our jobs where the output of one job does not ...
    Elton PintoElton Pinto
    Sep 2, 2010 at 5:35 pm
    Sep 2, 2010 at 6:56 pm
  • Hi, I'm trying to run the Cloudera hadoop distribution, but it seems it always fails. The log of DataNode: 2010-09-27 15:49:07,081 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG: ...
    Sep 27, 2010 at 1:58 pm
    Sep 28, 2010 at 10:11 am
  • I'm getting an exception when running Hadoop: 10/09/27 14:31:34 ERROR conf.Configuration: Failed to set setXIncludeAware(true) for parser ...
    Sep 27, 2010 at 12:46 pm
    Sep 27, 2010 at 12:52 pm
  • Hi, I have a question regarding the log files generated when running some simple hadoop examples in the jar file I was given. The log files contained the time it took to run the map and reduce tasks, ...
    Han DongHan Dong
    Sep 25, 2010 at 6:11 pm
    Sep 25, 2010 at 6:34 pm
  • Hi, I've a problem using the eclipse plugin... I've started Hadoop/MapReduce 0.20.2 with `bin/start-mapred.sh` or `bin/start-all.sh` and then executed the main class within Eclipse "Run on hadoop", ...
    Sep 25, 2010 at 12:52 pm
    Sep 25, 2010 at 12:55 pm
  • Moving mapreduce-user@, bcc common-user@. Please use the appropriate project lists for discussions. This shouldn't happen in the CS. Maybe you are hitting 'slowstart' for the reduces of Job1? From ...
    Arun C MurthyArun C Murthy
    Sep 13, 2010 at 8:16 pm
    Sep 15, 2010 at 10:55 am
  • Hello, As far as I know, the number of map tasks for "scan-based" mapreduce job is equal (not more than) number of underlying regions (for scan). Of course, if the max map task capacity is big ...
    Alex BaranauAlex Baranau
    Sep 14, 2010 at 5:10 pm
    Sep 14, 2010 at 5:16 pm
  • Hi All, Is there an instruction on how to use rumen? I tried the following command: java -cp ...
    Harold LimHarold Lim
    Sep 13, 2010 at 8:31 pm
    Sep 14, 2010 at 10:11 am
  • Hi :) In the current hadoop-common trunk, bin/start-all.sh has been deprecated. In order to start the cluster, we have to go to hadoop-hdfs trunk and hadoop-mapred trunk and run bin/start-dfs.sh and ...
    Rita LiuRita Liu
    Sep 6, 2010 at 4:34 am
    Sep 6, 2010 at 4:40 am
  • Hi, I am trying to run Hadoop pipes C++ example. I have successfully compiled it with gcc using includes and libs under hadoop-0.21.0/c++/Linux-amd64-64. But when I try to run it, it hangs. Here is ...
    David HaykazyanDavid Haykazyan
    Sep 29, 2010 at 3:46 pm
    Sep 29, 2010 at 3:46 pm
  • (adding mapreduce-user@ and re-scoping title) Can you jstack the client while it's waiting 20 seconds? Is it still waiting for the job to come back or it's something else? Is the job itself done ...
    Jean-Daniel CryansJean-Daniel Cryans
    Sep 27, 2010 at 7:18 pm
    Sep 27, 2010 at 7:18 pm
  • Possible beginner's question here but I can't find an obvious answer in the docs. Is there a way to configure a job such that it imposes a cap on the number of records each reduce process receives at ...
    George P. StathisGeorge P. Stathis
    Sep 27, 2010 at 6:28 pm
    Sep 27, 2010 at 6:28 pm
  • I'm currently implementing JUnit testcases and setup a Cluster like it's done in TestMapReduceLocal: @Test public void testWithLocal() throws Exception { MiniMRCluster mr = null; try { mr = new ...
    Sep 26, 2010 at 10:40 pm
    Sep 26, 2010 at 10:40 pm
  • I want to know if it is possible to write to multiple outputs from a hadoop pipes job. If it possible, any links to relevant document is appreciated. Most of our jobs are in C++ and we use hadoop ...
    George KolaGeorge Kola
    Sep 24, 2010 at 10:43 pm
    Sep 24, 2010 at 10:43 pm
  • Can anyone please help me in unsubscribing from the mailing lists?i send mails to the unsubscribe ids of different groups in response to&nbsp;which i get a confirmation mail, but the addresses on ...
    Kunal GuptaKunal Gupta
    Sep 24, 2010 at 7:15 am
    Sep 24, 2010 at 7:15 am
  • Hi, I continuously run a series of batch job using Hadoop Map Reduce. I also have a managing daemon that moves data around on the hdfs making way for more jobs to be run. I use capacity scheduler to ...
    Aniket rayAniket ray
    Sep 23, 2010 at 4:53 am
    Sep 23, 2010 at 4:53 am
  • ROOM CHANGE TO 211 (one floor up from usual) Hello Fellow Hadoopists, We are meeting at 7:15 pm on September 16th at the University Heights Community Center 5031 University Way NE Seattle WA 98105 ...
    Sean jensen-greySean jensen-grey
    Sep 15, 2010 at 1:02 am
    Sep 15, 2010 at 1:02 am
  • Hi I am using Pig jobs to run on Hadoop but always it runs 4 mappers simultaneously. How can I increase the number of simultaneous mappers to run ? What config do I have to change ? Thanks, Rahul
    Rahul MalviyaRahul Malviya
    Sep 14, 2010 at 5:20 pm
    Sep 14, 2010 at 5:20 pm
  • Hi, Sometimes when I run mumak, I'll get the following exception. I would rerun it with exactly same trace files and eventually it will succeed without any exceptions. Any idea why I'm getting this ...
    Harold LimHarold Lim
    Sep 14, 2010 at 12:40 am
    Sep 14, 2010 at 12:40 am
  • Moving mapreduce-user@, bcc common-user@. Please use the appropriate project lists for discussions. ---- The default scheduler tries to get all tasks of a single job done before moving onto the next ...
    Arun C MurthyArun C Murthy
    Sep 13, 2010 at 10:40 pm
    Sep 13, 2010 at 10:40 pm
  • According to the description of this property, directories that do not exist are ignored. <property <name mapreduce.cluster.local.dir</name <value ${hadoop.tmp.dir}/mapred/local</value <description ...
    Dan AdkinsDan Adkins
    Sep 8, 2010 at 7:21 pm
    Sep 8, 2010 at 7:21 pm
  • I restarted my jobtracker and got the following errors for _all_ active jobs at the time, after which the job history was subsequently deleted. 10/09/08 00:59:58 INFO mapred.JobTracker: Cleaning up ...
    Dan AdkinsDan Adkins
    Sep 8, 2010 at 6:28 pm
    Sep 8, 2010 at 6:28 pm
  • Hi, The last few weeks we built an application using Hadoop. Because we're implementing against special logfiles (line oriented, textual and gzipped) and we wanted to extract specific fields from ...
    Niels BasjesNiels Basjes
    Sep 7, 2010 at 10:02 pm
    Sep 7, 2010 at 10:02 pm
Group Navigation
period‹ prev | Sep 2010 | next ›
Group Overview
groupmapreduce-user @

55 users for September 2010

Martin Becker: 16 posts David Rosenstrauch: 9 posts Ted Yu: 9 posts Johannes.Lichtenberger: 8 posts Tom White: 8 posts Rita Liu: 7 posts Mario M: 6 posts Chase Bradford: 4 posts Elton Pinto: 4 posts Christopher Shain: 3 posts Arun C Murthy: 3 posts Bhaskar Ghosh: 3 posts Henning Blohm: 3 posts Jeff Hammerbacher: 3 posts Ratner, Alan S (IS): 3 posts Tao You: 3 posts Allen Wittenauer: 2 posts Aniket ray: 2 posts Clarence Gardner: 2 posts Cristi Cioriia: 2 posts
show more