FAQ

Hadoop > mapreduce-user > October 2009

Search Discussions

25 discussions - 75 posts

  • We are using Hadoop 0.20 and mapred.map.output.compression.Codec is set to DefaultCodec. We tried LZO but the performance seems very similar to DefaultCodec. I heard of a lot of good words about LZO. ...
    Zheng ShaoZheng Shao
    Oct 23, 2009 at 7:44 pm
    Oct 26, 2009 at 5:34 pm
  • Hi: all I have a such use case:i have three files,each file is key-value pairs, file1: file2: file3: key1-value1A key1-value1B key1-value1C key2-value2A key2-value2B key2-value2C key3-value3A ...
    AntyAnty
    Oct 26, 2009 at 11:45 am
    Oct 27, 2009 at 2:55 am
  • I'm having problems on 0.20.0 when map output compression is enabled. Map tasks complete (TaskRunner: Task 'attempt_*' done), but it looks like the JVM running the task crashes immediately after. ...
    Ed MazurEd Mazur
    Oct 26, 2009 at 3:37 am
    Oct 27, 2009 at 2:46 pm
  • Hi all, I've been dumping tables from mysql and loading them manually into HDFS, and but decided to look at the DBInputFormat to better automate the process. I see it issuing the "select... from ... ...
    Tim robertsonTim robertson
    Oct 12, 2009 at 2:44 pm
    Oct 16, 2009 at 8:33 pm
  • Hello, fellows! Have a problem with migration to Hadoop 0.20: I need to register MyOutputFormatClass which extends MultipleOutputFormat in Job object. But since Job.setOutputFormatClass(Class<? ...
    Maxim UlanovskiyMaxim Ulanovskiy
    Oct 21, 2009 at 12:01 pm
    Oct 23, 2009 at 1:29 pm
  • Hi all, Using 0.20.1 I have a MultipleTextOutputFormat with the following: protected String generateFileNameForKeyValue(Object key, Object value, String name) { return BASE_FILE + "/resource-" + ...
    Tim robertsonTim robertson
    Oct 27, 2009 at 10:34 am
    Oct 27, 2009 at 10:50 pm
  • Hi all, I am running a simple job working on an input tab file, running the following: - a simple Mapper which reading a field from the tab file row and emitting this as the key and the line as the ...
    Tim robertsonTim robertson
    Oct 27, 2009 at 1:25 pm
    Oct 27, 2009 at 1:52 pm
  • Hi all, I have some question regarding how to compile a simple hadoop program. setup Java 1.6 Ubuntu 9.02 Hadoop 0.19.2 //below is the mapper class import java.io.IOException; import ...
    Felix gaoFelix gao
    Oct 25, 2009 at 10:35 pm
    Oct 26, 2009 at 6:20 pm
  • Hi, Is there a command for JobTracker to reload the log4j settings at runtime? Thanks, Zheng
    Zheng ShaoZheng Shao
    Oct 21, 2009 at 5:41 am
    Oct 21, 2009 at 8:19 am
  • Hi all, I have a Reducer with the following (using new API): public static class Transpose extends Reducer<Text, IntWritable, IntWritable, Text { @Override protected void reduce(Text key, ...
    Tim robertsonTim robertson
    Oct 20, 2009 at 2:37 pm
    Oct 21, 2009 at 7:16 am
  • Hi, I have a situation where I need to "collect" data into some sort of common medium from a set of mapreduce jobs, then have another mapreduce job "consolidate" these to provide the final result. I ...
    Sujit PalSujit Pal
    Oct 16, 2009 at 9:36 pm
    Oct 19, 2009 at 2:50 pm
  • All, I have a MapRed job that ceases to produce output about halfway through. The obvious question is why? This job reads a file and uses MultipleTextOutputFormat to generate output files named with ...
    Geoffry RobertsGeoffry Roberts
    Oct 7, 2009 at 8:52 pm
    Oct 8, 2009 at 3:05 pm
  • Hi, I am using hadoop 0.20.0 release version. I was trying to enable intermediate output compression from code. (libhadoop.so.* was put in /usr/lib) I tried the following and it didnt give me the ...
    Robin AnilRobin Anil
    Oct 22, 2009 at 10:43 pm
    Oct 23, 2009 at 12:33 am
  • Hello, I have a small library that I created that runs fine with a standard Java Main (not on map-reduce) with plenty of heap (i.e. -Xmx2G). My algorithm is written to take advantage of machines with ...
    Matrangola, GeoffreyMatrangola, Geoffrey
    Oct 19, 2009 at 10:33 pm
    Oct 20, 2009 at 2:43 am
  • Hi, I've written custom input format to use with streaming, but I'm having trouble making it work. I pass in -inputformat <input format class and I get the following error: Exception in thread "main" ...
    Keith JacksonKeith Jackson
    Oct 18, 2009 at 2:28 am
    Oct 19, 2009 at 8:10 pm
  • I've been using the Hadoop-0.20.0-eclipse-plugin.jar for use with Eclipse 3.3.2. Is there an equivalent file for 0.20.1 or can I keep on using the 0.20.0 plug-in? (I've searched within ...
    Ratner, Alan S (IS)Ratner, Alan S (IS)
    Oct 12, 2009 at 5:39 pm
    Oct 12, 2009 at 7:27 pm
  • Hi, The same task can be done in different ways, i.e a sort task can be done by merge sort, heap sort etc. Is possible in MapReduce assign different task to the workers? Can the master assign a merge ...
    María Helena Mejía SalazarMaría Helena Mejía Salazar
    Oct 6, 2009 at 4:02 pm
    Oct 6, 2009 at 10:08 pm
  • Hi all, I am trying to learn how to use hadoop and I figured since I am learning might as well know the latest syntax for it. The code for LineIndexer is available online. public class ...
    Felix gaoFelix gao
    Oct 31, 2009 at 12:13 am
    Oct 31, 2009 at 12:13 am
  • Hi all, I'm trying to get the LinuxTaskController working (on the svn trunk) on a pseudo-distributed cluster. It's being quite frustrating. I compiled common, hdfs, and mapred jars with 'ant jar' and ...
    Aaron KimballAaron Kimball
    Oct 23, 2009 at 7:17 pm
    Oct 23, 2009 at 7:17 pm
  • Hello again, I've created a custom Map output class, called Document (which inherits from BinaryComparable), and set up my job the following way... Job zoneJob = new Job(conf, "zone"); ...
    Matrangola, GeoffreyMatrangola, Geoffrey
    Oct 21, 2009 at 2:34 pm
    Oct 21, 2009 at 2:34 pm
  • Hallo Currently I am using Hadoop version 0.19.0. During my Map task operation, I allocate a variety of resources in the form of opening and reading files as well as, in particular, spawn a seperate ...
    Andrich van WykAndrich van Wyk
    Oct 20, 2009 at 6:03 pm
    Oct 20, 2009 at 6:03 pm
  • Hi, I've been using Hadoop for three months on a test cluster with two machines and never had a single problem with it. However, after moving from version 0.20.0 to version 0.20.1, the tasktracker is ...
    Lucas Nazário dos SantosLucas Nazário dos Santos
    Oct 19, 2009 at 11:00 am
    Oct 19, 2009 at 11:00 am
  • Has anyone tried running WordCount in Hadoop 0.20.1 using the -skip option that makes use of the distributed cache? It seems that you can load stuff into the cache but not extract it. This works in ...
    Ratner, Alan S (IS)Ratner, Alan S (IS)
    Oct 12, 2009 at 8:11 pm
    Oct 12, 2009 at 8:11 pm
  • Hey Guys, After a couple of years using hadoop for personal issues, I finally had the chance to scale it to the enterprise. Since now, I really haven't had any need for compressing my files. After ...
    Utku Can TopçuUtku Can Topçu
    Oct 12, 2009 at 8:15 am
    Oct 12, 2009 at 8:15 am
  • Hi, The same task can be done in different ways, i.e a sort task can be done by merge sort, heap sort etc. Is possible in MapReduce assign different task to the workers? Can the master assign a merge ...
    María Helena Mejía SalazarMaría Helena Mejía Salazar
    Oct 6, 2009 at 3:59 pm
    Oct 6, 2009 at 3:59 pm
Oct 2009
Group Navigation
period‹ prev | Oct 2009 | next ›
Group Overview
groupmapreduce-user @
categorieshadoop
discussions25
posts75
users32
websitehadoop.apache.org...
irc#hadoop

32 users for October 2009

Tim robertson: 8 posts Aaron Kimball: 7 posts Anty: 5 posts Todd Lipcon: 5 posts Ed Mazur: 4 posts Zheng Shao: 4 posts Felix gao: 3 posts Jason Venner: 3 posts Amareshwari Sri Ramadasu: 2 posts Amogh Vasekar: 2 posts Geoffry Roberts: 2 posts Jeff Hammerbacher: 2 posts María Helena Mejía Salazar: 2 posts Matrangola, Geoffrey: 2 posts Maxim Ulanovskiy: 2 posts Owen O'Malley: 2 posts Ratner, Alan S (IS): 2 posts Robin Anil: 2 posts Sujit Pal: 2 posts Thomas Thevis: 2 posts
show more
Archives