FAQ

Search Discussions

36 discussions - 126 posts

  • Hello, I'm getting a ClassCastException, when running my application: 10/10/07 04:31:10 WARN mapred.LocalJobRunner: job_local_0001 java.lang.ClassCastException: class java.util.Date at ...
    Johannes.LichtenbergerJohannes.Lichtenberger
    Oct 7, 2010 at 3:03 am
    Oct 8, 2010 at 1:30 am
  • Hi, I want to modify the heartbeat message to carry more information from worker nodes to master node, and also want to modify the return message of heartbeat. Do you know which file and function ...
    Shen LIShen LI
    Oct 27, 2010 at 9:29 pm
    Oct 28, 2010 at 5:17 pm
  • Hi, is it possible to reverse the sort-order? I mean, it seems the reducers are getting the output in descending order. As I want to sort wikipedia revisions of articles it would be great in ...
    Johannes.LichtenbergerJohannes.Lichtenberger
    Oct 20, 2010 at 10:12 pm
    Oct 26, 2010 at 1:07 pm
  • Hi, How can I modify the task assignment strategy in hadoop which is used to assign tasks to different worker nodes? (Not the job scheduler) Big thanks, Shen
    Shen LIShen LI
    Oct 7, 2010 at 6:58 pm
    Oct 14, 2010 at 8:17 pm
  • Hi, My MR Job is processing gzipped files each around 450 MB and there are 24 of them. File block size is 512 MB. This job is failing consistently in the reduce phase with the following exception ...
    Rakesh kothariRakesh kothari
    Oct 12, 2010 at 7:53 pm
    Oct 13, 2010 at 3:59 pm
  • Hi forks, I am under hadoop 0.21.0. What I am trying to do is loading the mapper/reducer class at runtime according to a config file. I use -libjars to shipping JAR files that a job is dependent on. ...
    ExceptionException
    Oct 28, 2010 at 3:33 am
    Nov 1, 2010 at 2:17 pm
  • I would want to initialize a Map class with some parameters. How do I do that ? -Muncher
    Web serviceWeb service
    Oct 17, 2010 at 6:32 pm
    Oct 18, 2010 at 7:55 am
  • Hi, Is there any way to pass system properties, like java.library.path to each mapper/reducer? Thanks, Yin
    Yin LouYin Lou
    Oct 7, 2010 at 7:04 pm
    Oct 8, 2010 at 10:43 pm
  • Hi Folks I am wondering if anyone has the answer of this question. I am processing log files using Map reduce and get data to put some part into mysql and rest of hbase. At the moment, i am running ...
    Shuja RehmanShuja Rehman
    Oct 27, 2010 at 8:45 pm
    Oct 27, 2010 at 9:16 pm
  • Hi, Is "move" not supported in Hdfs ? I can't find any API for that. Looking at the source code for hadoop CLI it seems like it's implementing move by copying data from src to dest and deleting the ...
    Rakesh kothariRakesh kothari
    Oct 21, 2010 at 7:27 pm
    Oct 21, 2010 at 7:52 pm
  • I have a simple map-reduce program, which runs fine under eclipse. However when I execute it using hadoop, it gives me an out of memory error. Hadoop_heapsize is 2000MB Not sure what the problem is.
    Web serviceWeb service
    Oct 19, 2010 at 4:03 pm
    Oct 20, 2010 at 2:08 am
  • Hi, What's the way to access files copied to distributed cache from the map tasks ? e.g. if I run my M/R job as $hadoop jar my.jar -files hdfs://path/to/my/file.txt, How can I access file.txt in my ...
    Rakesh kothariRakesh kothari
    Oct 19, 2010 at 8:03 pm
    Oct 19, 2010 at 8:16 pm
  • Hi, I would like to create an example to read an index file and the data file that is produced as output in the map function. Can anyone give me an example, please? Thanks, -- Pedro
    Pedro CostaPedro Costa
    Oct 13, 2010 at 7:00 pm
    Oct 14, 2010 at 7:20 pm
  • Hello, I'm currently getting a NullPointerException: java.lang.NullPointerException at org.apache.hadoop.io.serializer.SerializationFactory.getSerializer(SerializationFactory.java:73) at ...
    Johannes.LichtenbergerJohannes.Lichtenberger
    Oct 8, 2010 at 12:02 pm
    Oct 8, 2010 at 7:45 pm
  • Hello All, I am trying to load a conf file located in hdfs. I was hoping following would work: Path apath = new Path(conf.get("fs.default.name"), "/home/shrijeet/blah.xml"); conf.addResource(apath); ...
    Shrijeet PaliwalShrijeet Paliwal
    Oct 29, 2010 at 7:18 pm
    Oct 29, 2010 at 8:17 pm
  • Hi, When launching a job in hadoop, usually we use "hadoop jar xxx.jar input output". Can I run a job simply by a java program (In fully distributed mode), without packing a jar. I know this will ...
    ExceptionException
    Oct 26, 2010 at 7:03 am
    Oct 26, 2010 at 2:14 pm
  • Hi, In running hadoop, is it possible to specify the number of computing nodes to use? Or does hadoop automatically configures to run on different nodes? For example, if I specify 12 map tasks to ...
    Han DongHan Dong
    Oct 25, 2010 at 11:05 pm
    Oct 26, 2010 at 6:05 am
  • Hi Users, How do I change the log4j logging level of task attempts (the information dumped in syslog) to DEBUG? I tried setting log4j.logger.org.apache.hadoop.mapred.TaskTracker to Debug, but that ...
    Shrijeet PaliwalShrijeet Paliwal
    Oct 12, 2010 at 6:35 pm
    Oct 13, 2010 at 5:35 pm
  • Hi, The Hadoop Definitive Guide book states that "if your input files are compressed, they will be automatically decompressed as they are read by MapReduce, using the filename extension to determine ...
    Patrick MarchwiakPatrick Marchwiak
    Oct 8, 2010 at 8:59 pm
    Oct 8, 2010 at 11:58 pm
  • Hi, I wonder how MultipleOutputFormat works. Suppose my reducer <k1, list(v1) - <k2, v2 . Does the method "generateFileNameForKeyValue" depends on k2 or k1? Is there anyway to generate file name ...
    Yin LouYin Lou
    Oct 31, 2010 at 6:32 pm
    Oct 31, 2010 at 9:09 pm
  • [Apologies for cross-posting] HI all, I am rewriting a hadoop java code for the new (0.20.2) API- the code was originally written for versions <= 0.19. 1. What is the equivalent of getPos() method ...
    Bibek PaudelBibek Paudel
    Oct 27, 2010 at 9:38 am
    Oct 27, 2010 at 9:48 am
  • Is there a configuration property or some other mechanism for changing how reduce tasks report progress? E.g., instead of the shuffle, merge, and reduce each counting for 33%, can this be changed? ...
    Ed MazurEd Mazur
    Oct 21, 2010 at 4:34 am
    Oct 21, 2010 at 5:46 am
  • How do I output log statements more specifically meant for debugging ? sysout don't help ... where exactly would be logs stored ? the output/_logs directory ? Muncher.
    Web serviceWeb service
    Oct 17, 2010 at 6:36 pm
    Oct 18, 2010 at 7:48 am
  • Hi, I use Hadoop 0.20.3-dev on Ubuntu. I use it in pseudo-distributed mode in a single node cluster. I have already run mapreduce programs for wordcount and building inverted index. I am trying to ...
    Bibek PaudelBibek Paudel
    Oct 8, 2010 at 11:48 am
    Oct 12, 2010 at 11:49 am
  • Is there a reason why block size should be set to some 2^N, for some integer N ? Does it help with block defragmentation etc. ? Thanks, -Rakesh
    Rakesh kothariRakesh kothari
    Oct 7, 2010 at 6:04 pm
    Oct 8, 2010 at 1:31 am
  • Hi, I am using hadoop0.20.1 Try to get the filename, but it didn't work by following codes. Any suggestion? FileSplit fileSplit = (FileSplit) context.getInputSplit(); String fileName = ...
    Y_823910Y_823910
    Oct 4, 2010 at 7:40 am
    Oct 4, 2010 at 7:48 am
  • Hi, What's the difference between a map output and a spill in Hadoop MapReduce? Thanks, -- Pedro
    Pedro CostaPedro Costa
    Oct 21, 2010 at 2:51 pm
    Oct 21, 2010 at 2:51 pm
  • Hi there, I wonder our oracle stored procedure is a good Hadoop M/R use case. That procedure starts with a parameter like (productid) then it will refer(do join) 32 tables. Like Bill of Materail(BOM) ...
    HmchiudHmchiud
    Oct 21, 2010 at 5:54 am
    Oct 21, 2010 at 5:54 am
  • Hello, I'm currently not sure what happens to records, when the block-size is reached. Let's assume a block-size of 128MB and my XMLRecordReader implementation splits on a specific element. I assume ...
    Johannes.LichtenbergerJohannes.Lichtenberger
    Oct 20, 2010 at 10:10 pm
    Oct 20, 2010 at 10:10 pm
  • Hi, I need some guidance to make RMI call from the map/reduce job, basically connect to "localhost" RMI server. This is usually involve setting security manager and providing policy file that allows ...
    Maksim LikharevMaksim Likharev
    Oct 20, 2010 at 3:13 am
    Oct 20, 2010 at 3:13 am
  • Hello Fellow Hadoopists, We are meeting at 7:15 pm on October 21st at the University Heights Community Center 5031 University Way NE Seattle WA 98105 Room #110 The meetings are informal and highly ...
    Sean jensen-greySean jensen-grey
    Oct 15, 2010 at 3:10 am
    Oct 15, 2010 at 3:10 am
  • Hi, Is there a way to disable the clean task running in the map reduce? Thanks, -- Pedro
    Pedro CostaPedro Costa
    Oct 14, 2010 at 2:26 pm
    Oct 14, 2010 at 2:26 pm
  • Hi, Hadoop MR has property that allows to preserve the temporary files that were created during an MR execution? Thanks -- Pedro
    Pedro CostaPedro Costa
    Oct 10, 2010 at 4:02 pm
    Oct 10, 2010 at 4:02 pm
  • Hi, 1 - In hadoop MapReduce which property should be defined in mapred-site.xml to keep the temporary files after a MR execution? -- Pedro
    Pedro CostaPedro Costa
    Oct 8, 2010 at 9:25 am
    Oct 8, 2010 at 9:25 am
  • Hi there, My files, each about 40G, content format like: <start datadatadatadatadata datadatadatadatadata datadatadatadatadata <end <start datadatadatadatadata datadatadatadatadata ...
    HmchiudHmchiud
    Oct 8, 2010 at 12:15 am
    Oct 8, 2010 at 12:15 am
  • Hello, How do I run my Hadoop application within Eclipse? I've compiled the plugin, but the following exception occurs (if run as "run on hadoop" or "java application"). 10/10/07 05:03:11 WARN ...
    Johannes.LichtenbergerJohannes.Lichtenberger
    Oct 7, 2010 at 3:07 am
    Oct 7, 2010 at 3:07 am
Group Navigation
period‹ prev | Oct 2010 | next ›
Group Overview
groupmapreduce-user @
categorieshadoop
discussions36
posts126
users39
websitehadoop.apache.org...
irc#hadoop

39 users for October 2010

Johannes.Lichtenberger: 17 posts Shrijeet Paliwal: 12 posts Ted Yu: 9 posts Rakesh kothari: 7 posts Yin Lou: 7 posts Pedro Costa: 6 posts Shen LI: 6 posts Alejandro Abdelnur: 4 posts Harsh J: 4 posts Henning Blohm: 4 posts Jeff Zhang: 4 posts Web service: 4 posts Bibek Paudel: 3 posts Exception: 3 posts Tom White: 3 posts Hmchiud: 2 posts Anthony Urso: 2 posts David Rosenstrauch: 2 posts Gregory Lawrence: 2 posts Jean-Daniel Cryans: 2 posts
show more