Search Discussions

57 discussions - 146 posts

  • Hi Everyone, I make a Project with Hadoop-MapRedeuce for my master-Thesis. I have a strange problem on my System. First of all, I use Hadoop-0.20.2 on Windows XP Pro with Eclipse Plug-In. When I ...
    Baran cakiciBaran cakici
    Mar 16, 2011 at 4:50 pm
    Oct 6, 2011 at 4:29 pm
  • Hi, Is there a good tutorial for writing custom InputFormat classes? Any help would be greatly appreciated. Thanks, Souri
    Souri dattaSouri datta
    Mar 4, 2011 at 5:58 pm
    Mar 6, 2011 at 7:51 am
  • I am not sure if this is the right listserv, forgive me if it is not. My goal is this: monitor HDFS until a file is create, and then kick off a job. Ideally I'd want to do this continuously, but the ...
    Jonathan CoveneyJonathan Coveney
    Mar 24, 2011 at 5:10 pm
    May 15, 2011 at 10:10 pm
  • Hi mapreduce gurus - Today while looking into a few TaskTracker's with full disks I came across the following directory using 207GB: ...
    Travis CrawfordTravis Crawford
    Mar 31, 2011 at 6:45 pm
    Apr 1, 2011 at 9:12 pm
  • Hi, I'm trying to increase the job-tracker heap by adding Xmx definition to hadoop-env.sh (adding it to HADOOP_TASKTRACKER_OPTS). I keep getting this error while starting hadoop: Invalid maximum heap ...
    Lior SchachterLior Schachter
    Mar 17, 2011 at 4:38 pm
    Mar 17, 2011 at 6:19 pm
  • Hi, I am working on a problem to compare two different datasets, and rank each record of the first with respect to the other, in terms of how similar they are. The records are dimensional, but do not ...
    Sonal GoyalSonal Goyal
    Mar 7, 2011 at 7:07 pm
    Mar 10, 2011 at 6:39 am
  • Hi, I downloaded CDH3 VM for hadoop but if I want to use something like: import org.apache.hadoop.conf.Configuration; in my java code, what else do I need to do ? Do i need to download hadoop from ...
    Mapred LearnMapred Learn
    Mar 8, 2011 at 12:22 am
    Mar 9, 2011 at 3:50 am
  • Re: Mapper and Reducer of Terasort examplehi, i inadvertently got onto this list and it's not all that relevant for me, can anyone suggest how i get off it? Many thanks ----- Original Message ----- ...
    David HeneghanDavid Heneghan
    Mar 31, 2011 at 1:56 pm
    Apr 1, 2011 at 8:23 am
  • Hi, I can't seem to be able to find either jira or implementation of MultipleOutputFormat in new api in either 0.21 or 0.22 branches. Are there any plans to port that to new api as well? thanks in ...
    Dmitriy LyubimovDmitriy Lyubimov
    Mar 30, 2011 at 3:40 am
    Mar 30, 2011 at 4:13 am
  • Hi, is it possible to configure the total number of slots that a TaskTracker has, to run the map and reduce tasks? Thanks, -- Pedro
    Pedro CostaPedro Costa
    Mar 25, 2011 at 6:26 pm
    Mar 25, 2011 at 7:25 pm
  • Hi all , I am a beginner to hadoop . I am passing a FileName as the Key and the Contents of he FIle as the Value I want to write the contents of the FIle to a text file with Key as the fileName, from ...
    Dinesh shanDinesh shan
    Mar 24, 2011 at 12:19 pm
    Mar 24, 2011 at 4:03 pm
  • Hi What's the purpose of the parameter "mapred.min.split.size"? Thanks, -- Pedro
    Pedro CostaPedro Costa
    Mar 18, 2011 at 7:55 pm
    Mar 18, 2011 at 10:59 pm
  • Hi, I want to know that io.sort.mb and io.sort.factor are properties that set for the whole cluster, or can be set differently per node? -- Best regards!
    Debbie FuDebbie Fu
    Mar 14, 2011 at 8:27 am
    Mar 14, 2011 at 9:38 am
  • Hi all I have a hadoop cluster with a namenode and 3 datanodes, I've installed it by normal process. everything's fine, but it couldn't run wordcount map reduce job. Follow are output logs ...
    Yorgo SunYorgo Sun
    Mar 14, 2011 at 4:12 am
    Mar 14, 2011 at 6:41 am
  • We see a lot of IOExceptions coming from HDFS during a job that does nothing but untar 100 files (1 per Mapper, sizes vary between 5GB and 80GB) that are in HDFS, to HDFS. DataNodes are also showing ...
    Evert LammertsEvert Lammerts
    Mar 9, 2011 at 11:30 am
    Mar 9, 2011 at 4:57 pm
  • Hi everyone, I'm experiencing a chain of runtime exceptions which I believe starts with a ClassNotFoundException for a Trove library class I'm using in my custom jar. Here's a copy of the error ...
    John TherrellJohn Therrell
    Mar 31, 2011 at 10:58 pm
    Apr 9, 2011 at 8:33 am
  • I tried to run hadoop on multi-node cluster. Each node installs hadoop on different path. But not success Is it possible to run hadoop on multi-node cluster and these nodes install hadoop on ...
    Zhengjun chenZhengjun chen
    Mar 3, 2011 at 2:36 pm
    Apr 7, 2011 at 2:56 am
  • I'm looking to get acquainted with the new API in 0.20.2 but all the online documentation I've found uses the old API. I need to understand how to chain two mapreduce jobs together efficiently that ...
    John TherrellJohn Therrell
    Mar 30, 2011 at 6:09 pm
    Apr 6, 2011 at 8:28 pm
  • Hi, We recently did some experiment on mapreduce job scheduling and found that sometimes there were 2 jobs running on the same machine and each of them ran very slowly. We used to think that 2nd job ...
    Wisteria_longyan=126 ComWisteria_longyan=126 Com
    Mar 29, 2011 at 2:21 pm
    Mar 29, 2011 at 10:00 pm
  • Hi, We have a map/reduce task that insert to hbase (in the reduce phase). Our problem is that some reduce jobs finish early and then the framework use them in-order to "backup" running reducers (As ...
    Lior SchachterLior Schachter
    Mar 28, 2011 at 3:02 pm
    Mar 28, 2011 at 8:38 pm
  • Hi, 1 - A Task is composed by several phases: STARTING, MAP/REDUCE, SHUFFLE, SORT, CLEANUP. A JobTracker and a TaskTracker also has phases? 2 - It exists also the following phases RUNNING, SUCCEEDED, ...
    Pedro CostaPedro Costa
    Mar 24, 2011 at 6:38 pm
    Mar 24, 2011 at 6:47 pm
  • Hello, I have a hadoop cluster that has 10 nodes. I use both of the hdfs/mapred(0.20.2) and the hbase(0.20.6). Every node is both of a datanode and a tasktracker. But I just configured 9 nodes as the ...
    Jameson LiJameson Li
    Mar 21, 2011 at 11:31 am
    Mar 22, 2011 at 2:46 am
  • Hi, I would like define the number of map tasks to use in the GridMix2. For example, I would like to run the GridMixMonsterQuery at GridMix2 with 5 maps, another with 10 and another with 20 maps. How ...
    Pedro CostaPedro Costa
    Mar 18, 2011 at 5:05 pm
    Mar 19, 2011 at 2:02 am
  • Hi , I am trying to run a job which does not require reducer, I commented out the reducer configuration from the job, but now its throwing following error java.lang.ClassCastException: class ...
    Sagar KohliSagar Kohli
    Mar 11, 2011 at 5:53 am
    Mar 11, 2011 at 7:32 am
  • Hi I am new to Hadoop, so maybe I am missing something obvious. I have written a small map reduce program that runs two jobs. I want the output of the first job to serve as the input to the second ...
    John SandaJohn Sanda
    Mar 3, 2011 at 2:22 am
    Mar 3, 2011 at 4:48 am
  • Dear all, I am working on a project involving the implementation of different parallel programming patterns on Hadoop. Right now I am trying to find what other patterns might be useful for Hadoop ...
    Dan Serban TarasDan Serban Taras
    Mar 1, 2011 at 11:57 am
    Mar 2, 2011 at 6:14 pm
  • Hello all, I'm having trouble getting a large mapping job to complete. Several of thousands of mappers are failing with this error: java.io.FileNotFoundException: File does not exist: ...
    Yuval OrenYuval Oren
    Mar 22, 2011 at 5:45 pm
    Apr 9, 2011 at 9:20 am
  • I'd like to get some idea on how the task scheduler relies on RecordReader.getProgress() with version 0.20.2. There are times when I don't have an accurate count of the total records to be processed, ...
    Jane ChenJane Chen
    Mar 28, 2011 at 11:11 pm
    Apr 9, 2011 at 9:12 am
  • Looking under http://svn.apache.org/repos/asf/hadoop/mapreduce/branches/ it does not seem to be present. "pointers" to correct location appreciated.
    Stephen BoeschStephen Boesch
    Mar 28, 2011 at 11:00 pm
    Mar 28, 2011 at 11:17 pm
  • Hi, in this MR example, it exists the field "Reduce input groups" and "Reduce input records". What's the difference between these 2 fields? $ hadoop jar cloud9.jar ...
    Pedro CostaPedro Costa
    Mar 25, 2011 at 4:24 pm
    Mar 25, 2011 at 9:50 pm
  • Hi, during the setup phase and the cleanup phase of the tasks, the Hadoop MR uses map tasks to do it. These tasks appears in the counters shown at the end of an example? For example, the counter ...
    Pedro CostaPedro Costa
    Mar 25, 2011 at 5:09 pm
    Mar 25, 2011 at 9:05 pm
  • Hi, when I'm running the Gridmix2 examples, during the execution the tests halt and the following error is displayed: [code] 11/03/23 12:52:06 WARN mapred.JobClient:544 Use GenericOptionsParser for ...
    Pedro CostaPedro Costa
    Mar 23, 2011 at 12:03 pm
    Mar 23, 2011 at 12:08 pm
  • HI, How many tasks would assign per TT heartbeat? Is there a parameter for it to configure in hadoop? -- Best regards!
    Debbie FuDebbie Fu
    Mar 13, 2011 at 2:41 am
    Mar 13, 2011 at 4:40 am
  • Hi, I'm running hadoop map-reduce in clustering, and I've a Reduce Task that it remains in the state COMMIT_PENDING, and it doesn't finish. This is happening because I've made some changes to the ...
    Pedro CostaPedro Costa
    Mar 9, 2011 at 9:47 am
    Mar 12, 2011 at 6:34 am
  • Hi all, I encounter a wired problem while running a job on a cluster of 8 machines(1 master,7 slaves). I am under hadoop 0.21. What I am trying to do is storing the values from the map side into a ...
    Mar 11, 2011 at 7:48 am
    Mar 11, 2011 at 8:51 am
  • Hello experts, I am recently testing a set of logs that I converted to avro format in hadoop. I am notice really really slow performance when compare to raw logs. The map logs showing below seems to ...
    Felix gaoFelix gao
    Mar 2, 2011 at 6:26 pm
    Mar 2, 2011 at 8:11 pm
  • Hi, I am running a mapReduce job that uses up all the processor perfectly, however only 20% of available memory is being used. Is there a way to better utilize all the memory to speed up the ...
    Ondřej NevělíkOndřej Nevělík
    Mar 1, 2011 at 10:23 am
    Mar 1, 2011 at 12:05 pm
  • Hi, is there a config param that prevents m/r from deleting the job directory after the job is finished/failed? I need to look into the jar file with which the job is executed. Best regards, PŁ
    Paweł ŁozińskiPaweł Łoziński
    Mar 31, 2011 at 1:47 pm
    Mar 31, 2011 at 1:47 pm
  • Hi all, I need to generate random data with random writer. Can somebody tell me how I can generate less than 10 GB of data in total ? Which options should I configure in the configuration file ? Many ...
    Robert GrandlRobert Grandl
    Mar 25, 2011 at 9:43 am
    Mar 25, 2011 at 9:43 am
  • Jon LedermanJon Lederman
    Mar 21, 2011 at 9:34 pm
    Mar 21, 2011 at 9:34 pm
  • Hi, In the datanode logs I see an error about Duplicate MetricsName and Error register getProtocolVersion. This may be related to an error in repiicatiion that I encountered earlier. Does anyone have ...
    Jon LedermanJon Lederman
    Mar 21, 2011 at 4:11 pm
    Mar 21, 2011 at 4:11 pm
  • Hi, Upon trying a simple copyFromLocal I am receiving an error message "could only be replicated to 0 nodes instead of 1" as indicated below. All of my configuration appears ok and all daemons are ...
    Jon LedermanJon Lederman
    Mar 21, 2011 at 3:56 pm
    Mar 21, 2011 at 3:56 pm
  • Hi, I don't know what the examples of the Gridmix do. Where can I find an explanation of that? Thank -- Pedro
    Pedro CostaPedro Costa
    Mar 18, 2011 at 5:20 pm
    Mar 18, 2011 at 5:20 pm
  • Hi all, I want to modify WordCount application in order to delay the execution of maps. I have tried to put a sleep in map function but even a 1 ns sleep and for 128 MB blocks it took almost 30 ...
    Robert GrandlRobert Grandl
    Mar 18, 2011 at 12:16 pm
    Mar 18, 2011 at 12:16 pm
  • I sent this to the HBase mailing list, but thought I would also send this here in case anyone has any idea what might be going on. Thanks ---------- Forwarded message ---------- From: Nichole ...
    Nichole TreadwayNichole Treadway
    Mar 17, 2011 at 8:25 pm
    Mar 17, 2011 at 8:25 pm
  • Hello Fellow Hadoopists, We are meeting at 7:15 PM April 21st at the University Heights Community Center 5031 University Way NE Seattle WA 98105 Room #110 The meetings are informal and highly ...
    Sean jensen-greySean jensen-grey
    Mar 17, 2011 at 3:39 am
    Mar 17, 2011 at 3:39 am
  • Hi, I was trying to start up a single-node Hadoop cluster using 0.20.2. The namenode, datanode and jobtracker all started fine. The task tracker failed with the following error: 2011-03-16 ...
    Jane ChenJane Chen
    Mar 16, 2011 at 9:29 pm
    Mar 16, 2011 at 9:29 pm
  • [ Moving this to mapreduce-user@ - since it isn't a common-development related question, but a mapreduce specific one || bcc-ing common-dev@ ] Could we additionally know what scheduler is in effect ...
    Harsh JHarsh J
    Mar 15, 2011 at 4:43 pm
    Mar 15, 2011 at 4:43 pm
  • What are we supposed to use instead to submit jobs? I spent all day going through tutorials and documentations and could not find anything. Thanks in advance. Steven.
    S Steven KangS Steven Kang
    Mar 15, 2011 at 4:13 am
    Mar 15, 2011 at 4:13 am
  • Hi, I have some types of data that I have to upload on HDFS as Sequence Files. Initially, I had thought of creating a .jr file at runtime depending on the type of schema and use rcc DDL tool by ...
    Mapred LearnMapred Learn
    Mar 11, 2011 at 11:51 pm
    Mar 11, 2011 at 11:51 pm
Group Navigation
period‹ prev | Mar 2011 | next ›
Group Overview
groupmapreduce-user @

57 users for March 2011

Harsh J: 21 posts Pedro Costa: 14 posts Marcos Ortiz Valmaseda: 9 posts Mapred Learn: 6 posts Souri datta: 6 posts Baran cakici: 5 posts Lior Schachter: 5 posts Alejandro Abdelnur: 4 posts David Rosenstrauch: 4 posts John Therrell: 4 posts Allen Wittenauer: 3 posts Debbie Fu: 3 posts Dmitriy Lyubimov: 3 posts John Sanda: 3 posts Sonal Goyal: 3 posts Yorgo Sun: 3 posts Amareshwari Sri Ramadasu: 2 posts Dinesh shan: 2 posts Evert Lammerts: 2 posts Felix gao: 2 posts
show more