FAQ

Search Discussions

68 discussions - 241 posts

  • Hi, Apologies for yet another question from me, but here goes! I've written a map task that will on occasion not compute the correct result. This can easily be detected, at which point I'd like the ...
    Ojh06Ojh06
    Jul 30, 2007 at 8:42 pm
    May 29, 2008 at 5:07 pm
  • Hi all, I've been reading the docs and the code, but I'm still somewhat hazy as to what is the exact step-by-step procedure to perform a failover between a primary NameNode and a SecondaryNameNode, ...
    Andrzej BialeckiAndrzej Bialecki
    Jul 20, 2007 at 4:47 pm
    Aug 1, 2008 at 8:47 am
  • Hi there: i got two questions: Q1: I am try to call the FsShell.doMain() with my own code , which is only a easy wrapper of the FsShell. But when i am trying to create many dirs , 10000 etc. ...
    KrzyCubeKrzyCube
    Jul 24, 2007 at 2:49 am
    Jul 31, 2007 at 7:08 am
  • I've deployed hadoop-0.13.0 and successfully run some examples. Now I am trying to compile and run the examples prior to starting to develop my own code. I've managed to do little more than get a ...
    C GC G
    Jul 26, 2007 at 10:01 pm
    Jul 27, 2007 at 10:21 pm
  • Is replica management built into HDFS ? What I mean is if I set replication factor to 3 and if I lose 3 disks is that data lost forever ? I mean all 3 disks dying at the same time I know is a far ...
    PhantomPhantom
    Jul 17, 2007 at 10:46 am
    Jul 17, 2007 at 9:50 pm
  • I have a HDFS with 2 datanodes and 1 namenode in 3 different machines, 2G ram each. Datanode A contains around 700,000 blocks and Datanode B contains 1,200,000+ blocks, the namenode fails to start ...
    ErolagnabErolagnab
    Jul 16, 2007 at 2:11 am
    Jul 17, 2007 at 1:49 pm
  • Oops... I executed the following command: ./hadoop dfs -rmr . Everything on the DFS, including the trash seems to be deleted. Is there a way to recover my data? Thanks, Mathijs -- Knowlogy Helperpark ...
    Mathijs HommingaMathijs Homminga
    Jul 17, 2007 at 2:26 pm
    Jul 17, 2007 at 9:21 pm
  • How are datanodes added? Do they get added and started only at start of DFS filesystem? Can they be added while hadoop fs is running by editing slaves file or does hadoop have to be restarted? Ankur ...
    Ankur SethiAnkur Sethi
    Jul 17, 2007 at 4:25 pm
    Jul 19, 2007 at 2:34 am
  • Specifically this: bin=`dirname "$0"` bin=`cd "$bin"; pwd` . "$bin"/hadoop-config.sh The problem is that the 'cd' command on cygwin and Fedora is not silent, so if one tries: bin/hadoop namenode ...
    Charlie wCharlie w
    Jul 18, 2007 at 4:16 pm
    Jul 18, 2007 at 6:18 pm
  • I am about to attempt setting up a hadoop file system for an application. Hadoop Filesystem has single point of failure, namenode. Can you explain steps necessary for bringing the HDFS backup in case ...
    Ankur SethiAnkur Sethi
    Jul 14, 2007 at 6:54 pm
    Jul 16, 2007 at 8:05 pm
  • Hello, I have a modified WordCount program with the following characteristics: input file: urla.com,urlb.com urla.com,urlc.com urlb.com,urlc.com urlc.com,urla.com urld.com,urlc.com mapreduce output: ...
    Peter W.Peter W.
    Jul 2, 2007 at 12:38 am
    Jul 5, 2007 at 6:22 pm
  • hello, how to start hadoop on Windows? bin/start-all.sh can't be executed on Windows,so how to ?
    Wayne LiuWayne Liu
    Jul 4, 2007 at 4:30 pm
    Jul 5, 2007 at 2:43 pm
  • hello, I tried nutch with hadoop nightly builds (in hudson #135 and newer) and got following problem: java.io.IOException: Lock obtain timed out: ...
    DESDES
    Jul 26, 2007 at 11:07 pm
    Oct 15, 2007 at 8:08 am
  • Hi all, I'm trying to get Hadoop 0.13 running from Intellij so I can test my own Map and Reduce classes on the local job runner. To do that I'm first trying to get the WordCount sample to work and ...
    Jeroen VerhagenJeroen Verhagen
    Jul 23, 2007 at 1:50 pm
    Jul 26, 2007 at 9:59 am
  • Hello, I'm working on a MapReduce job that requires some third-party jars. I know that the devs are working on [https://issues.apache.org/jira/browse/HADOOP-1622] HADOOP-1622, which should make this ...
    Stu HoodStu Hood
    Jul 23, 2007 at 10:09 pm
    Jul 23, 2007 at 10:53 pm
  • The title pretty much says it all, although I would say that it might be of interest even if you're not using Amazon Web Services. Tom
    Tom WhiteTom White
    Jul 19, 2007 at 8:56 pm
    Jul 20, 2007 at 9:30 pm
  • Is it possible to have multiple job jar files being submitted to hadoop at once? If not, is this a feature that might be useful? I can see this being useful for custom Nutch development, having a ...
    Dennis KubesDennis Kubes
    Jul 19, 2007 at 12:14 am
    Jul 20, 2007 at 4:59 pm
  • Hi Hadoopers ! I'm working on Hadoop for an internship, trying to find out its possibilities in use with Lucene... my problem is that I'v been reading loads of docs for a week or so, such as ...
    Samuel LEMOINESamuel LEMOINE
    Jul 18, 2007 at 2:30 pm
    Jul 19, 2007 at 7:41 am
  • Hello! I'm trying to run nutch on two computers. Here is content of my "slaves" file: localhost morpheus When I type bin/start-al.sh, I get the next output: starting namenode, logging to ...
    Ilya VishnevskyIlya Vishnevsky
    Jul 12, 2007 at 1:44 pm
    Jul 16, 2007 at 10:21 am
  • Hi When I submit jobs in Hadoop how do the physical class files get distributed to the nodes on which the Map/Reduce jobs run ? Is some kind of dynamic class loading used or are the jar files copied ...
    PhantomPhantom
    Jul 12, 2007 at 8:56 pm
    Jul 13, 2007 at 3:40 am
  • Hi all, Just wondering what is the reason causing NameNode is on SafeMode forever? I've left my machine running for 2 days and it's still on Safe Mode. Trung -- View this message in context: ...
    ErolagnabErolagnab
    Jul 10, 2007 at 6:53 am
    Jul 11, 2007 at 3:30 am
  • Hi, I'm writing a mapreduce task that will take a load of complex numbers, do some processing on each then return a double. As this processing will be complex and could take up to 10 minutes I am ...
    Oliver HaggartyOliver Haggarty
    Jul 3, 2007 at 3:45 pm
    Jul 5, 2007 at 1:12 pm
  • Hello, I was reading Hadoop's getting started ( http://wiki.apache.org/lucene-hadoop/GettingStartedWithHadoop), and in the section named "Starting up a larger cluster" I had a doubt about starting ...
    Lucas Nazário dos SantosLucas Nazário dos Santos
    Jul 31, 2007 at 1:39 pm
    Aug 2, 2007 at 8:38 pm
  • Hi, I've got a GUI based program that I'm working on, and I'm trying to add some funcionality to it where it runs a map reduce job on hadoop. For the moment I am assuming that anyone who is running ...
    Ojh06Ojh06
    Jul 27, 2007 at 10:36 am
    Jul 27, 2007 at 3:23 pm
  • Hi All Is there a way to find out on which nodes in my cluster the Map/Reduce jobs are running after I submit my job ? Also is there anyways to determine given a file where the different blocks of ...
    PhantomPhantom
    Jul 19, 2007 at 3:58 pm
    Jul 19, 2007 at 7:21 pm
  • Hi, I've been trying to set up Hadoop as an Eclipse project, but am having some difficulties. I've tried importing a project and using the Eclipse subversion plugin, I've tried creating a project ...
    Ojh06Ojh06
    Jul 18, 2007 at 1:56 pm
    Jul 18, 2007 at 2:46 pm
  • Hello, I'm wondering how does work hadoop on a cluster of few machine with different hardware. For instance, i have a cluster of 2 machine with the same hardware (same CPU p4 2.8 , same memory RAM ...
    EmmanuelEmmanuel
    Jul 15, 2007 at 9:17 am
    Jul 17, 2007 at 3:15 am
  • Hi All, I am new to hadoop and learning how to use it. I have a problem which can be solvable using map-reduce technique. But, in my map step, I need to consider some extra information which depends ...
    Novice userNovice user
    Jul 16, 2007 at 1:19 pm
    Jul 16, 2007 at 6:28 pm
  • Can anyone who is running large clusters (50+) tell me what you are seeing with hard disk failure rates. Something that we are seeing is that certain machines will consistently have double or triple ...
    Dennis KubesDennis Kubes
    Jul 31, 2007 at 2:07 pm
    Aug 1, 2007 at 6:10 pm
  • Hi, I am exploring hadoop and using it for one of my machine learning application. I have a problem in which I need to route a particular input to each map task separately. For example, I have list ...
    Novice userNovice user
    Jul 24, 2007 at 4:42 am
    Jul 24, 2007 at 12:24 pm
  • Hi Folks, I'd love to hear more about how Hadoop is being used in the wild. If you are using Hadoop, please add your project to our PoweredBy page, and/or respond to this email. ...
    Eric BaldeschwielerEric Baldeschwieler
    Jul 20, 2007 at 11:40 pm
    Jul 21, 2007 at 12:42 am
  • Hi , We have upgraded our code to nutch-0.9 with hadoop-0.12.2-core.jar. After running say 50 nutch jobs(which includes inject/generate/fetch/parse etc.) we start getting "Too many open files" error ...
    Shailendra MudgalShailendra Mudgal
    Jul 17, 2007 at 6:46 am
    Jul 17, 2007 at 6:17 pm
  • Hello, How well does Hadoop scale for multiple client inputs? For instance, could a reasonably powerful namenode handle 100 client machines copying in 10 MB every 10 minutes? Assume all of the ...
    Stu HoodStu Hood
    Jul 13, 2007 at 3:44 pm
    Jul 13, 2007 at 4:27 pm
  • Hi I'm using the latest version of Hadoop. Does it support specifying a pattern for input file names, apart from specifying an input path thru jobConf.setInputPath(). In my case, logfiles for over a ...
    Sandhya ESandhya E
    Jul 10, 2007 at 9:23 am
    Jul 10, 2007 at 5:06 pm
  • Hi folks, I have hadoop installed on an NFS, for which I have a file limit of 20000. I have the HDFS using the /tmp directory on several network machines. Everything works fine, but I am finding that ...
    Oliver HaggartyOliver Haggarty
    Jul 6, 2007 at 6:25 am
    Jul 9, 2007 at 12:02 pm
  • I'm running hadoop streaming from svn (version 552930, recent within a week or so). My map/reduce job maps ~1M records, but then a few reduces succeed and many fail, eventually terminating the job ...
    John HeidemannJohn Heidemann
    Jul 5, 2007 at 7:47 pm
    Jul 6, 2007 at 6:53 pm
  • How do I finalize the DFS Upgrade? Do I just need to remove the previous directories or is there a script or command line option that will do this for me? Dennis Kubes
    Dennis KubesDennis Kubes
    Jul 31, 2007 at 8:43 pm
    Jul 31, 2007 at 8:47 pm
  • Hi All I have been trying to use the DataOutputBuffer class for its obvious memory efficiency. I basically write some data into the buffer and then write the buffer into a file (an instance of ...
    PhantomPhantom
    Jul 28, 2007 at 3:06 am
    Jul 28, 2007 at 3:29 am
  • Hi all ! I'm working on hadoop and currently i'm using the examples provided (WordCound & Grep especially). I've managed to make those examples work on a local machine, and now I'd like to go on to ...
    Samuel LEMOINESamuel LEMOINE
    Jul 26, 2007 at 3:28 pm
    Jul 26, 2007 at 3:57 pm
  • Hi, Iam new to hadoop, Wanted to use hadoop in my application. Currently I want to simulate something like "SELECT FROM WHERE " FieldSelectionMapReduce Class can be used to reduce the no of ...
    Meda vijendharreddyMeda vijendharreddy
    Jul 25, 2007 at 1:48 pm
    Jul 25, 2007 at 4:19 pm
  • Hello, I'm trying to get Hadoop running as Windows service, but strange things happen, preventing me from finishing the task. Notice that I can run Hadoop in the usual way, using cygwin over Windows ...
    Lucas Nazário dos SantosLucas Nazário dos Santos
    Jul 24, 2007 at 5:53 pm
    Jul 25, 2007 at 12:18 am
  • I have started to use the following log4j xml to send logs to both the mapreduce tasklog and to the syslog daemon. Unfortunately, it creates a new log split in the tasklog for each log entry. Is this ...
    Anthony D. UrsoAnthony D. Urso
    Jul 19, 2007 at 2:43 am
    Jul 19, 2007 at 4:12 pm
  • Hi I have two MapReduces running sequentially to accomplish a job. I first started running the jobs locally in a single machine. First MapReduce produces a set of keys which were stored inmemory in a ...
    Sandhya ESandhya E
    Jul 18, 2007 at 5:06 am
    Jul 18, 2007 at 3:26 pm
  • hi is it possible to access the HDFS from normal linux filesystem? I have some C modules which propulate files in filesystem, and i like to scale it into HDFS Thx Andrey K.
    AndreyAndrey
    Jul 16, 2007 at 12:48 am
    Jul 16, 2007 at 12:49 am
  • Is there a way to initialize Mapper classes based on some args passed through the command line ? Ideally I would like to do conf.setMapperClass( Mapper.class ) which I am assuming ultimately creates ...
    PhantomPhantom
    Jul 13, 2007 at 5:50 pm
    Jul 13, 2007 at 5:59 pm
  • Makefile has some problems in that it places the path to the location of the libhdfs.so, in the name of shared library by passing the path to -soname flag. I have it changed and was wondering who I ...
    PhantomPhantom
    Jul 6, 2007 at 7:16 pm
    Jul 6, 2007 at 9:17 pm
  • Hi all, I am given a task to extract data from a big file to HDFS. The input is a 1G text file contains millions of lines. The line starts with # which indicates a record. Subsequence lines which ...
    Nguyen Kien TrungNguyen Kien Trung
    Jul 6, 2007 at 7:29 am
    Jul 6, 2007 at 7:29 am
  • Would someone be kind enough to share with me any code/sample they have for using the MapFile class ? Thanks A
    PhantomPhantom
    Jul 2, 2007 at 6:50 pm
    Jul 4, 2007 at 9:09 am
  • This time i checkout source from trunk and run it. The i just came to terminal and want to see how many file there in the HDFS. and i just execute "bin/hadoop -ls" , then error message throw out as ...
    KrzyCubeKrzyCube
    Jul 3, 2007 at 6:55 am
    Jul 3, 2007 at 7:23 am
  • Hi, Hadoop version 0.13.1. The job seems to be finished, but the reduce percentage is stopped at 82.56%. *Started at:* Mon Jul 30 14:04:28 EEST 2007 *Status:* Succeeded *Finished at:* Tue Jul 31 ...
    Ion BaditaIon Badita
    Jul 31, 2007 at 7:55 am
    Jul 31, 2007 at 7:55 am
Group Navigation
period‹ prev | Jul 2007 | next ›
Group Overview
groupcommon-user @
categorieshadoop
discussions68
posts241
users63
websitehadoop.apache.org...
irc#hadoop

63 users for July 2007

Ted Dunning: 20 posts Oliver Haggarty: 15 posts Phantom: 15 posts Nguyen Kien Trung: 12 posts Doug Cutting: 9 posts KrzyCube: 9 posts Mahajan, Neeraj: 8 posts Raghu Angadi: 8 posts Andrzej Bialecki: 7 posts Dennis Kubes: 7 posts Dhruba Borthakur: 7 posts Peter W.: 7 posts Ankur Sethi: 5 posts C G: 5 posts Devaraj Das: 5 posts Emmanuel JOKE: 5 posts Lucas Nazário dos Santos: 5 posts Stu Hood: 5 posts Briggs: 4 posts Jeroen Verhagen: 4 posts
show more