FAQ

Hadoop > common-user > June 2007

Search Discussions

65 discussions - 216 posts

  • I see a whole lot of this in my namenode log. Is this benign or is there something something more sinister going on ? Why would this be happening ? 2007-06-21 00:16:24,782 INFO ...
    PhantomPhantom
    Jun 22, 2007 at 5:09 pm
    Jun 25, 2007 at 4:30 pm
  • I've been experiencing some issues where my mapred tasks have been hanging after a lengthy period of execution. I believe I've found the problem and wanted to get other's thoughts about it. The ...
    Calvin YuCalvin Yu
    Jun 1, 2007 at 3:20 pm
    Jun 5, 2007 at 1:14 pm
  • Hi All What are minimal requirements on my Linux machine for building libhdfs ? On my Linux box I do not seem to have jni.h and what are the other binaries I need for this to work ? Could someone ...
    PhantomPhantom
    Jun 8, 2007 at 10:03 pm
    Jun 11, 2007 at 8:55 pm
  • Hi, Are there any HBase samples out there not using Junit? I would like to: a. create a master server, region and table descriptor. b. read in and convert a 'csv' file to byte[] (populating a family ...
    Peter W.Peter W.
    Jun 27, 2007 at 6:43 pm
    Jun 28, 2007 at 10:07 pm
  • Hi Can this only be done for read only and write only mode ? How do I do appends ? Because if I am using this for writing logs then I would want to append to the file rather overwrite which is what ...
    PhantomPhantom
    Jun 13, 2007 at 9:49 pm
    Jun 14, 2007 at 5:08 pm
  • Hi all, Could anyone help me to figure out why hadoop does not see my input file? I have three computers rosetta8, rosetta9,and rosetta10. rosetta8 is listed in masters, rosetta9 and rosetta10 are ...
    Erdong (Roger) CHENErdong (Roger) CHEN
    Jun 2, 2007 at 4:10 am
    Jun 3, 2007 at 3:24 pm
  • All, I upgraded to the most recent trunk of Hadoop and I started getting the error below, where /d01/hadoop/dfs/name is our namenode directory: org.apache.hadoop.dfs.InconsistentFSStateException: ...
    Dennis KubesDennis Kubes
    Jun 1, 2007 at 6:00 am
    Jun 1, 2007 at 5:36 pm
  • How do I get the documentation for the C API ? Also when I try to use the hdfsConnect() function what do I pass as argument ? Should I pass in the host name of the master and the port it is listening ...
    PhantomPhantom
    Jun 13, 2007 at 4:04 am
    Jun 14, 2007 at 5:42 pm
  • Hi. I ran some performance tests (randomwrite/sort) on a small Hadoop cluster. The numbers are unexpected. Some numbers are so far off, I suspect that either I didn't tune all the right knobs, or I ...
    Bwolen YangBwolen Yang
    Jun 8, 2007 at 8:31 pm
    Jun 9, 2007 at 1:18 am
  • Hi, I am wondering if anyone has experienced this problem. Sometimes when I ran a job, a few map tasks (often just one) hang in the initializing phase for more than 3 minutes (it normally finishes in ...
    Jun RaoJun Rao
    Jun 21, 2007 at 12:21 am
    Jul 6, 2007 at 7:29 am
  • Hi, I tried to update my db, using the following command: bin/nutch updatedb crawld/crawldb crawld/segments/20070628095836 and my 2 nodes had an error and i can see the following exception: ...
    Emmanuel JOKEEmmanuel JOKE
    Jun 30, 2007 at 5:32 pm
    Jul 2, 2007 at 3:01 pm
  • Is there a way to kill a job that's currently running? pb
    PatrikPatrik
    Jun 27, 2007 at 6:11 pm
    Jun 27, 2007 at 8:25 pm
  • Hi there: I found that "File[] editFiles" in FSEditLog.java , then i trace the call stack and found that it can be configured as multi-case of "dfs.name.dir" . Is this means the NameNode data can be ...
    KrzyCubeKrzyCube
    Jun 25, 2007 at 2:37 am
    Jun 26, 2007 at 6:27 pm
  • Hi,all: I am using Eclipse to View Hadoop source code , and i want to trace to see how it works, I code a few code to call the FSClient and when i call into the RPC object, it can not to be deep more ...
    KrzyCubeKrzyCube
    Jun 21, 2007 at 9:08 am
    Jun 22, 2007 at 7:17 am
  • Hi I am assuming that if I need a C/C++ interface to HDFS I must build libhdfs. This may be a problem very specific to my environment but would appreciate if someone could tell me what is going on ? ...
    PhantomPhantom
    Jun 11, 2007 at 10:16 pm
    Jun 12, 2007 at 6:17 am
  • Hello! I'm deploying Nutch on two computers. When I run start-all.sh script all goes good but data node on slave computer does not log anything. All other parts of Hadoop (namenode, jobtracker, both ...
    Ilya VishnevskyIlya Vishnevsky
    Jun 7, 2007 at 12:33 pm
    Jun 11, 2007 at 7:43 pm
  • Hi all, MapFile doesn't support append mode of creation, so every time the existing mapfile would be overwritten if a new one with same name is created. Is there anyway I can append to an MapFile or ...
    Open StudyOpen Study
    Jun 26, 2007 at 3:12 pm
    Jun 27, 2007 at 3:34 am
  • I tried running the hdfs_test from a machine which is not part of the Hadoop cluster. Could someone please tell me what I am doing wrong (error shown below)? I get the following error : 07/06/20 ...
    PhantomPhantom
    Jun 21, 2007 at 3:19 am
    Jun 21, 2007 at 10:00 pm
  • Hi, I am trying to build the native code (which includes the compression library) on Solaris 10 (x86 64-bit). I get the following error while building: gmake all-recursive gmake[1]: Entering ...
    Mahajan, NeerajMahajan, Neeraj
    Jun 19, 2007 at 2:44 am
    Jun 20, 2007 at 7:20 pm
  • Hi When I format my namenode it does format the directory specified under dfs.name.dir. However there is a folder under /tmp called hadoop-alakshman. What is this for ? Will all blocks be stored ...
    PhantomPhantom
    Jun 15, 2007 at 5:40 am
    Jun 15, 2007 at 7:37 pm
  • Hi, I am trying to figure out if Hadoop can be used for one functionality that I am trying to develop. I have large volumes of data already stored on disks that are locally/remotely mounted on many ...
    Neeraj MahajanNeeraj Mahajan
    Jun 11, 2007 at 6:38 pm
    Jun 11, 2007 at 7:50 pm
  • Hi. Upgrading from 0.12.3 to 0.13 seems to work fine. (at least the before and after "fsck "and "dfs -lsr" outputs matches). Then I shut down the DFS cluster, and reformat the DFS (to start a new ...
    Bwolen YangBwolen Yang
    Jun 9, 2007 at 12:22 am
    Jun 11, 2007 at 6:26 am
  • Here is a summary of my remaining questions from the [write and sort performance] thread. - Looks like every 5GB data I put into Hadoop DFS, it uses up ~18GB of raw disk space (based on block counts ...
    Bwolen YangBwolen Yang
    Jun 9, 2007 at 1:57 am
    Jun 9, 2007 at 4:26 pm
  • Hi, is it possible to pipe/redirect something to HDFS? Say you have a gz file that you want to put in distributed filesystem in an uncompressed form, do you have to make a local file first? What if ...
    Mark MeissonnierMark Meissonnier
    Jun 1, 2007 at 12:17 am
    Jun 1, 2007 at 5:11 pm
  • Hi, I have setup hadoop on 2 machines and am now trying to see if it is working properly. I have 3 questions: 1. Do I need to setup files specially for them to work with sort? My self-made test files ...
    Kevin LimKevin Lim
    Jun 28, 2007 at 9:26 pm
    Jul 1, 2007 at 11:33 pm
  • while this is not exactly hadoop related, I thought the people reading this would have the answer or know where to look for one. A friend of mine is looking to write a distributed name server/lock ...
    Ian HolsmanIan Holsman
    Jun 28, 2007 at 2:39 am
    Jun 29, 2007 at 1:33 am
  • I entered the following in hadoop-site.xml and am getting 'connection refused' stacktrace at Linux command line. What could cause this? <?xml version="1.0"? <?xml-stylesheet type="text/xsl" ...
    DANIEL CLARKDANIEL CLARK
    Jun 27, 2007 at 7:20 pm
    Jun 28, 2007 at 3:02 pm
  • Hi, having got a few of the examples working on a small cluster of machines I tried writing my own map reduce task to run. Its basically similar to the PiEstimator (infact I copied much of the ...
    Oliver HaggartyOliver Haggarty
    Jun 26, 2007 at 1:26 pm
    Jun 27, 2007 at 12:55 pm
  • Hello all I'm trying to get Hadoop going on my Windows machine without Cygwin. So far, I've sorted out the Java Service Wrapper configurations to run the services necessary for DFS to work: ...
    Albert StrasheimAlbert Strasheim
    Jun 24, 2007 at 10:20 pm
    Jun 24, 2007 at 11:06 pm
  • Is it possible to keep a file open for say 1 hour and write to it every once in a while and then close it. I constantly get the same error on attempt to close the handle of the file when I am done ...
    PhantomPhantom
    Jun 22, 2007 at 3:55 am
    Jun 22, 2007 at 6:23 am
  • Hello Hadoop users, I've been scratching my head over this one and wondered if anybody had ever encountered something similar : I have 2 output files from a MapReduce job. Now I want to use these ...
    Alexandre RochetteAlexandre Rochette
    Jun 21, 2007 at 12:43 am
    Jun 21, 2007 at 9:36 pm
  • Hi, I am trying to setup hadoop on two machines running solaris 10. After fixing the scripts in bin to conform to bourne shell scripting standard, I was able to start the jobtracker and tasktrackers. ...
    Mahajan, NeerajMahajan, Neeraj
    Jun 15, 2007 at 12:51 am
    Jun 15, 2007 at 9:10 pm
  • Hi, I'm having a few problems getting hadoop to run on a single node. I had it up and running fine a couple of days ago, and then progressed to trying to get it going on a small cluster but didn't ...
    Oliver HaggartyOliver Haggarty
    Jun 14, 2007 at 11:42 am
    Jun 14, 2007 at 4:30 pm
  • Hi I had a question about ways of setting up large clusters. I did read the WIKI which has a posting on this matter and I have also been through the exercise of setting up a cluster of 15 nodes. If I ...
    PhantomPhantom
    Jun 8, 2007 at 5:35 pm
    Jun 8, 2007 at 5:49 pm
  • Hi, Given that map/reduce produces a partitioned set of sorted output files, I was wondering if a map implementation exists for doing lookups or iterate thru subranges of these files. This would be ...
    Bwolen YangBwolen Yang
    Jun 4, 2007 at 10:00 pm
    Jun 4, 2007 at 10:37 pm
  • Hi Is there a way to chain Map/Reduce tasks ? What I mean is I want the output a MapReduce task to serve as input to another MapReduce task ? Could someone please show me how I can acheive this ? ...
    PhantomPhantom
    Jun 2, 2007 at 8:15 pm
    Jun 4, 2007 at 10:35 pm
  • Does anyone have an example of how I can read a file that lives in HDFS from a Servlet? Thank you for your time, -Cesar
    Cesar DelgadoCesar Delgado
    Jun 2, 2007 at 7:09 pm
    Jun 3, 2007 at 8:40 pm
  • Hi all, I have some simple questions that I would like answered to get a better understanding of what Hadoop/Mapreduce is. I noticed in the code of the WordCount example: conf.setInputPath(new ...
    Jeroen VerhagenJeroen Verhagen
    Jun 27, 2007 at 1:16 pm
    Jun 27, 2007 at 8:17 pm
  • I have text data available in split files. Per file there are say n groups of data lines that I need to process as one unit of data. Later based on different parametrs I would combine the results of ...
    Mahajan, NeerajMahajan, Neeraj
    Jun 21, 2007 at 9:19 pm
    Jun 21, 2007 at 9:38 pm
  • Hi all, In an ideal world, my TaskTrackers would be working for me all the time. That is: the average number of tasks a TaskTracker is handling/processing would be close to ...
    Mathijs HommingaMathijs Homminga
    Jun 21, 2007 at 7:38 pm
    Jun 21, 2007 at 8:09 pm
  • Hi All I know this is a tall ask. I am going through the source code. But could someone please tell me the intuition behind the design of the MapFile class. If I were using the MapFile against the ...
    PhantomPhantom
    Jun 20, 2007 at 3:50 pm
    Jun 20, 2007 at 5:05 pm
  • Hi all, I tried to run word count on a cluster with 30 nodes, and I always get the same error. Could someone give me some insights? What type of errors I may need to investigate? thanks. 07/06/17 ...
    Erdong (Roger) CHENErdong (Roger) CHEN
    Jun 17, 2007 at 7:55 pm
    Jun 17, 2007 at 7:55 pm
  • Hi, I noticed that HADOOP-975 and HADOOP-1000 made the log4j from child vms go to a different place than the stdout for the task. My tasks send some of their debugging information to stdout, and some ...
    Michael BieniosekMichael Bieniosek
    Jun 14, 2007 at 11:15 pm
    Jun 15, 2007 at 11:26 am
  • It seems I'm having a lot of trouble trying to configure hadoop on one machine. I've followed the wiki tutorial and I've configured every thing on 1 machine. I tried to start hadoop using ...
    Emmanuel JOKEEmmanuel JOKE
    Jun 13, 2007 at 1:01 pm
    Jun 13, 2007 at 6:37 pm
  • Hey. I've been trying to figure out a nice and clean way of getting the key class of the input values in my mapper's configuration method. Since the JobConf.getInputKeyClass() method is deprecated I ...
    Johan OskarssonJohan Oskarsson
    Jun 13, 2007 at 5:49 pm
    Jun 13, 2007 at 6:02 pm
  • Exception in thread "main" org.apache.hadoop.ipc.RPC$VersionMismatch:Protocol org.apache.hadoop.dfs.ClientProtocol version mismatch. (client = 11, server = 9) dev030.sctm.com: at ...
    PhantomPhantom
    Jun 13, 2007 at 3:02 am
    Jun 13, 2007 at 6:08 am
  • I am developing an application that is currently based on mysql. I believe it will require significant scaling and so I am exploring hadoop. I know nothing about map-reduce and am trying to figure ...
    Hank williamsHank williams
    Jun 12, 2007 at 10:57 am
    Jun 12, 2007 at 1:13 pm
  • Hi all: I have tried to compress the output data, so I use the sentence below: TextOutputFormat.setCompressOutput(conf, true); After I finish my hadoop job, the output data is like these: ...
    张茂森张茂森
    Jun 9, 2007 at 11:47 pm
    Jun 10, 2007 at 12:59 am
  • Hello, I try to modify the input to MapClass. More specifically, taking the example WordCount, the map method in the MapClass accepts given keys and values, and in this case, it is LineRecordReader ...
    Richard YangRichard Yang
    Jun 5, 2007 at 8:01 am
    Jun 5, 2007 at 4:03 pm
  • Hello, I'm trying to put the contents of a regular java Map into a Hadoop MapFile but get an java.io.EOFException error. The origin map has document ids and double values and the destination map ...
    Peter W.Peter W.
    Jun 4, 2007 at 2:36 am
    Jun 5, 2007 at 1:43 am
Jun 2007
12
Group Navigation
period‹ prev | Jun 2007 | next ›
Group Overview
groupcommon-user @
categorieshadoop
discussions65
posts216
users62
websitehadoop.apache.org...
irc#hadoop

62 users for June 2007

Phantom: 25 posts Doug Cutting: 18 posts Mahajan, Neeraj: 13 posts Bwolen Yang: 10 posts Peter W.: 10 posts Devaraj Das: 8 posts Arun C Murthy: 6 posts Dhruba Borthakur: 6 posts Erdong (Roger) CHEN: 6 posts Konstantin Shvachko: 6 posts Michael Bieniosek: 6 posts Dennis Kubes: 5 posts KrzyCube: 5 posts Oliver Haggarty: 5 posts Raghu Angadi: 5 posts Avinash Lakshman: 4 posts Calvin Yu: 4 posts James Kennedy: 4 posts Michael Stack: 4 posts Neeraj Mahajan: 4 posts
show more
Archives