FAQ

Hadoop > common-user > August 2008

Search Discussions

167 discussions - 539 posts

  • Hi, I'm a beginner to Hadoop. Is there any good book on this technology ? Regards, Feris
    Feris ThiaFeris Thia
    Aug 25, 2008 at 5:16 pm
    Sep 16, 2008 at 10:42 am
  • Hello, can someone please explain oder point me to some documentation or papers, where i can read well proven facts, why scaling a relational db is so hard and scaling a document oriented db isnt? So ...
    Mork0075Mork0075
    Aug 7, 2008 at 12:46 pm
    Aug 28, 2008 at 6:46 am
  • Hi folks; I'm new to Hadoop, and I'm trying to set it up on a cluster for which almost all the disk is mounted via the Lustre filesystem. That filesystem is visible to all the nodes, so I don't ...
    Joel WellingJoel Welling
    Aug 21, 2008 at 5:01 pm
    Sep 3, 2008 at 11:04 pm
  • Hi, I have installed Hadoop on 20 nodes (data storage) and one master (namenode) to which i want to add data. I have learned that this is possible through a Java API or via the Hadoop shell. However, ...
    Sebastian VieiraSebastian Vieira
    Aug 6, 2008 at 12:05 pm
    Feb 22, 2011 at 7:52 pm
  • Hi All, This might seem too silly, but I couldn't find a satisfactory answer to this yet. What are the advantages / disadvantages of using Hadoop Streaming over the normal mode (wherein you write ...
    Gaurav VedaGaurav Veda
    Aug 12, 2008 at 10:10 pm
    Aug 20, 2008 at 1:04 am
  • Hi, I am evaluating Hadoop's Portability Across Heterogeneous Hardware and Software Platforms. For this I am trying to setup a grid (Hadoop 0.17) having Linux ( RHEL5 / FC 9), Solaris (SunOS 5) and ...
    Arun VenugopalArun Venugopal
    Aug 13, 2008 at 9:22 am
    Jan 5, 2009 at 11:30 am
  • Hi, I read a lot of tutorial and FAQ founded using search enginers but I didn't find anything abount integration of HADOOP and Map/Reduce paradigm into HPC environment. I manage a HPC cluster of 20 ...
    Filippo SpigaFilippo Spiga
    Aug 17, 2008 at 5:57 pm
    Sep 5, 2008 at 9:31 am
  • Hi Everybody! I'm a newbie with Hadoop, I've installed it as a single node as a pseudo-distributed environment, but I would like to go further and configure a complete hadoop cluster. But I got the ...
    Gerardo VelezGerardo Velez
    Aug 28, 2008 at 9:29 pm
    Sep 2, 2008 at 1:42 am
  • Hi, Can some expert differentiate or compare HDFS with KFS ? Apparently it looks like similar architecture with little difference and same objective. Thanks, Wasim
    Wasim BariWasim Bari
    Aug 21, 2008 at 1:45 pm
    Aug 26, 2008 at 4:49 am
  • Hello list, Thought I would share this tidbit that frustrated me for a couple of hours. Beware! Hadoop reuses the Writable objects given to the reducer. For example: public void reduce(K key, ...
    Stuart SierraStuart Sierra
    Aug 19, 2008 at 7:19 pm
    Aug 20, 2008 at 9:53 pm
  • Hi there: We would like to know what are the most likely causes of this sort of error: Exception closing file ...
    Piotr KozikowskiPiotr Kozikowski
    Aug 7, 2008 at 7:53 pm
    Aug 12, 2008 at 5:31 pm
  • Hello, Can someone point me out what are the extra tasks that need to be performed in order to set up a cluster where nodes are spread over the Internet, in different LANs? Do I need to free any ...
    Lucas Nazário dos SantosLucas Nazário dos Santos
    Aug 8, 2008 at 3:17 am
    Aug 10, 2008 at 9:40 pm
  • Hi, This is about dfs only, not to consider mapreduce. It may sound like a strange need, but sometimes I want to read a block from a specific data node which holds a replica. Figuring out which ...
    KevinKevin
    Aug 6, 2008 at 2:00 am
    Aug 7, 2008 at 3:12 pm
  • The reason have revealed itself quite easily - some tasks have failed according to JobTracker's web interface. Record reporting such a failure usually looks somehow like this (usually appears at ...
    ИванИван
    Aug 29, 2008 at 10:06 am
    Sep 5, 2008 at 2:01 pm
  • hi I found we miss lib/native/Linux-amd64-64 on hadoop-0.17.2.tar.gz ? thanks -- Yi-Kai Tsai (cuma) <yikai@yahoo-inc.com , Asia Regional Search Engineering.
    Yi-Kai TsaiYi-Kai Tsai
    Aug 20, 2008 at 4:27 am
    Aug 28, 2008 at 3:21 pm
  • I am using Hadoop streaming and I need to pass arguments to my map/reduce script. Because a map/reduce script is triggered by hadoop, like hadoop .... -file MAPPER -mapper "$MAPPER" -file REDUCER ...
    Gopal GandhiGopal Gandhi
    Aug 21, 2008 at 7:15 am
    Aug 22, 2008 at 9:59 pm
  • Hi, I am planning to use "distributed lucene" from hadoop.contrib.index for indexing. Has anyone used this or tested it? Any issues or comments? I see that the design described is different from HDFS ...
    Deepika KheraDeepika Khera
    Aug 7, 2008 at 12:41 am
    Aug 18, 2008 at 3:06 pm
  • Hi communities, Do you have any idea how to get the pairs of all row key combinations w/o repetition on Map/Reduce as describe below? Input : (MapFile or Hbase Table) <Key1, Value or RowResult <Key2, ...
    Edward J. YoonEdward J. Yoon
    Aug 13, 2008 at 12:17 pm
    Aug 13, 2008 at 2:35 pm
  • Hi colleagues, As you know, the append writer will be available in version 0.18. We are here waiting for the feature and want to know the rough time of release. Thanks!
    11 Nov.11 Nov.
    Aug 13, 2008 at 6:52 am
    Aug 22, 2008 at 7:23 am
  • Hello -- As far as I can tell, "hadoop dfs -rmr" only checks the permissions of the directory to be deleted and it's parent. Unlike Unix, however, it does not seem to check the permissions of the ...
    Brian KarlakBrian Karlak
    Aug 7, 2008 at 6:27 pm
    Aug 14, 2008 at 5:14 pm
  • Hello, i'am planning to use HDFS as a DFS in a web application evenvironment. There are two requirements: fault tolerence, which is ensured by the replicas and load balancing. Is load balancing part ...
    Mork0075Mork0075
    Aug 27, 2008 at 7:55 am
    Sep 1, 2008 at 2:24 pm
  • Hi, I would like the reducer to output to different files based upon the value of the key. I understand that both MultipleOutputs and MultipleOutputFormat can do this. Is that correct? However, I ...
    Shirley CohenShirley Cohen
    Aug 27, 2008 at 10:11 pm
    Aug 29, 2008 at 3:17 pm
  • Hi, I need help if it's possible. My name is Leandro Alvim and i`m a graduated in computer science in Brazil. So, i'm using hadoop in my university project and i used your tutorials to learn how to ...
    Leandro AlvimLeandro Alvim
    Aug 27, 2008 at 2:55 pm
    Aug 27, 2008 at 9:33 pm
  • I'm trying to install Hadoop 0.17.2 version on a linux box (xen os) So, bin/start-all.sh works fine, but hadoop-hadoop-jobtracker-softtek-helio-dev.log shows me error showed below. Do you now how to ...
    Gerardo VelezGerardo Velez
    Aug 21, 2008 at 6:06 pm
    Aug 22, 2008 at 10:00 am
  • Hi, how can i add a datanode dynamically to a hadoop cluster without restarting the whole cluster? I was trying to run "hadoop datanode" on the new datanode with the appropriate config (pointing to ...
    Kai MosebachKai Mosebach
    Aug 14, 2008 at 8:49 am
    Aug 15, 2008 at 1:11 pm
  • Can someone help explain in a little more detail some of the reasons for the hardware specs that were recently added to the wiki for the NameNode. I guess i'm interested in learning how others have ...
    Manish ShahManish Shah
    Aug 12, 2008 at 6:25 pm
    Aug 15, 2008 at 11:38 am
  • Seeing as there is no search function on the archives, I'm relegated to asking a possibly redundant question or four: I have, as a sample setup: idx1-tracker JobTracker idx2-namenode NameNode ...
    James Graham (Greywolf)James Graham (Greywolf)
    Aug 6, 2008 at 5:38 pm
    Aug 7, 2008 at 10:42 am
  • Hi all. I'm running a clustering HDFS on linux and I need to access files (I/O) from eclipse Java application running on Windows. It seems simple, but is it possible? I have write code using API but ...
    Alberto ForcénAlberto Forcén
    Aug 5, 2008 at 9:18 am
    Aug 7, 2008 at 10:36 am
  • Hello, i just discovered the Hadoop project and it looks really interesting to me. As i can see at the moment, Hadoop is really useful for data intensive computations. Is there a Hadoop scenario for ...
    Mork0075Mork0075
    Aug 5, 2008 at 6:12 pm
    Aug 6, 2008 at 8:18 pm
  • Anybody have one? Any success building it with create-hadoop-image? Thanks, -Stuart
    Stuart SierraStuart Sierra
    Aug 29, 2008 at 1:49 pm
    Sep 4, 2008 at 2:28 pm
  • Hi all: I would like to sort a large number of records in a big file based on a given field (key). If I run just one reducer, it works fine because the reducer will sort all records based on the key. ...
    Charles duCharles du
    Aug 26, 2008 at 7:40 am
    Aug 28, 2008 at 3:09 am
  • Hi everybody, When i was running hadoop 0.17.1 it gave me some WARNs like this: 2008-08-09 10:53:37,728 WARN org.apache.hadoop.dfs.StateChange: DIR* FSDirectory.unprotectedDelete: failed to remove ...
    Mr.ThienMr.Thien
    Aug 9, 2008 at 4:32 am
    Aug 25, 2008 at 11:25 pm
  • Release 0.18.0 contains many improvements, new features, bug fixes and optimizations. For release details and downloads, visit: http://hadoop.apache.org/core/releases.html Thanks to all who ...
    Nigel DaleyNigel Daley
    Aug 22, 2008 at 8:45 pm
    Aug 25, 2008 at 5:26 pm
  • Hi, For a given input key, K, in a reduce task, does Hadoop guarantee that all mapper-emitted values for key K are available in the iterator? Is it possible that multiple reduce tasks can receive the ...
    Alex HolmesAlex Holmes
    Aug 22, 2008 at 11:00 pm
    Aug 24, 2008 at 11:22 pm
  • Hi group, Is it possible to customize the job name when using "bin/hadoop jar ..."? Best, -Kevin
    KevinKevin
    Aug 22, 2008 at 8:50 pm
    Aug 22, 2008 at 9:32 pm
  • Hi mailing, Are there any way to know whether the mapper is processing the last record that assigned to this node, or know how many records remain to be processed in this node? Qin
    Qin GaoQin Gao
    Aug 20, 2008 at 7:02 pm
    Aug 21, 2008 at 2:00 pm
  • Hi, I'm new to Hadoop - so hope you can help with this problem. I'm trying to set up a small (2-zone) hadoop cluster on Solaris. start-dfs.sh runs without error, e.g., it prints the following to the ...
    Bernard ButlerBernard Butler
    Aug 14, 2008 at 5:51 pm
    Aug 14, 2008 at 9:12 pm
  • IMHO, there must be one process that fails the write.
    11 Nov.11 Nov.
    Aug 10, 2008 at 4:30 am
    Aug 11, 2008 at 11:01 pm
  • Hi, I am a new hadoop developer and am struggling to understand why I cannot pass TupleWritable between a map and reduce function. I have modified the wordcount example to demonstrate the issue. Also ...
    Michael AndrewsMichael Andrews
    Aug 7, 2008 at 11:18 pm
    Aug 8, 2008 at 2:19 am
  • I'm trying to set up 2 Hadoop installations on my master node, one of which will have permissions that allow more users to run Hadoop. But I don't really need anything different on the datanodes, so ...
    Meng MaoMeng Mao
    Aug 4, 2008 at 6:11 pm
    Aug 5, 2008 at 6:03 pm
  • I'm getting the following exceptions while starting the name node - ERROR dfs.NameNode: java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:375) at ...
    Wanjari, AmolWanjari, Amol
    Aug 4, 2008 at 9:54 am
    Aug 5, 2008 at 7:50 am
  • when done, HADOOP-3387 would allow you to do that. In our implementation we can tell Hadoop the exact # maps and it will group splits if necessary.
    Alejandro AbdelnurAlejandro Abdelnur
    Aug 1, 2008 at 8:36 am
    Aug 4, 2008 at 7:37 am
  • Thanks. It looks like IsolationRunner is what I'm asking for. I'll try it out. I was aware of the logs, but unfortunately, have problems where inputs hang or don't log meaningful information. ...
    John HeidemannJohn Heidemann
    Aug 7, 2008 at 11:45 pm
    Sep 26, 2008 at 8:20 pm
  • Hello all, I'm new to Hadoop. I'm trying to write a small hadoop map/reduce program that instead of reading/writing the primitive LongWritable,IntWritable, etc. classes I'm using a custom object that ...
    Ryan LeCompteRyan LeCompte
    Aug 30, 2008 at 2:10 pm
    Aug 30, 2008 at 8:19 pm
  • Hello, I am working on a Hadoop application that produces different (key,value) types after the map and reduce phases so I'm aware that I need to use "JobConf.setMapOutputKeyClass" and ...
    Jim TwenskyJim Twensky
    Aug 29, 2008 at 10:52 pm
    Aug 30, 2008 at 3:37 am
  • Hi, When I am running two hadoop processes in parallel and both process has to read same file. It fails. Of course one solution is to keep copy of file into different location so that accessing ...
    Deepak DiwakarDeepak Diwakar
    Aug 28, 2008 at 1:07 pm
    Aug 29, 2008 at 10:50 am
  • Hi, I want to know how many records were written by the reducer via API. Should I define my own counter or is there a way to get the value of this counter: REDUCE_OUTPUT_RECORD,S that is used by ...
    Tarandeep SinghTarandeep Singh
    Aug 28, 2008 at 9:33 pm
    Aug 29, 2008 at 12:07 am
  • Minor issue in case anyone else gets tripped up trying to use IE7 to view Hadoop Web GUIs like I have been. My server configuration is all Fedora 9. I have two master machines running NameNode and ...
    Andy FraleyAndy Fraley
    Aug 26, 2008 at 8:57 pm
    Aug 28, 2008 at 2:20 am
  • Hi, I want to use HDFS as DFS to store files. I have one data server with 50Gb data and I plan to use 3 new machines with installed HDFS to duplicate this data. These 3 machines are: 1 name node, 2 ...
    Victor SamoylovVictor Samoylov
    Aug 26, 2008 at 9:54 pm
    Aug 27, 2008 at 5:44 pm
  • Hi All Have anyone tried the Eclipse plugin for Hadoop? I've been able to add the plugin to Eclipse, however, I can see couple of problems there - probably I did something wrong. (1) When I try to ...
    Arijit MukherjeeArijit Mukherjee
    Aug 26, 2008 at 5:07 am
    Aug 26, 2008 at 6:50 am
Aug 2008
1234
Group Navigation
period‹ prev | Aug 2008 | next ›
Group Overview
groupcommon-user @
categorieshadoop
discussions167
posts539
users179
websitehadoop.apache.org...
irc#hadoop

179 users for August 2008

Kevin: 26 posts Arun C Murthy: 16 posts Lohit: 16 posts Mork0075: 13 posts Karl Anderson: 12 posts Owen O'Malley: 12 posts Steve Loughran: 11 posts James Graham (Greywolf): 10 posts Edward J. Yoon: 9 posts Gerardo Velez: 9 posts Allen Wittenauer: 7 posts Deepika Khera: 7 posts Joel Welling: 7 posts Konstantin Shvachko: 7 posts Pete Wyckoff: 7 posts Qin Gao: 7 posts Jason Venner: 6 posts Leon Mergen: 6 posts Miles Osborne: 6 posts Shirley Cohen: 6 posts
show more
Archives