FAQ

Search Discussions

127 discussions - 492 posts

  • Does hadoop 0.20.205 supports configuring HDFS backup nodes ? Thanks, Praveenesh
    Praveenesh kumarPraveenesh kumar
    Dec 7, 2011 at 7:10 am
    Dec 14, 2011 at 9:58 pm
  • Hey Guys, So I have a very naive question in my mind regarding Hadoop cluster nodes ? more cores or more nodes – Shall I spend money on going from 2-4 core machines, or spend money on buying more ...
    Praveenesh kumarPraveenesh kumar
    Dec 13, 2011 at 4:51 am
    Dec 17, 2011 at 2:58 pm
  • or Do I have to apply some hadoop patch for this ? Thanks, Praveenesh
    Praveenesh kumarPraveenesh kumar
    Dec 2, 2011 at 11:32 am
    Mar 1, 2012 at 2:44 am
  • Hi all, Can anyone guide me how to automate the hadoop installation/configuration process? I want to install hadoop on 10-20 nodes which may even exceed to 50-100 nodes ? I know we can use some ...
    Praveenesh kumarPraveenesh kumar
    Dec 5, 2011 at 10:33 am
    Dec 31, 2011 at 5:58 am
  • someone please help me to configure hadoop such as core-site.xml, hdfs-site.xml, mapred-site.xml etc. please provide some example. it is badly needed. because i run in a 2 node cluster. when i run ...
    Humayun kabirHumayun kabir
    Dec 22, 2011 at 9:25 am
    Dec 26, 2011 at 4:53 pm
  • Hi, Can someone please send me the Hadoop comic. Saw references about it in the mailing list. Regards, Shreya This e-mail and any files transmitted with it are for the sole use of the intended ...
    Shreya PalShreya Pal
    Dec 7, 2011 at 9:48 am
    Dec 8, 2011 at 8:38 am
  • Hello, I am having problems getting my hadoop eclipse plugin to work on Mac OS X Lion. I have tried the following combinations: Hadoop 0.20.203, Eclipse 3.6.2 (32-bit), ...
    Will LWill L
    Dec 2, 2011 at 8:27 am
    Dec 10, 2011 at 8:21 am
  • Hi everyone, I want to run a MR job continuously. Because i have streaming data and i try to analyze it all the time in my way(algorithm). For example you want to solve wordcount problem. It's the ...
    BurakkkBurakkk
    Dec 5, 2011 at 8:49 pm
    Dec 11, 2011 at 11:14 am
  • All I am encountering the following out-of-memory error during the reduce phase of a large job. Map output copy failure : java.lang.OutOfMemoryError: Java heap space at ...
    Niranjan BalasubramanianNiranjan Balasubramanian
    Dec 7, 2011 at 6:51 pm
    Dec 11, 2011 at 1:32 am
  • Hi guys ! Apart from generating the job traces from RUMEN , can i get logs or job traces of varied sizes from some organizations. How can i make sure that the rumen generates only say 25 jobs,50 jobs ...
    ArunKumarArunKumar
    Dec 1, 2011 at 3:18 am
    Dec 7, 2011 at 5:55 pm
  • I have looked up how to up this value on the web and have tried all suggestions to no avail. Any help would be great. Here is some background: Version: 0.20.2, r911707 Compiled: Fri Feb 19 08:07:34 ...
    Joey KrabacherJoey Krabacher
    Dec 15, 2011 at 11:31 pm
    Dec 29, 2011 at 4:33 pm
  • Hi folks, I've just done a fresh install of Hadoop, Namenode and datanode are up, Task/job Tracker also up, but when I run the Map reduce worcount exemple I got this error on Task tracker: 2011-12-23 ...
    Anthony garnierAnthony garnier
    Dec 23, 2011 at 2:33 pm
    Dec 27, 2011 at 4:18 pm
  • Hi guys ! I want to see the behavior of a single node of Hadoop cluster when IO intensive / CPU intensive workload and mix of both is submitted to the single node alone. These workloads must stress ...
    ArunKumarArunKumar
    Dec 9, 2011 at 7:25 am
    Dec 10, 2011 at 3:10 am
  • In my company, we intend to set up an hadoop cluster to run analylitics applications. This cluster would have about 120 data nodes with dual sockets servers with a GB interconnect. We are also ...
    CussolCussol
    Dec 16, 2011 at 11:51 am
    Apr 19, 2012 at 9:06 am
  • Hi Hadoopers, We're running Hadoop 0.20 CentOS5.5. I am finding the way to collect CPU time, memory usage, IOPS of each hadoop Job. What would be the good starting point ? document ? api ? Thanks in ...
    Patai SangbutsarakumPatai Sangbutsarakum
    Dec 20, 2011 at 8:44 pm
    Jan 3, 2012 at 12:09 pm
  • Hi everyone, I am trying to build Pig from SVN trunk on hadoop 0.20.205. While doing that, I am getting the following error : Any idea why its happening ? Thanks, Praveenesh root@lxe ...
    Praveenesh kumarPraveenesh kumar
    Dec 30, 2011 at 5:29 am
    Dec 30, 2011 at 7:45 am
  • Hey people, How can we setup another machine in the cluster as Secondary Namenode in hadoop 0.20.205 ? Can a DN also act as SNN, any pros and cons of having this configuration ? Thanks, Praveenesh
    Praveenesh kumarPraveenesh kumar
    Dec 26, 2011 at 11:36 am
    Dec 27, 2011 at 7:09 am
  • Our CDH2 production grid just crashed with some sort of master node failure. When I went in there, JobTracker was missing and NameNode was up. Trying to ls on HDFS met with no connection. We decided ...
    Meng MaoMeng Mao
    Dec 17, 2011 at 7:03 am
    Dec 17, 2011 at 9:05 pm
  • Hi guys ! I see that we can set number of map/reduce tasks per tasktracker but how do i set number of tasktrackers per node in a cluster ? Arun -- View this message in context: ...
    ArunKumarArunKumar
    Dec 15, 2011 at 5:14 am
    Dec 15, 2011 at 6:54 am
  • Hi, Can someone please tell me which versions of hadoop contain the 20-appender code and will work with HBase? According to the Hbase docs (http://hbase.apache.org/book/hadoop.html), Hadoop 0.20.205 ...
    JcfolsomJcfolsom
    Dec 6, 2011 at 7:26 pm
    Dec 6, 2011 at 8:15 pm
  • Hello all, I have asked this question a couple of days ago but no one responded. I built a 6 node hadoop cluster, guided Michael Noll, starting with a single node and expanding it one by one. Every ...
    Sloot, Hans-PeterSloot, Hans-Peter
    Dec 20, 2011 at 12:43 pm
    Dec 20, 2011 at 8:27 pm
  • Hi, I am using 0.20.X branch. However, I need to use the new API because it has the cleanup(context) method in Mapper. However, I am confused about how to load the cached files in mapper. I could ...
    Shi YuShi Yu
    Dec 15, 2011 at 10:39 pm
    Dec 17, 2011 at 11:18 pm
  • I would like join some db tables, possibly from different databases, in a MR job. I would essentially like to use MultipleInputs, but that seems file oriented. I need a different mapper for each db ...
    Justin VincentJustin Vincent
    Dec 5, 2011 at 6:43 pm
    Dec 6, 2011 at 11:40 am
  • I have the following error in running hadoop streaming, PipeMapRed\.waitOutputThreads(): subprocess failed with code 126 at ...
    Daniel YehdegoDaniel Yehdego
    Dec 4, 2011 at 3:10 am
    Dec 4, 2011 at 4:01 am
  • Hi, I am trying to add nodes dynamically to a running hadoop cluster.I started tasktracker and datanode in the node. It works fine. But when some node try fetch values ( for reduce phase) it fails ...
    Madhu phatakMadhu phatak
    Dec 17, 2011 at 11:06 am
    Jan 3, 2012 at 2:18 pm
  • Hi, Installed 0.22.0 on CentOS 5.7. I can start dfs and mapred and see their processes. Ran the first grep example: bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'. It seems the ...
    Pat FlahertyPat Flaherty
    Dec 23, 2011 at 11:40 pm
    Dec 29, 2011 at 11:23 pm
  • Hey guys, How we can make hadoop as multiuser ? One way to think as whatever group we currently assigned to use hadoop, add users to same group and change permissions to hadoop.tmp.dir, ...
    Praveenesh kumarPraveenesh kumar
    Dec 27, 2011 at 11:07 am
    Dec 29, 2011 at 12:26 pm
  • Hi, I have a 7-node setup (1 - Namenode/JobTracker, 6 - Datanodes/TaskTrackers) running Hadoop version 0.20.203. I performed the following test: Initially cluster is running smoothly. Just before ...
    Rajat GoelRajat Goel
    Dec 27, 2011 at 1:09 pm
    Dec 28, 2011 at 6:34 am
  • Hi all, How many files a datanode can hold? In my test case, when a datanode save 14million files, the cluster can't work. Bourne
    Bourne1900Bourne1900
    Dec 23, 2011 at 2:35 am
    Dec 26, 2011 at 1:48 am
  • When installing hadoop on slave machines, do we have to install hadoop at same locations on each machine ? Can we have hadoop installation at different location on different machines at same cluster ...
    Praveenesh kumarPraveenesh kumar
    Dec 23, 2011 at 12:51 pm
    Dec 23, 2011 at 8:05 pm
  • Hi, I am having problems with changing the default hadoop scheduler (i assume that the default scheduler is a FIFO scheduler). I am following the guide located in hadoop/docs directory however I am ...
    Merto MertekMerto Mertek
    Dec 20, 2011 at 4:51 pm
    Dec 21, 2011 at 5:08 pm
  • HI guys ! I have a single node set up as per http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/ 1 I have put some sysout statements in Jobtracker and wordcount ...
    ArunKumarArunKumar
    Dec 13, 2011 at 2:39 pm
    Dec 13, 2011 at 5:40 pm
  • Hai guys ! Can i access the Job counters displayed in WEB GUI in Hadoop code when the job finished their execution ? If so, how can i access the values like "average task run time" and counters ...
    ArunKumarArunKumar
    Dec 11, 2011 at 6:45 am
    Dec 12, 2011 at 8:55 am
  • Hi, Does anyone know any discussion in Apache Hadoop regarding the claim by Parrallel Iron with their patent against use of HDFS? Thanks in advance. Regards, JS
    JS JangJS Jang
    Dec 9, 2011 at 2:53 am
    Dec 9, 2011 at 4:08 am
  • Hi - We have two namenodes set up at our company, say: hdfs://A.mycompany.com hdfs://B.mycompany.com Hadoop fs –ls hdfs://A.mycompany.com//some-dir And Hadoop fs –ls ...
    Frank AstierFrank Astier
    Dec 8, 2011 at 10:43 pm
    Dec 9, 2011 at 2:32 am
  • hi, i am reading hadoop-0.23 source code, mainly focusing on hadoop yarn. However i have some problems in reading the source code. There is no Debugging tools for hadoop, so i can't track the code ...
    陈竞陈竞
    Dec 8, 2011 at 5:51 am
    Dec 8, 2011 at 8:30 am
  • After my Hadoop job has successfully completed I'd like to log the total amount of time it took. This is the "Finished in" statistic in the web UI. How do I get this number programmatically? Is there ...
    W.P. McNeillW.P. McNeill
    Dec 2, 2011 at 5:57 pm
    Dec 3, 2011 at 6:19 am
  • Hi All, I wrote a map reduce program to fetch data from MySQL and process the data(word count). The program executes successfully . But I noticed that the reduce task starts after finishing the map ...
    JAGANADH GJAGANADH G
    Dec 28, 2011 at 4:53 pm
    Dec 30, 2011 at 5:41 am
  • Dear All, any one did it before: map/reduce by C# & Hadoop ??? As you know for developing map/reduce app in hadoop we should extend and implement special map and reduce abstract classes and ...
    MasoudMasoud
    Dec 27, 2011 at 9:02 am
    Dec 28, 2011 at 1:26 am
  • Hi, In HDFS, how do I find out what are all the blocks that belong to a particular file and where each one of these blocks (incl. the replicas) is located? Thank you. Regards, Hemanth Makkapati
    Hemanth MakkapatiHemanth Makkapati
    Dec 24, 2011 at 6:11 pm
    Dec 25, 2011 at 6:36 pm
  • Hello All, I have a situation to dump cassandra data to hadoop cluster for further analytics. Lot of other relevant data which is not present in cassandra is already available in hdfs for analysis. ...
    Ravikumar visweswaraRavikumar visweswara
    Dec 23, 2011 at 3:14 pm
    Dec 24, 2011 at 5:55 am
  • For a hadoop cluster that starts medium size (50 nodes) but could grow to hundred of nodes, what is the recommended network in the rack? 1gig or 10gig We have machines with 8 cores, 4 X 1tb drive ...
    Koert KuipersKoert Kuipers
    Dec 23, 2011 at 5:24 pm
    Dec 23, 2011 at 7:37 pm
  • Is there a tool or a method to measure the throughput of the cluster at a given time? It would be a great feature to add -- --- Get your facts first, then you can distort them as you please.--
    RitaRita
    Dec 22, 2011 at 12:58 pm
    Dec 22, 2011 at 1:53 pm
  • dear all i am trying for many days to get a simple hadoop cluster (with 2 nodes) to work but i have trouble configuring the network parameters. i have properly configured the ssh keys, and the ...
    MirrorXMirrorX
    Dec 21, 2011 at 1:22 am
    Dec 22, 2011 at 5:41 am
  • Hello all, I've found the "mapFinishTime" in the job statistic log very strange in the 0.23 version of Hadoop. Sometimes the difference between "mapFinishTime" and "startTime" is negative. Sometimes, ...
    FujizawaMiyukiFujizawaMiyuki
    Dec 18, 2011 at 7:54 am
    Dec 20, 2011 at 6:31 am
  • Hi guys, We recently had the following problem on our production cluster: The filesystem containing the editlog and fsimage had no free inodes. As a result the namenode wasn't able to obtain an inode ...
    Guy DoulbergGuy Doulberg
    Dec 15, 2011 at 9:17 am
    Dec 18, 2011 at 1:19 am
  • Hi all, I would like to know what options I have to ingest terabytes of data that are being generated very fast from a small set of sources. I have thought about : 1. Flume 2. Have an intermediate ...
    Periya.DataPeriya.Data
    Dec 16, 2011 at 1:01 am
    Dec 16, 2011 at 2:52 am
  • Hi all New to the community and using hadoop and was looking for some advice as to optimal configurations on very large servers. I have a single server with 48 cores and 512GB of RAM and am looking ...
    Dale McDiarmidDale McDiarmid
    Dec 15, 2011 at 7:50 pm
    Dec 15, 2011 at 10:15 pm
  • Hi, I am planning a system to process information with Hadoop, and I will have a few look-up tables that each processing node will need to query. There are perhaps 20-50 such tables, and each has on ...
    Mark KerznerMark Kerzner
    Dec 12, 2011 at 6:05 pm
    Dec 13, 2011 at 4:56 am
  • Hi The system we are going to work on will receive 50mio+ new datarecords every day. We need to keep a history of 2 years of data (thats 35+ billion datarecords in the storage all in all), and that ...
    Per SteffensenPer Steffensen
    Dec 8, 2011 at 1:14 pm
    Dec 9, 2011 at 7:08 am
Group Navigation
period‹ prev | Dec 2011 | next ›
Group Overview
groupcommon-user @
categorieshadoop
discussions127
posts492
users155
websitehadoop.apache.org...
irc#hadoop

155 users for December 2011

Harsh J: 31 posts ArunKumar: 26 posts Praveenesh kumar: 26 posts Alo alt: 17 posts Michel Segel: 13 posts Joey Krabacher: 10 posts Prashant Kommireddi: 10 posts Bejoy Ks: 9 posts Joey Echeverria: 8 posts Amar Kamat: 7 posts Praveen Sripati: 7 posts Todd Lipcon: 7 posts Uma Maheswara Rao G: 7 posts Anthony garnier: 6 posts Arun C Murthy: 6 posts JAGANADH G: 6 posts Shi Yu: 6 posts Will L: 6 posts Shreya Pal: 5 posts Devaraj K: 5 posts
show more