FAQ

Hadoop > common-user > January 2011

Search Discussions

118 discussions - 514 posts

  • Hi folks, I'm a TOTAL newbie on hadoop. I have an existing webapp that has a growing number of rows in a mysql database that I have to compare against one another once a day from a batch job. This is ...
    Brian McSweeneyBrian McSweeney
    Jan 8, 2011 at 11:33 pm
    Jan 14, 2011 at 8:24 pm
  • Hi All, I have been working on running Hadoop on a new microprocessor architecture in pseudo-distributed mode. I have been successful in getting SSH configured. I am also able to start a namenode, ...
    Jon LedermanJon Lederman
    Jan 1, 2011 at 6:11 pm
    Jan 4, 2011 at 4:48 am
  • Dear all, Yesterday I performed a kind of testing between *Hadoop in Standalone Servers* & *Hadoop in Cloud. *I establish a Hadoop cluster of 4 nodes ( Standalone Machines ) in which one node act as ...
    Adarsh SharmaAdarsh Sharma
    Jan 17, 2011 at 4:08 am
    Feb 3, 2011 at 11:36 am
  • Dear all, Yesterday I was working on a cluster of 6 Hadoop nodes ( Load data, perform some jobs ). But today when I start my cluster I came across a problem on one of my datanodes. Datanodes fails to ...
    Adarsh SharmaAdarsh Sharma
    Jan 11, 2011 at 10:36 am
    Jan 18, 2011 at 11:44 am
  • Hi, For the first time I am about to apply a patch to HDFS. https://issues.apache.org/jira/browse/HDFS-630 Above is the one that I am trying to do. But there are like 15 patches and I don't know ...
    Edward choiEdward choi
    Jan 10, 2011 at 9:11 am
    Jan 22, 2011 at 5:55 am
  • Hello, I try to measure the performance of HDFS, but the writing rate is quite low. When the replication factor is 1, the rate of writing to HDFS is about 60MB/s. When the replication factor is 3, ...
    Da ZhengDa Zheng
    Jan 25, 2011 at 8:36 pm
    Jan 27, 2011 at 4:18 am
  • Hi, we would like to limit the number of maximum tasks per job on our hadoop 0.20.2 cluster. Is the Capacity Scheduler [1] will allow to do this ? Is it correctly working on hadoop 0.20.2 (I remember ...
    Renaud DelbruRenaud Delbru
    Jan 25, 2011 at 11:50 am
    Jan 29, 2011 at 11:04 am
  • All, I have been running terasort on a 480 node hadoop cluster. I have also collected cpu,memory,disk, network statistics during this run. The system stats are quite intersting. I can post it when I ...
    Raj VRaj V
    Jan 11, 2011 at 5:07 am
    Jan 13, 2011 at 4:51 pm
  • Hi, Is there a way to drain a tasktracker. What we require is not to schedule any more map/red tasks onto a tasktracker(mark it offline) but still the running tasks should not be affected. -- --- ...
    Rishi pathakRishi pathak
    Jan 28, 2011 at 9:10 am
    Jan 31, 2011 at 5:35 pm
  • I am seeing very perplexing segfaults and standard allocation exceptions in my native code (.so files passed to the distributed cace) which is called via JNI from the map task. This code runs ...
    Keith WileyKeith Wiley
    Jan 28, 2011 at 1:46 am
    Jan 29, 2011 at 6:16 am
  • Hello, I see over on http://search-hadoop.com/?q=monit+daemontools that people *do* use tools like monit and daemontools (and a few other ones) to keep revive their Hadoop processes when they die. ...
    Otis GospodneticOtis Gospodnetic
    Jan 3, 2011 at 10:23 am
    Jan 6, 2011 at 7:20 pm
  • Does anyone know of a source of click stream data for a student research project? Bruce Williams Concepts, like individuals, have their histories and are just as incapable of withstanding the ravages ...
    Bruce WilliamsBruce Williams
    Jan 30, 2011 at 6:55 pm
    Jan 30, 2011 at 9:03 pm
  • Hi Hadoopers, I am trying to connect hadoop from eclipse to run MapReduce jobs. I am getting the following error while connecting to the DFS Locations in eclipse, "Error: Call to ...
    Raghu RRaghu R
    Jan 19, 2011 at 4:32 am
    Jan 28, 2011 at 5:48 pm
  • Hello, I need to build hadoop in Linux as I need to make some small changes in the code, but I don't know what is the simplest way to build hadoop. I googled it and so far I only found two places ...
    Da ZhengDa Zheng
    Jan 1, 2011 at 6:20 pm
    Jan 3, 2011 at 4:42 am
  • Hi, all I want to run lucene on Hadoop, The problem as follows: IndexWriter writer = new IndexWriter(FSDirectory.open(new File("index")),new StandardAnalyzer(), true, ...
    Jander gJander g
    Jan 1, 2011 at 5:26 pm
    Jan 3, 2011 at 2:30 am
  • Hi- I have a basic question. Appologies for my ignorance, but is hadoop a mis-fit for a data architect with zero java knowledge? -- View this message in context: ...
    ManoranjandManoranjand
    Jan 26, 2011 at 3:43 pm
    Jan 27, 2011 at 9:21 pm
  • We have a jobs which runs in several map/reduce stages. In the first job, a large number of map tasks -82 are initiated, as expected. And that cause all nodes to be used. In a later job, where we are ...
    Tali KTali K
    Jan 7, 2011 at 8:59 pm
    Jan 8, 2011 at 4:13 am
  • Hi, all I have a question about the file transmission between Map and Reduce stage, in current implementation, the Reducers get the results generated by Mappers through HTTP Get, I don't understand ...
    Nan ZhuNan Zhu
    Jan 13, 2011 at 8:28 am
    Jan 16, 2011 at 2:24 pm
  • Hi, my list file command hadoop fs -ls hdfs://master-url/ works locally on the master, but cannot connect from any of the slaves. What should I check for? Thank you, Mark
    Mark KerznerMark Kerzner
    Jan 14, 2011 at 1:02 am
    Jan 14, 2011 at 4:51 am
  • Hi, Could anyone please guide me as to how to use the -libjars option in HDFS? I have added the necessary jar file (the hbase jar - to be precise) to the classpath of the node where I am starting the ...
    C.V.Krishnakumar IyerC.V.Krishnakumar Iyer
    Jan 11, 2011 at 6:11 pm
    Jan 11, 2011 at 11:37 pm
  • Hi All, number of map task: 1000s number of reduce task:single digit In such cases the reduce task wont started even when few map task are completed. Example: In my observation of a sample run of ...
    Sagar naikSagar naik
    Jan 4, 2011 at 6:54 pm
    Jan 5, 2011 at 6:40 am
  • Hi All, I am new to Hadoop and currently evaluating if Hadoop can be the choice for the existing system at work. following system is a good candidate for Hadoop System details: 1 Tax calculating ...
    Saurabh GokhaleSaurabh Gokhale
    Jan 2, 2011 at 12:14 am
    Jan 4, 2011 at 4:37 am
  • My Java mappers use JNI to call native .so files compiled from C++. In some cases, the task status ends with exit 139, which generally indicates a seg-fault. I would like to see the core-dump, but I ...
    Keith WileyKeith Wiley
    Jan 21, 2011 at 8:58 pm
    Jan 28, 2011 at 9:16 pm
  • Hi, Am trying to install Hadoop on a linux cluster(Fedora 12). However, am not able to SSH to localhost and gives the following error. *ssh_exchange_identification: Connection closed by remote host* ...
    Real great..Real great..
    Jan 25, 2011 at 6:08 am
    Jan 25, 2011 at 12:22 pm
  • Dear all, Designing the architecture is very important for the Hadoop in Production Clusters. We are researching to run Hadoop in Cloud in Individual Nodes and in Cloud Environment ( VM's ). For ...
    Adarsh SharmaAdarsh Sharma
    Jan 4, 2011 at 7:27 am
    Jan 6, 2011 at 5:54 am
  • I installed the Apache distribution <http://hadoop.apache.org/ of Hadoop on my laptop and set it up to run in local mode. It's working for me, but I can't find the hadoop-streaming.jar file. It is ...
    W.P. McNeillW.P. McNeill
    Jan 1, 2011 at 7:48 pm
    Jan 1, 2011 at 8:12 pm
  • Hi all, I was working on a MapReduce program which does BytesWritable dataprocessing. But currently I am basically running two MapReduces consecutively to get the final output : Input ...
    Matthew JohnMatthew John
    Jan 25, 2011 at 11:46 am
    Feb 23, 2011 at 3:41 am
  • Hello everyone, I am building an hadoop "app" to quickly index a corpus of documents. This app will accept one or more XML file that will contain the corpus. Each document is made up of several ...
    Marco DidonnaMarco Didonna
    Jan 28, 2011 at 10:50 am
    Jan 29, 2011 at 8:19 pm
  • Hi, I am seeing an odd problem when writing block compressed sequence files. If I write 400,000 records into a sequence file w/o compression, all 400K end up in the file. If I write with block, ...
    David SinclairDavid Sinclair
    Jan 21, 2011 at 9:04 pm
    Jan 24, 2011 at 9:36 pm
  • I want to get metrics information (e.g. number of Maps, number of Reduces, memory use, load) by APIs. I found two useful classes - ClusterStatus and ClusterMetrics. My question is how I can get ...
    Zhenhua GuoZhenhua Guo
    Jan 22, 2011 at 5:58 pm
    Jan 23, 2011 at 2:21 pm
  • After a tragic cluster crash it looks like some blocks are missing. Total size: 343918527293 B (Total open files size: 67108864 B) Total dirs: 5897 Total files: 5574 (Files currently being written: ...
    Mike andersonMike anderson
    Jan 21, 2011 at 8:03 pm
    Jan 21, 2011 at 8:20 pm
  • Hi Does anybody has idea of how (.hiverc) file looks like. Thanks Ankit -- View this message in context: http://lucene.472066.n3.nabble.com/Hive-rc-tp2296028p2296028.html Sent from the Hadoop ...
    AbhatnagarAbhatnagar
    Jan 20, 2011 at 10:29 pm
    Jan 21, 2011 at 3:14 pm
  • Hi, I am using Hadoop 0.20.2 version on my cluster. To write multiple output files from a reducer, I want to use MultipleOutputs class. in this class, I need to call addNamedOutput. addNamedOutput ...
    Jun Young KimJun Young Kim
    Jan 20, 2011 at 9:29 am
    Jan 20, 2011 at 11:55 am
  • Hi, In my cluster, Hadoop somehow cannot work, and I found that it was due to the Jetty-6.1.14 which is not able to start up. However, Jetty 7 can work in my cluster. Could any body know how to ...
    Xiufeng liuXiufeng liu
    Jan 16, 2011 at 9:42 am
    Jan 20, 2011 at 9:24 am
  • Hey all Why does the FCFS scheduler only let a node chooses one task at a time in one job? In order to increase the data locality, it is reasonable to let a node to choose all its local tasks (if it ...
    He ChenHe Chen
    Jan 15, 2011 at 5:46 am
    Jan 17, 2011 at 4:47 pm
  • Hi, I have a sort job consisting of only the Mapper (no Reducer) task. I want my results to contain only the top n records. Is there any way of restricting the number of records that are emitted by ...
    Rakesh DavanumRakesh Davanum
    Jan 12, 2011 at 6:45 pm
    Jan 14, 2011 at 4:47 pm
  • How to create the hadoop-0.21.0-core.jar using the source code? Now when I compile the code, I need three or more jar files common,hdfs and mapred. I want to build the hadoop-0.21.0-core.jar to run a ...
    MagicorMagicor
    Jan 13, 2011 at 11:52 pm
    Jan 13, 2011 at 11:52 pm
  • Hi, I have a Java program to upload and download files from the HDFS. I am using Hadoop with Kerberos. I am able to get a TGT(From the Authentication Server ) and a service Ticket(From Ticket ...
    Muruga Prabu MMuruga Prabu M
    Jan 13, 2011 at 4:47 am
    Jan 13, 2011 at 6:58 am
  • hi i have format the name node and now when i restart the cluster, i am getting the strange error. kindly let me know how to fix it. thnx /************************************************************ ...
    Shuja RehmanShuja Rehman
    Jan 8, 2011 at 9:09 pm
    Jan 10, 2011 at 5:58 am
  • Hi Can any one you let me know what command do I need to execute for Decommissioning TaskTracker? Datanode decommissioning I have achieved using hadoop dfsadmin -refreshNodes. Similar to HDFS is ...
    SandeepSandeep
    Jan 6, 2011 at 11:35 am
    Jan 7, 2011 at 9:11 pm
  • Hello, direct IO can make huge performance difference, especially when Atom processors are used. but as far as I know, hadoop doesn't enable direct IO of Linux. Does anyone know any unofficial ...
    Da ZhengDa Zheng
    Jan 2, 2011 at 8:01 pm
    Jan 5, 2011 at 8:36 pm
  • Hi, what would you think of these exercises<http://hadoopinpractice.blogspot.com/2011/01/exercises-for-chapter-1-how-do-they.html for the Hadoop intro chapter? Thank you, Mark
    Mark KerznerMark Kerzner
    Jan 5, 2011 at 3:49 pm
    Jan 5, 2011 at 6:05 pm
  • Hello, I monitored system calls of HDFS with systemtap and found HDFS actually sends many 1-byte data to the network. I could also see many 8-byte and 64-byte data written to the OS though I don't ...
    Da ZhengDa Zheng
    Jan 29, 2011 at 3:09 am
    Jan 31, 2011 at 5:04 pm
  • Hi, I'm trying to develop an image processing application with hadoop. All image files are in HDFS. But I don't know how to read this files with binary/byte stream. What is correct decleration of ...
    F.Ozgur CatakF.Ozgur Catak
    Jan 25, 2011 at 7:46 pm
    Jan 28, 2011 at 1:28 am
  • Hi, we are currently using Hadoop (version 0.20.2) to manage some web archiving processes like fulltext indexing, and it works very well with small records that contains html. Now, we would like to ...
    Jérôme ThièvreJérôme Thièvre
    Jan 6, 2011 at 10:53 am
    Jan 25, 2011 at 10:55 pm
  • Hi, I am a beginner user of a hadoop. almost examples to learn hadoop suggest to use a jar style to use a hadoop framework. (like workcount.jar) in this case, I could find out a job history. but, if ...
    Jun Young KimJun Young Kim
    Jan 24, 2011 at 9:30 am
    Jan 25, 2011 at 3:08 am
  • Hi, I got this error when I executed a hadoop with a my jar application. $ hadoop jar test-hdeploy.jar Test Exception in thread "main" java.lang.NoSuchMethodError: ...
    Jun Young KimJun Young Kim
    Jan 25, 2011 at 2:45 am
    Jan 25, 2011 at 3:01 am
  • I modified Hadoop source code. I need to include my own jar. Where should I put the jar? Do I need to change Hadoop config file? Gerald
    Zhenhua GuoZhenhua Guo
    Jan 24, 2011 at 12:35 pm
    Jan 24, 2011 at 2:44 pm
  • Dear all, I am researching about the below error and could not able to find the reason : Data Size : 3.4 GB Hadoop-0.20.0 hadoop@ws32-test-lin:~/project/hadoop-0.20.2$ bin/hadoop jar ...
    Adarsh SharmaAdarsh Sharma
    Jan 7, 2011 at 12:44 pm
    Jan 12, 2011 at 7:19 am
  • "If you are not using hbase-0.20.3, you will need to rebuild the handler with the HBase jar matching your version, and change the --auxpath above accordingly. Failure to use matching versions will ...
    Adarsh SharmaAdarsh Sharma
    Jan 5, 2011 at 9:08 am
    Jan 8, 2011 at 3:07 am
Jan 2011
123
Group Navigation
period‹ prev | Jan 2011 | next ›
Group Overview
groupcommon-user @
categorieshadoop
discussions118
posts514
users144
websitehadoop.apache.org...
irc#hadoop

144 users for January 2011

Harsh J: 31 posts Adarsh Sharma: 29 posts Ted Dunning: 19 posts Da Zheng: 18 posts Brian McSweeney: 14 posts Jun Young Kim: 13 posts Allen Wittenauer: 12 posts Keith Wiley: 12 posts Steve Loughran: 11 posts Jon Lederman: 10 posts Black, Michael (IS): 9 posts Ted Yu: 9 posts Edward Capriolo: 8 posts Greg Roelofs: 8 posts Li ping: 8 posts Maha: 8 posts Mark Kerzner: 8 posts Konstantin Boudnik: 7 posts W.P. McNeill: 7 posts Cavus,M.,Fa. Post Direkt: 6 posts
show more
Archives