Search Discussions

103 discussions - 433 posts

  • I'm a newbie and I am confused by the Hadoop releases. I thought 0.21.0 is the latest & greatest release that I should be using but I noticed 0.20.203 has been released lately, and 0.21.X is marked ...
    Teruhiko KurosakaTeruhiko Kurosaka
    Jul 14, 2011 at 11:34 pm
    Jul 19, 2011 at 12:10 pm
  • Hi guys! I'd like some help fine tuning my cluster. I currently have 20 boxes exactly alike. Single core machines with 600MB of RAM. No chance of upgrading the hardware. My cluster is made out of 1 ...
    Juan P.Juan P.
    Jul 7, 2011 at 8:30 pm
    Jul 17, 2011 at 4:54 pm
  • Hi all, Does anybody have examples of how one moves files from the local filestructure/HDFS to the distributed cache in MapReduce? A Google search turned up examples in Pig but not MR. -- Roger Chen ...
    Roger ChenRoger Chen
    Jul 29, 2011 at 5:26 pm
    Aug 1, 2011 at 12:25 pm
  • Hi, I am working on a open source project Nectar<https://github.com/zinnia-phatak-dev/Nectar where i am trying to create the hadoop jobs depending upon the user input. I was using Java Process API to ...
    Madhu phatakMadhu phatak
    Jul 26, 2011 at 9:59 am
    Jul 28, 2011 at 5:32 am
  • I am trying to parallelize some very long RNA sequence for the sake of predicting their RNA 2D structures. I am using a binary executable c program called pknotsRG as my mapper. I tried the following ...
    Daniel YehdegoDaniel Yehdego
    Jul 22, 2011 at 3:56 pm
    Dec 3, 2011 at 3:33 am
  • Hi, I am trying to monitor the time to complete a map phase and reduce phase in hadoop. Is there any way to measure the time taken to complete map and reduce phase in a cluster. Thanks, Amit -- View ...
    Jul 5, 2011 at 2:56 am
    Jul 8, 2011 at 5:35 am
  • Dear all, Today I am stucked with the strange problem in the running hadoop cluster. After starting hadoop by bin/start-all.sh, all nodes are started. But when I check through web UI ( ...
    Adarsh SharmaAdarsh Sharma
    Jul 7, 2011 at 11:13 am
    Jul 15, 2011 at 2:56 pm
  • I was asked by our IT folks if we can put hadoop name nodes storage using a shared disk storage unit. Does anyone have experience of how much IO throughput is required on the name nodes? What are the ...
    Jonathan HwangJonathan Hwang
    Jul 31, 2011 at 7:09 pm
    Aug 3, 2011 at 1:05 am
  • We are in the process of upgrading to the most current version of Hadoop. At the same time we are in need of upgrading Java. We are currently running u17. I have read elsewhere that u21 or up is the ...
    High pointeHigh pointe
    Jul 18, 2011 at 5:26 pm
    Jul 27, 2011 at 10:42 am
  • Hi Folks, Just doing a sanity check here. I have a map-only job, which produces a filename for a key and data as a value. I want to write the value (data) into the key (filename) in the path ...
    Tom MelendezTom Melendez
    Jul 25, 2011 at 8:26 pm
    Jul 26, 2011 at 7:08 pm
  • Hi, Has anyone here used hadoop to process more than 3TB of data? If so we would like to know how many machines you used in your cluster and about the hardware configuration. The objective is to know ...
    Karthik KumarKarthik Kumar
    Jul 6, 2011 at 10:44 am
    Jul 7, 2011 at 6:33 am
  • Hi all ! I have downloaded hadoop-0.21.I am behind my college proxy. Installed :- ivy version : 2.1.0~rc2-3ubuntu1 Ant version : 1.7.1-4ubuntu1.1 I get the following error while building mumak : $cd ...
    Arun KArun K
    Jul 29, 2011 at 10:03 am
    Sep 10, 2011 at 5:17 pm
  • I am trying to run some benchmarks test with integrated hadoop schedulers and make some analysis of its performances. Because the current version from svn is not stable I am planning to do some ...
    Jul 20, 2011 at 5:18 pm
    Sep 23, 2011 at 1:35 pm
  • In which Hadoop version is next gen introduced? -- Regards, R.V.
    Real great..Real great..
    Jul 28, 2011 at 12:32 pm
    Aug 1, 2011 at 1:10 pm
  • Dear All: I am trying to run Hadoop on Windows 7 so as to test programs before moving to Unix/Linux. I have downloaded the Hadoop 0.20.2 and Eclipse 3.6 because I want to use the plugin. I am also ...
    A DfA Df
    Jul 26, 2011 at 6:34 pm
    Jul 27, 2011 at 4:08 pm
  • Not considering replication, if I use following command from a hadoop client outside the cluster(the client is not a datanode) hadoop dfs -put <localfilename hdfs://<datanode ip :50010/<filename Can ...
    Jul 21, 2011 at 1:34 am
    Jul 22, 2011 at 3:01 pm
  • I have a dataset which is several terabytes in size. I would like to query this data using hbase (sql). Would I need to setup mapreduce to use hbase? Currently the data is stored in hdfs and I am ...
    Jul 11, 2011 at 11:32 am
    Jul 13, 2011 at 12:27 pm
  • Hi, What is the difference between DFS Used and Non-DFS used ? Thanks, Sagar DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent ...
    Sagar ShuklaSagar Shukla
    Jul 7, 2011 at 10:01 am
    Jul 8, 2011 at 12:34 pm
  • Hi all, I am attempting to implement MultipleOutputFormat to write data to multiple files dependent on the output keys and values. Can somebody provide a working example with how to implement this in ...
    Roger ChenRoger Chen
    Jul 26, 2011 at 4:12 pm
    Jul 27, 2011 at 1:52 pm
  • I'm having some difficulty with using ArrayWritable in the following test code: ArrayWritable array = new ArrayWritable(IntWritable.class); IntWritable[] ints = new IntWritable[4]; for (int i =0 ; i ...
    Dhruv KumarDhruv Kumar
    Jul 4, 2011 at 6:55 pm
    Dec 13, 2011 at 6:09 pm
  • Hi, I am not sure if this question (as title) has been asked before, but I didn't find an answer by googling. I'd like to explain the scenario of my problem: My program launches several threads in ...
    Yaozhen PanYaozhen Pan
    Jul 1, 2011 at 12:43 pm
    Aug 20, 2011 at 12:28 am
  • Hi, I run Hadoop in 4 Ubuntu 11.04 on VirtualBox. On the master node (, I configure fs.default.name = hdfs:// Then i configure everything on 3 other node When i start ...
    Doan NinhDoan Ninh
    Jul 28, 2011 at 11:52 am
    Jul 28, 2011 at 2:26 pm
  • I am a Hadoop novice so kindly pardon my ingorance. I am running the following Hadoop program in Fully Distributed Mode to count the number of lines in a file. I am running this job from eclipse and ...
    Jul 20, 2011 at 2:05 am
    Jul 21, 2011 at 11:45 am
  • Hi, Are there any plans to support Hadoop architectures natively by any of the existing RDBMS databases such as MySQL etc. My understanding is Hadoop can be the way to store and compute, which are ...
    Raja Nagendra KumarRaja Nagendra Kumar
    Jul 13, 2011 at 6:05 am
    Jul 13, 2011 at 6:14 am
  • Hi, I am new to hadoop. I have a set of files and I want to assign each file to a mapper. Also in mapper there should be a way to know the complete path of the file. Can you please tell me how to do ...
    Govind KothariGovind Kothari
    Jul 5, 2011 at 9:15 pm
    Aug 10, 2011 at 4:04 pm
  • When I run the job, the throws the following error. 11/07/29 22:22:22 INFO mapred.JobClient: Task Id : attempt_201107292131_0011_m_000000_2, Status : FAILED java.io.IOException: Type mismatch in ...
    Jul 29, 2011 at 2:41 pm
    Aug 3, 2011 at 12:00 pm
  • Hi All: I am have Hadoop 0.20.2 and I am using cygwin on Windows 7. I modified the files as shown below for the Hadoop configuration. conf/core-site.xml: <configuration <property <name ...
    A DfA Df
    Jul 27, 2011 at 4:25 pm
    Jul 28, 2011 at 7:50 pm
  • Hello, Please any assistance?? I am using Hadoop for a school project and managed to install it on two computers testing with the wordcount example. However, after stopping Hadoop and restarting the ...
    Kobina KwarkoKobina Kwarko
    Jul 19, 2011 at 7:48 pm
    Jul 19, 2011 at 8:08 pm
  • Hello, It seems that data replication in HDFS is simply data copy among nodes. Has anyone considered to use a better encoding to reduce the data size? Say, a block of data is split into N pieces, and ...
    Da ZhengDa Zheng
    Jul 18, 2011 at 7:41 am
    Jul 19, 2011 at 5:38 am
  • Hi, How can I give filename as key to mapper ? I want to know the occurence of word in set of docs, so I want to keep key as filename. Is it possible to give input key as filename in map function ? ...
    Praveenesh kumarPraveenesh kumar
    Jul 15, 2011 at 12:15 pm
    Jul 15, 2011 at 2:51 pm
  • I have many large files ranging from 2gb to 800gb and I use hadoop fs -cat a lot to pipe to various programs. I was wondering if its possible to prefetch the data for clients with more bandwidth. ...
    Jul 6, 2011 at 10:09 am
    Jul 7, 2011 at 12:36 pm
  • Hi, I faced a problem that the jobs are still running after executing "hadoop job -kill jobId". I rebooted the cluster but the job still can not be killed. The hadoop version is 0.20.2. Any idea? ...
    Juwei ShiJuwei Shi
    Jul 1, 2011 at 3:53 pm
    Jul 5, 2011 at 5:30 pm
  • All When starting hadoop on OSX I am getting this error. is there a fix for it java[22373:1c03] Unable to load realm info from SCDynamicStore
    Ben CuthbertBen Cuthbert
    Jul 27, 2011 at 7:37 pm
    Feb 22, 2012 at 8:51 pm
  • Hi, We have a job where the map tasks are given the path to an output folder. Each map task writes a single file to that folder. There is no reduce phase. There is another thread, which constantly ...
    Jul 28, 2011 at 8:09 am
    Jul 28, 2011 at 11:21 am
  • Hi Folks, I have a bunch of binary files which I've stored in a sequencefile. The name of the file is the key, the data is the value and I've stored them sorted by key. (I'm not tied to using a ...
    Tom MelendezTom Melendez
    Jul 27, 2011 at 6:29 am
    Jul 27, 2011 at 8:41 pm
  • Is there anyway I can write out the results of my mapreduce job into 1 local file... ie the opposite of getmerge? Thanks
    Jul 5, 2011 at 3:09 pm
    Jul 27, 2011 at 10:57 am
  • Hi, We faced a problem of loading logging class when start the name node. It seems that hadoop can not find commons-logging-*.jar We have tried other commons-logging-1.0.4.jar and ...
    Juwei ShiJuwei Shi
    Jul 20, 2011 at 6:16 am
    Jul 27, 2011 at 10:41 am
  • as described in ...
    Jul 18, 2011 at 7:06 pm
    Jul 25, 2011 at 9:50 am
  • Hi, We released Nectar,first open source predictive modeling on Apache Hadoop. Please check it out. Info page http://zinniasystems.com/zinnia.jsp?lookupPage=blogs/nectar.jsp Git Hub ...
    Madhu phatakMadhu phatak
    Jul 24, 2011 at 6:16 am
    Jul 24, 2011 at 6:23 am
  • hi, all. I am a new hadoop beginner, I try to construct a map and reduce task to run, however encountered an exception while continue going further. Exception: java.io.IOException: Type mismatch in ...
    Teng, JamesTeng, James
    Jul 12, 2011 at 6:46 am
    Jul 12, 2011 at 1:16 pm
  • I have many blocks. Around 50~90m each datanode. They often do not respond while 1~3 min and i think this is because of full scanning for block report. So if i set dfs.blockreport.intervalMsec to ...
    moon soo Leemoon soo Lee
    Jul 8, 2011 at 2:10 am
    Jul 9, 2011 at 1:37 am
  • Hi, I'm new to Hadoop. I'm trying to set up Eclipse for Hadoop debugging. I have: Eclipse 3.5.2, configured to run apps with JRE 1.5. MacOS 10.6.8 Hadoop 0.21.0 I copied ...
    Teruhiko KurosakaTeruhiko Kurosaka
    Jul 7, 2011 at 9:04 am
    Jul 7, 2011 at 11:14 pm
  • Hi guys.. I am previously using hadoop and Hbase... So for Hbase to run perfectly fine we need Hadoop-0.20-append for Hbase jar files.. So I am using Hadoop-0.20-append jar files.. which made both my ...
    Praveenesh kumarPraveenesh kumar
    Jul 2, 2011 at 7:39 am
    Jul 4, 2011 at 7:30 am
  • Hi All, How can I determine if a file is being written to (by any thread) in HDFS. I have a continuous process on the master node, which is tracking a particular folder in HDFS for files to process. ...
    Nitin KhandelwalNitin Khandelwal
    Jul 28, 2011 at 5:52 am
    Jul 29, 2011 at 12:25 am
  • Hello I don't know if the question has been answered. I am trying to understand the overlap between FILE_BYTES_READ and HDFS_BYTES_READ. What are the various components that provide value to this ...
    R VR V
    Jul 28, 2011 at 12:31 am
    Jul 28, 2011 at 9:49 pm
  • Just trying to understand what happens if there are 3 nodes with replication set to 3 and one node fails. Does it fail the writes too? If there is a link that I can look at will be great. I tried ...
    Mohit AnchliaMohit Anchlia
    Jul 27, 2011 at 11:39 pm
    Jul 28, 2011 at 4:19 pm
  • Hi everyone, I am new to it, and want to do some debug/log. I'd like to check what the value is for each mapper execution. If I add the following code in bold, where can I find the log info? If I ...
    Jul 28, 2011 at 6:17 am
    Jul 28, 2011 at 7:05 am
  • Hi, I want to build Hadoop 0.20.2 from source using the Eclipse IDE. Can anyone help me with this? Regards, Vighnesh
    Vighnesh AvadhaniVighnesh Avadhani
    Jul 27, 2011 at 5:32 am
    Jul 27, 2011 at 9:19 pm
  • Good evening, I have built an Rtree on HDFS, in order to improve the query performance of high-selectivity spatial queries. The Rtree is composed of a number of hdfs files (each one created by one ...
    Sofia GeorgiakakiSofia Georgiakaki
    Jul 25, 2011 at 9:41 pm
    Jul 25, 2011 at 11:15 pm
  • Hi, I have basic question on property "fs.default.name" is that I am not able to open NameNode URL when I set fs.default.name=file:/// . If we define not use HDFS as our file system then how hadoop ...
    Mahesh ShindeMahesh Shinde
    Jul 23, 2011 at 10:47 am
    Jul 24, 2011 at 3:31 am
Group Navigation
period‹ prev | Jul 2011 | next ›
Group Overview
groupcommon-user @

136 users for July 2011

Harsh J: 37 posts Madhu phatak: 17 posts Robert Evans: 15 posts Joey Echeverria: 14 posts Michel Segel: 13 posts Steve Loughran: 13 posts Rita: 12 posts Roger Chen: 10 posts Adarsh Sharma: 9 posts Uma Maheswara Rao G 72686: 9 posts Jeff Schmitz: 8 posts Raja Nagendra Kumar: 8 posts Daniel Yehdego: 7 posts Juan P.: 7 posts Tom Melendez: 7 posts Arun Murthy: 6 posts Devaraj K: 6 posts A Df: 5 posts Bharath Mundlapudi: 5 posts Eric Payne: 5 posts
show more