FAQ

Hadoop > common-user > December 2009

Search Discussions

132 discussions - 543 posts

  • Hi all I have installed hadoop with version 0.18.2, and I want to use fuse in hadoop. Following the src/contrib/fuse-dfs/README, I executed "ant compile-contrib -Dlibhdfs=1 -Dfusedfs=1 ", an error ...
    LwmLwm
    Dec 13, 2009 at 2:42 pm
    Dec 22, 2009 at 5:21 pm
  • hadoop@hadoopmaster ~ $ hadoop-0.20.1/bin/hadoop dfsadmin -report report: org.apache.hadoop.security.AccessControlException: Superuser privilege is required
    Pavel kolodinPavel kolodin
    Dec 1, 2009 at 6:21 am
    Dec 3, 2009 at 4:31 pm
  • Hi all , I want to emit <key , <list of values as map out .. One naive solution is to use some "Text" object and a "delimiter" .. concatenate everything into one "Text" object and parse it in the ...
    Bharath vBharath v
    Dec 28, 2009 at 3:59 am
    Jan 7, 2010 at 11:14 pm
  • I am running Hadoop-0.20.1 on a Solaris box with dfs.permissions set to false. There is a working version of whoami on the system. Folders and files created by my program show up with an owner of ...
    Habermaas, WilliamHabermaas, William
    Dec 7, 2009 at 8:30 pm
    Dec 23, 2009 at 7:47 pm
  • Hi, today I tried to run the cloudera debian dist on a 4 machine cluster. I still have some itches, see my list below. Some of them may require a fix in the packaging. Therefor I thought that it may ...
    Thomas KochThomas Koch
    Dec 30, 2009 at 6:54 pm
    Jan 11, 2010 at 2:38 pm
  • Hello- I have a namenode/JT with 3 slaves running cloudera's 0.20 release. Everything was going fine until yesterday when the web interface randomly went down (the http://hostname:50070 and ...
    Mark VigeantMark Vigeant
    Dec 1, 2009 at 3:41 pm
    Dec 15, 2009 at 4:01 am
  • We have a hadoop cluster with a 100TB capacity, and according to the dfs web interface we are using 50% of our capacity (50TB). However doing 'hadoop fs -dus /' says the total size of everything is ...
    Nick BaileyNick Bailey
    Dec 9, 2009 at 8:33 pm
    Dec 10, 2009 at 12:20 am
  • Hello everyone I want to save the value to a LinkedList, but THE ***activeList.get(0)*** have been changed every time. I think it should be the first element, but it's not Thanks Song ==========Part ...
    Oo famcrOo famcr
    Dec 26, 2009 at 10:59 pm
    Jan 3, 2010 at 12:33 am
  • Hello everyone, I am trying to write different output types from mapper. For example, say I have 3 classes A,B and C such that: - C extends A and - B extends A and my mapper is: Mapper<K1, V1, K2, A ...
    Mehul SutariyaMehul Sutariya
    Dec 10, 2009 at 9:07 am
    Dec 22, 2009 at 2:44 pm
  • The *org.apache.hadoop.mapred.JobConf* class is deprecated in version 0.20.1. I tried to use the *org.apache.hadoop.mapreduce.Job* class instead but found no way to set the jar file. Can anyone tell ...
    Farhan HusainFarhan Husain
    Dec 17, 2009 at 10:44 pm
    Dec 18, 2009 at 5:42 pm
  • Hi, I am using hadoop-0.20.1 to run terasort and randsort benchmarking tests on a small 8-node linux cluster. Most runs consist of usually low (<50%) core utilizations in the map and reduce phase, as ...
    Vasilis LiaskovitisVasilis Liaskovitis
    Dec 2, 2009 at 8:22 pm
    Dec 9, 2009 at 7:16 pm
  • Does anybody have any recommmendations on a CF management app? The ones that I am looking at are Puppet, CFengine, and BCFG2. Thanks for the recommendation in advance. -John
    John MartyniakJohn Martyniak
    Dec 9, 2009 at 6:59 pm
    Dec 10, 2009 at 5:33 pm
  • Hello All, I am creating 2 files in the constructor of my reducer and I store the file Handles as member variables. I write some data in these files for each call to reduce method. For some reason ...
    Parth J. BrahmbhattParth J. Brahmbhatt
    Dec 5, 2009 at 1:11 am
    Dec 7, 2009 at 6:47 pm
  • Hello. I am using hadoop-0.20.1 on two VPS nodes with gentoo linux (hardware = 16 xeon cpu, 64bit). Previously i was using the same version on 2 separated 32-bit machines and all was fine. Seems to ...
    Pavel kolodinPavel kolodin
    Dec 7, 2009 at 4:17 am
    Dec 7, 2009 at 5:16 am
  • Dear all, Can anybody tell me how concurrent write happens in HDFS ? Thanks and Best Regards, Krishna Kumar Senior Storage Engineer Why do we have to die? If we had to die, and everything is gone ...
    Krishna KumarKrishna Kumar
    Dec 1, 2009 at 1:57 pm
    Dec 2, 2009 at 12:49 am
  • Hello. I am running hadoop-0.20.1. conf/hadoop-env.sh have default setting, excluding one "JAVA_HOME". ---------------------- Success with 2 such nodes: 1) laptop, pentium M760, 2GB RAM 2) VirtualBox ...
    M77771111M77771111
    Dec 13, 2009 at 5:48 am
    Dec 14, 2009 at 2:09 am
  • Hi, i want to write a file to hdfs, using hadoop pipes. can anyone tell me how to do that? im using an external library that writes its output to disk, so probably i have to read that output and ...
    HorsonHorson
    Dec 7, 2009 at 6:06 pm
    Dec 9, 2009 at 1:54 am
  • Running jobs sometimes displays progress as follows: map 100% reduce 10% and then map 100% reduce 0% What causes this? Is this speculative execution restarting tasks? Cheers, DS
    Doopah ShafDoopah Shaf
    Dec 27, 2009 at 10:20 am
    Jan 5, 2010 at 1:37 am
  • Hey Everyone- I've been playing around with Hadoop and Hbase for a while and I noticed that when running a program to upload data into an HTable I saw the output: INFO mapred.MapTask: io.sort.mb = ...
    Mark VigeantMark Vigeant
    Dec 22, 2009 at 4:10 pm
    Dec 23, 2009 at 5:31 pm
  • Hi, I am currently using hadoop 0.19.2 to run large data processing. But I noticed when the job is launched, there are only two map/reduce tasks running in the very beginning. after one heartbeat ...
    Starry SHIStarry SHI
    Dec 21, 2009 at 4:10 am
    Dec 22, 2009 at 2:41 pm
  • Hi everyone, I run multiple map/reduce jobs which are chained together. The output of one map/reduce is the input of another. There are also some integer valued variables which are outputted from one ...
    HimanshuHimanshu
    Dec 23, 2009 at 7:51 am
    Dec 26, 2009 at 10:12 am
  • Hi, I have spent several days on the customized file input format in hadoop. Basically, we need split one giant square shaped image (.tif) into four square shaped smaller images. Where does the ...
    Cao KangCao Kang
    Dec 20, 2009 at 8:52 pm
    Dec 23, 2009 at 3:12 am
  • Hi. My Hadoop cluster (0.20.1) has multiple users. When I use different user's account to create a file in HDFS, I find that no matter what group the user belonging to in Linux, the files in HDFS ...
    Starry SHIStarry SHI
    Dec 17, 2009 at 12:12 pm
    Dec 18, 2009 at 6:14 am
  • I have a question that I got an interesting and helpful answer for on the IRC channel today, but thought I'd open it up to a larger group as well. My problem is hopefully a very common problem. I'm ...
    Andy SautinsAndy Sautins
    Dec 11, 2009 at 12:56 am
    Dec 14, 2009 at 11:21 pm
  • First of all, I'd like to say hi to all people on the list. I ran across Hadoop and Cloudera projects recently, and I was immediately intrigued with it, because I'm in the middle of writing a project ...
    Maciej TrebaczMaciej Trebacz
    Dec 2, 2009 at 9:51 pm
    Dec 2, 2009 at 11:04 pm
  • I have followed http://www.cloudera.com/blog/2009/11/17/hadoop-at-twitter-part-1-splittable-lzo-compression/and http://code.google.com/p/hadoop-gpl-compression/wiki/FAQ to build the requisite ...
    Steve KuoSteve Kuo
    Dec 31, 2009 at 5:54 pm
    Jan 23, 2010 at 7:08 pm
  • Hi, what happens when I submit a few jobs on the cluster? To me, it seems like they all are running - which I know can't be, because I only have 2 slaves. Where do I read about this? I am using ...
    Mark KerznerMark Kerzner
    Dec 29, 2009 at 5:16 pm
    Dec 29, 2009 at 5:59 pm
  • Does anyone have any recommendations for / against using a NAS / SAN system as the underlying physical storage for a hadoop cluster, instead of local data node DAS?
    Doopah ShafDoopah Shaf
    Dec 22, 2009 at 4:25 pm
    Dec 23, 2009 at 8:18 pm
  • Hi, we just encountered some problems when restarting our namenode. I'd really appreciate if anyone has any clue of what is going on here. The error message is as follows: 09/12/16 14:25:03 INFO ...
    Erik BernhardssonErik Bernhardsson
    Dec 16, 2009 at 2:55 pm
    Dec 18, 2009 at 2:47 pm
  • Hi, First, I would like to apologise if this question has been asked before (I am quite sure it has been) and I would appreciate very much if someone replies with a link to the answer. My question is ...
    Eguzki Astiz LezaunEguzki Astiz Lezaun
    Dec 16, 2009 at 1:35 pm
    Dec 16, 2009 at 5:46 pm
  • Tanx a lot for the suggestion. But I am pretty new to this. Can you tell me where to use this. as to do we need to conifugure this in conf? or when we submit as hadoop jar <jar_name <job <in <out ...
    Anjali nairAnjali nair
    Dec 16, 2009 at 5:21 am
    Dec 16, 2009 at 8:37 am
  • Hi, I've been trying to code a pretty simple InputFormat. The idea is this: I have an array of numbers (say, the range [0-5000]) and I want each mapper to receive a split of size 500 i.e. 500 ...
    Antonio D'EttoleAntonio D'Ettole
    Dec 11, 2009 at 7:03 pm
    Dec 15, 2009 at 9:40 am
  • Hello. I am running 0.20.1. There is an options HADOOP_*_OPTS in file hadoop-env.sh. When i don't put "-Xmx" in them at all, java can't initialize any VM at all. But this happens only on 64bit VPS ...
    Pavel kolodinPavel kolodin
    Dec 11, 2009 at 11:09 am
    Dec 15, 2009 at 9:31 am
  • Dear All, Can anybody please let me know about some of the current features of hadoop on which development work is going on / or planning to go in future, like : 1. Record append 2. Snapshot 3. ...
    Krishna KumarKrishna Kumar
    Dec 8, 2009 at 6:56 am
    Dec 8, 2009 at 6:18 pm
  • Hi, Can anybody please give an example of a streaming mapper/reducer written in C++? I don't seem to find even a single example on web. Thanks. Upendra
    Upendra DadiUpendra Dadi
    Dec 4, 2009 at 11:10 pm
    Dec 4, 2009 at 11:41 pm
  • Hi, I'm trying to run the map/reduce tutorials with windows XP and cygwin. In the 2nd step: javac -classpath ${HADOOP_HOME}/hadoop-${HADOOP_VERSION}-core.jar -d wordcount_classes WordCount.java , I ...
    Mikko LahtiMikko Lahti
    Dec 2, 2009 at 7:02 pm
    Dec 3, 2009 at 5:19 pm
  • Hello, I am using hadoop-0.20.1. I need to know the input file name in my map processes and pass an integer and a string to my reduce processes. I used the following method calls for that: ...
    Farhan HusainFarhan Husain
    Dec 30, 2009 at 5:22 pm
    Jan 5, 2010 at 8:20 pm
  • Hi , I've implemented a simple VectorWritable class as follows package com; import org.apache.hadoop.*; import org.apache.hadoop.io.*; import java.io.*; import java.util.Vector; public class ...
    Bharath vBharath v
    Dec 29, 2009 at 5:30 am
    Dec 29, 2009 at 5:25 pm
  • Hello everyone I want to save the value to a LinkedList, but THE ***activeList.get(0)*** have been changed every time. I think it should be the first element, but it's not. Thanks Song ==========Part ...
    Oo famcrOo famcr
    Dec 23, 2009 at 2:30 pm
    Dec 23, 2009 at 3:11 pm
  • Trying to figure out how hadoop actually achieves its speed. Assuming that data locality is central to the efficiency of hadoop, how does the magic actually happen, given that data still gets moved ...
    Doopah ShafDoopah Shaf
    Dec 21, 2009 at 7:14 am
    Dec 22, 2009 at 6:21 pm
  • When I run the balancer, it says it's moving 10G, but after those 10G are moved, it's reported that only ~ 1G is moved. What could be causing this discrepancy? Thanks so much in advance for your ...
    Jacob OlsenJacob Olsen
    Dec 21, 2009 at 10:19 pm
    Dec 22, 2009 at 4:35 am
  • Hi All, I am interested to know that can we use hadoop for applications where they need more control over the data and it can specify which node will do which part of the processing or the storage. ...
    Ahmad Ali IqbalAhmad Ali Iqbal
    Dec 16, 2009 at 11:44 pm
    Dec 21, 2009 at 2:48 am
  • Quick question: is it possible to configure a job such that its map tasks will only run node-local? We have some mappers that need to access data that is only available on the nodes themselves and ...
    Ives AertsIves Aerts
    Dec 17, 2009 at 6:23 pm
    Dec 19, 2009 at 3:33 am
  • Hi all, When I invoke the balancer command, I found that it seems hdfs want to balance the block number of each data node rather than the percentage of each machine's remaining capacity. Not sure if ...
    Jeff ZhangJeff Zhang
    Dec 18, 2009 at 10:30 am
    Dec 18, 2009 at 5:04 pm
  • Hi, Is there any example how a sequence file can be read and split in hadoop? Many thanks! Cao
    Cao KangCao Kang
    Dec 18, 2009 at 4:21 pm
    Dec 18, 2009 at 4:54 pm
  • Hi, We are using hadoop installation over 6 nodes, that looks like: 172.19.0.10 (master) 172.19.0.11 172.19.0.12 172.19.0.13 172.19.0.14 172.19.0.15 Each node has (4x1.5 TB) hard disk (hdparm -t ...
    Dmitriy LyfarDmitriy Lyfar
    Dec 9, 2009 at 1:52 pm
    Dec 17, 2009 at 12:07 am
  • Hi, The counters are updated as the records are *consumed*, for both mapper and reducer. Can you confirm if all the values returned by your iterators are consumed on reduce side? Also, are you having ...
    Amogh VasekarAmogh Vasekar
    Dec 11, 2009 at 7:56 am
    Dec 16, 2009 at 8:43 pm
  • I have configured capacity scheduler and set up queues other than the default queue. However I do not understand where to change a job from the default queue to a user defined queue. Now whenever I ...
    Anjali nairAnjali nair
    Dec 15, 2009 at 5:50 am
    Dec 16, 2009 at 7:21 am
  • Hi, guys, first of all, I have added this section to hadoop-site.xml <property <name mapred.child.java.opts</name <value -Xmx1024m</value </property Secondly, I am running on the EC2 Hadoop clusters ...
    Mark KerznerMark Kerzner
    Dec 8, 2009 at 12:27 am
    Dec 8, 2009 at 7:25 am
  • Hi, Just one week since upgrading to 0.20.1, I've been hit twice by NN crashes. The symptoms were the same. The NN log says: 2009-12-01 12:04:00,420 INFO ...
    Zhang, ZhangZhang, Zhang
    Dec 1, 2009 at 11:53 pm
    Dec 5, 2009 at 5:13 am
Dec 2009
123
Group Navigation
period‹ prev | Dec 2009 | next ›
Group Overview
groupcommon-user @
categorieshadoop
discussions132
posts543
users159
websitehadoop.apache.org...
irc#hadoop

159 users for December 2009

Todd Lipcon: 40 posts Pavel kolodin: 24 posts Jeff Zhang: 19 posts Brian Bockelman: 18 posts Allen Wittenauer: 14 posts Edward Capriolo: 14 posts Gang Luo: 14 posts 鲁伟明: 14 posts Jason Venner: 13 posts Mark Kerzner: 13 posts Eli Collins: 12 posts Owen O'Malley: 11 posts Starry SHI: 10 posts Farhan Husain: 9 posts Mark Vigeant: 9 posts Edmund Kohlwey: 8 posts Mike Kendall: 8 posts Amogh Vasekar: 7 posts Anjali nair: 7 posts Nick Bailey: 7 posts
show more
Archives