FAQ

Search Discussions

134 discussions - 558 posts

  • hi,recently,i got some problem. at first,I start the hadoop #bin/start-all.sh then,I mount hdfs to local #fuse_dfs_wrapper.sh dfs://cent52ip32:9000/ /dfs-test/ In /dfs-test,I do some work,like ...
    Yibo820217Yibo820217
    Oct 13, 2009 at 8:01 am
    Oct 16, 2009 at 1:31 pm
  • Hey Folks, I am seeing a very weird problem in FileSystem.get(Configuration). I want to get a FileSystem given the configuration, so I am using Configuration conf = new Configuration(); _fs = ...
    Bhupesh BansalBhupesh Bansal
    Oct 15, 2009 at 8:35 pm
    Oct 15, 2009 at 10:51 pm
  • Quick question for the hadoop / linux masters out there: I recently observed a stalled tasktracker daemon on our production cluster, and was wondering if there were common tests to detect failures so ...
    James warrenJames warren
    Oct 8, 2009 at 6:20 am
    Oct 15, 2009 at 11:10 am
  • Hi, I run Elastic MapReduce. The output of my application is a text file, where each line is essentially a set of fields. It will fit very nicely into a simple database, but which database 1. Is ...
    Mark KerznerMark Kerzner
    Oct 13, 2009 at 10:13 pm
    Oct 13, 2009 at 10:44 pm
  • Can map function be called recursively? -- View this message in context: http://www.nabble.com/map-function-tp25859056p25859056.html Sent from the Hadoop lucene-users mailing list archive at ...
    HellpizzaHellpizza
    Oct 12, 2009 at 4:41 pm
    Oct 13, 2009 at 5:24 am
  • Hi, I need to get the position of the key being processed in a mapper task. My inputFile is a sequence file .... I tried the Context, but the best i could get was the inputsplit position and the file ...
    Ishwar ramaniIshwar ramani
    Oct 8, 2009 at 11:24 pm
    Oct 12, 2009 at 5:23 pm
  • I need a cache, that is read by many nodes often, written by a few nodes rarely. Its not too big in size (200.000-2Mio records/1Gb), but may be too big to fit into one node (so keeping local caches ...
    Bob SchulzeBob Schulze
    Oct 7, 2009 at 9:58 am
    Oct 7, 2009 at 4:58 pm
  • I am using Amazon EC2 with our HDFS on EBS volumes. While running a job today, our EBS volumes apparently died out of nowhere. You can see the logfile is even cut off: 2009-10-05 13:37:00,321 INFO ...
    Malcolm MatalkaMalcolm Matalka
    Oct 5, 2009 at 6:42 pm
    Oct 5, 2009 at 8:52 pm
  • Hi, that is what I observe, that in the resulting text file the two are separated by a tab. Can I change it, is it in any way configurable? Thank you, Mark
    Mark KerznerMark Kerzner
    Oct 2, 2009 at 1:56 am
    Oct 2, 2009 at 3:29 am
  • Hi. I'm looking to spread the meta-data writing across several disks, including NFS, to provide greater survivability. What make sense more - to write NameNode meta-data to NFS, or to write the ...
    Stas OskinStas Oskin
    Oct 1, 2009 at 1:16 pm
    Oct 2, 2009 at 1:06 am
  • Hi all, I have installed hadoop 0.18.3 on my own cluster with 5 machines, now I want to install hadoop 0.20, but I do not run to uninstall the hadoop 0.18.3. So what things should I modify to ...
    Jeff ZhangJeff Zhang
    Oct 30, 2009 at 5:44 am
    Oct 30, 2009 at 6:02 am
  • Hi, Using Hadoop 0.20 (CDH2) I'm trying to pass some JVM options to my child tasks on the command-line, like this: $ hadoop jar streaming.jar -D mapred.reduce.tasks=0 -D ...
    Brian VargasBrian Vargas
    Oct 28, 2009 at 7:11 pm
    Oct 28, 2009 at 8:00 pm
  • I was testing a job on a single node hadoop cluster running Hadoo9 0.19. The single tasktracker has 2 reduce slots. After finishing 8 reduce tasks out of 17 total reduce tasks, the tasktracker ...
    Runping QiRunping Qi
    Oct 26, 2009 at 5:29 am
    Oct 26, 2009 at 11:04 pm
  • Dear All, I am implementing a clustering algorithm in which I need to compare each line to two specific lines (they all have the same format ) and output two scores denoting the similarity between ...
    Boyu ZhangBoyu Zhang
    Oct 26, 2009 at 12:47 am
    Oct 26, 2009 at 7:40 pm
  • hi here I choose a machine as a namenode,and a machine as a secondary namenode, a machine as a datanode. when i start up hadoop(bin/start-all.sh), there are some errors in secondary namenode,like ...
    Yibo820217Yibo820217
    Oct 21, 2009 at 9:09 am
    Oct 23, 2009 at 8:57 am
  • Hi all, These days, I begin look into source code hadoop. And I want to know whether I need some distributed computing algorithm if I want to deep into source code of hadoop ? Thank you. Jeff zhang
    Jeff ZhangJeff Zhang
    Oct 21, 2009 at 3:17 pm
    Oct 22, 2009 at 3:39 am
  • hi, I use hadoop0.20 and 8 nodes, there is a job that has 130 map to run ,and completed 128 map, but only 2 map fail ,and its fail in my case is accepted ,but the job fail ,the last 128 map also ...
    梁景明梁景明
    Oct 21, 2009 at 3:28 am
    Oct 22, 2009 at 2:07 am
  • Hi! Well, I have a kinda simple question, but I can not spot a proper doc for it: how you, guys, restricting access to the web interfaces? :-) It is somewhere in jetty or there is no feature like ...
    Bogdan M. MaryniukBogdan M. Maryniuk
    Oct 18, 2009 at 3:50 pm
    Oct 21, 2009 at 1:32 am
  • NOTE: for amd64 architecture, libhdfs will not compile unless you edit the Makefile in src/c++/libhdfs/Makefile and set OS_ARCH=amd64 (probably the same for others too). See ...
    杨杰杨杰
    Oct 19, 2009 at 10:07 pm
    Oct 20, 2009 at 2:40 am
  • Hi Everybody, I'm doing a project where I have to read a large set of compress files (gz). I'm using python and streaming to achieve my goals. However, I have a problem, there are corrupt compress ...
    Xavier QuintunaXavier Quintuna
    Oct 19, 2009 at 5:58 pm
    Oct 19, 2009 at 6:53 pm
  • Overview The SQL Data Migration Specialist plays a crucial role in converting new Client's data onto Brilig's service platforms. We are looking for a talented and energetic full-time freelance ...
    AlevinAlevin
    Oct 15, 2009 at 7:25 pm
    Oct 15, 2009 at 7:59 pm
  • I have already installed hadoop correctly. but what does that mean? ~/hive/build/dist/bin$ ./hive Hive history file=/tmp/yangzhuoluo/hive_job_log_yangzhuoluo_200910151919_995767046.txt hive show ...
    Clark Yang (杨卓荦)Clark Yang (杨卓荦)
    Oct 15, 2009 at 11:38 am
    Oct 15, 2009 at 5:02 pm
  • hi all,there is my problem. when add a datanode to hadoop,the way is; 1.in namenode add the new datanode to conf/slave 2.in new datanode cd $HADOOP_HOME then $ bin/hadoop-daemon.sh start datanode $ ...
    Yibo820217Yibo820217
    Oct 14, 2009 at 6:14 am
    Oct 14, 2009 at 12:50 pm
  • Greetings, I would like to let everyone know that we will be having our 4th Hadoop Users Group DC Meetup Friday, October 16, 2009 at 6:30 PM. This meetup will have a couple talks about some new ...
    Lalit KapoorLalit Kapoor
    Oct 8, 2009 at 7:17 pm
    Oct 13, 2009 at 10:12 pm
  • One of my datanode is stop Can I start the datanode and add it to the cluster without restarting the hole cluster?
    Eason.LeeEason.Lee
    Oct 12, 2009 at 9:44 am
    Oct 13, 2009 at 12:52 am
  • Hi, I'm running Hadoop 0.20.1 and Hive (checked out revision 824063). Direct MapReduce task succeeds, but Map task created by Hive fails: hive select * from pokes where foo 100; Total MapReduce jobs ...
    Touretsky, GregoryTouretsky, Gregory
    Oct 11, 2009 at 2:41 pm
    Oct 12, 2009 at 6:04 pm
  • Hi, I get the following error when trying to mount the fuse dfs, the first problem is: [root@puppet ~]# fuse_dfs_wrapper.sh dfs://100.207.100.25:9000/ /dfs port=9000,server=100.207.100.25 fuse-dfs ...
    Yibo820217Yibo820217
    Oct 9, 2009 at 6:35 am
    Oct 9, 2009 at 12:55 pm
  • I gave a presentation on Hadoop last night to a technical group (http://lambdalounge.org/) here in St. Louis. I pointed out that the name node is currently a single point of failure and it's one ...
    Tom WheelerTom Wheeler
    Oct 2, 2009 at 5:54 pm
    Oct 3, 2009 at 7:20 pm
  • Hi all. I'm struggling a bit to figure this out and wondering if anyone had any pointers. I'm using SequenceFiles as output from a MapReduce job ( using SequenceFileOutputFormat ) and then in a ...
    Andy SautinsAndy Sautins
    Oct 1, 2009 at 4:11 pm
    Oct 2, 2009 at 8:49 pm
  • Hi, We have a 200 node hadoop cluster (0.20.0) and have tweaked namenode and datanode handler to 40 and 10. The xcievers also had to be changed to 8192. But during the mapred jobs, we are seeing lot ...
    Murali Krishna. PMurali Krishna. P
    Oct 2, 2009 at 3:28 am
    Oct 2, 2009 at 3:33 pm
  • Hi, I use the org.apache.hadoop.io.Text object to set its value "測試" in chinese text(six bytes in UTF-8 encoding), and when I invoke its "getBytes()" method that return the raw bytes (11 bytes), but ...
    ChingShenChingShen
    Oct 31, 2009 at 3:03 pm
    Oct 31, 2009 at 5:25 pm
  • Hi all, I found the the default value of HADOOP_VERSION is 0.17.0 in hadoop-ec2-env.sh of hadoop 0.18.3, and I can create hadoop 0.17.0 cluster in ec2 succesully, but I can not create hadoop 0.18.3 ...
    Jeff ZhangJeff Zhang
    Oct 30, 2009 at 1:07 pm
    Oct 30, 2009 at 2:31 pm
  • Hi Everyone, I'm experimenting with Hadoop and trying to get it running in pseudo-distributed mode as described in Appendix A of Tom White's book Hadoop The Definitive Guide. I've got the ...
    David GreerDavid Greer
    Oct 28, 2009 at 10:46 pm
    Oct 29, 2009 at 8:50 pm
  • I'm running Hadoop 0.20.1+133 (Cloudera distro) I tried setting up a multi-node Hadoop cluster and on executing the command: hadoop jar /usr/lib/hadoop/hadoop-0.20.1+133-examples.jar grep input ...
    Hassaan KhanHassaan Khan
    Oct 28, 2009 at 3:42 pm
    Oct 28, 2009 at 3:48 pm
  • Hi All, We are facing the issue with distribution of data in a cluster where nodes have differnt storage capacity. We have 4 nodes with 100G capacity and 1 node with 2TB capacity. The storage of the ...
    Vibhooti VermaVibhooti Verma
    Oct 28, 2009 at 9:25 am
    Oct 28, 2009 at 9:42 am
  • Greetings, (You're receiving this e-mail because you're on a DL or I think you'd be interested) It's time for another Hadoop/Lucene/Apache "Cloud" stack meetup! This month it'll be on Wednesday, the ...
    Bradford StephensBradford Stephens
    Oct 19, 2009 at 12:11 am
    Oct 27, 2009 at 11:08 pm
  • Hi, I am new to Hadoop. I am following the tutorial on http://hadoop.apache.org/common/docs/current/quickstart.html I have downloaded the hadoop-0.20.1.tar.gz package and unpackaged it. First, I ...
    Dong ZhangDong Zhang
    Oct 26, 2009 at 5:48 am
    Oct 26, 2009 at 6:09 am
  • Hi all, I'd like to contribute the hadoop, and I'd like to get started with fixing bugs. But I found in the jira, it says that I have no permission to work on the jira item. So how can I get the ...
    Jeff ZhangJeff Zhang
    Oct 24, 2009 at 11:37 am
    Oct 24, 2009 at 11:44 am
  • Hello! We have a cluster of 5 nodes and we are concentrating on the development of a DFS(Distributed File System). with the incorporation of Hadoop. Now, Can I get some ideas on how can I design ...
    Sugandha NaolekarSugandha Naolekar
    Oct 20, 2009 at 11:39 am
    Oct 22, 2009 at 3:56 am
  • I am trying to stream data from HDFS on a workstation outside of hadoop. I have a small method to initialize the DistributedFileSystem and i pass the IP and port of the namenode, but that fails with ...
    Stephane BrossierStephane Brossier
    Oct 21, 2009 at 12:56 am
    Oct 21, 2009 at 5:03 am
  • Hi, What is the preferred method to distribute the classes (in various Jars) to my Hadoop instances, that are required by my Mapper class? thanks!
    Yz5od2Yz5od2
    Oct 18, 2009 at 10:08 pm
    Oct 19, 2009 at 12:53 pm
  • Hello All, I was wondering if our map reduce code can just return the location of the file? Or place the actual file in a given output directory by searching based on a keyword. Let me make myself ...
    ShwitzuShwitzu
    Oct 15, 2009 at 10:23 pm
    Oct 15, 2009 at 10:31 pm
  • Hi, Is it safe to run the jobs during the datanode decommissioning? Around 10% of the total datanodes are being decommissioned and i want to run a job mean while. Wanted to confirm whether it is safe ...
    Murali Krishna. PMurali Krishna. P
    Oct 15, 2009 at 4:02 pm
    Oct 15, 2009 at 5:23 pm
  • Hi all, I'm very new to Hadoop and I am using the Pro Hadoop book to learn. I'm doing a project for class, where we are required to learn and use Hadoop. We are supposed to use Hadoop for converting ...
    DavydanyDavydany
    Oct 12, 2009 at 3:06 pm
    Oct 12, 2009 at 3:40 pm
  • Hi all, I am fairly new to Hadoop, so please bear with me if you think my question is too straight forward. I am to computes the average of the values in a file F. F contains a large number of ...
    Congchong liuCongchong liu
    Oct 11, 2009 at 9:23 pm
    Oct 11, 2009 at 9:40 pm
  • Hello Hadoop Users, Now that we have our cluster up and running ( mostly ) the question has come up about much time will be required to run the system. We have Hadoop running on 64 nodes, 40TB of ...
    Nick RathkeNick Rathke
    Oct 9, 2009 at 3:51 pm
    Oct 9, 2009 at 5:46 pm
  • FYI---the University of Maryland is seeking an assistant professor in cloud computing. See job description below. ================= College of Information Studies, Maryland's iSchool University of ...
    Jimmy LinJimmy Lin
    Oct 8, 2009 at 9:23 pm
    Oct 9, 2009 at 4:16 am
  • I don't know if everyone saw this JIRA come across the wire: https://issues.apache.org/jira/browse/HDFS-677 Looks like if both quotas are full, you lose data. Time to disable those quotas.. :(
    Allen WittenauerAllen Wittenauer
    Oct 7, 2009 at 4:38 pm
    Oct 7, 2009 at 8:03 pm
  • Hi, I'm running Hadoop 0.19.1 on 19 nodes. I've been benchmarking a Hadoop workload with 115 Map tasks, on two different distributed filesystems (KFS and PVFS); in some tests, I also have a ...
    Esteban Molina-EstolanoEsteban Molina-Estolano
    Oct 3, 2009 at 8:31 am
    Oct 6, 2009 at 10:22 pm
  • Hey- I'm trying to update a custom recordreader written for 0.18.3 and was wondering if either A) Anyone has any example code for extending RecordReader in 0.20.1 (in the mapreduce package, not the ...
    Mark VigeantMark Vigeant
    Oct 6, 2009 at 9:26 pm
    Oct 6, 2009 at 9:38 pm
Group Navigation
period‹ prev | Oct 2009 | next ›
Group Overview
groupcommon-user @
categorieshadoop
discussions134
posts558
users177
websitehadoop.apache.org...
irc#hadoop

177 users for October 2009

Amandeep Khurana: 22 posts Stas Oskin: 22 posts Jason Venner: 20 posts Tim robertson: 20 posts Todd Lipcon: 19 posts Edward Capriolo: 15 posts Aaron Kimball: 14 posts Amogh Vasekar: 14 posts Mark Kerzner: 13 posts Steve Loughran: 13 posts Brian Bockelman: 12 posts Bogdan M. Maryniuk: 10 posts Sudha sadhasivam: 10 posts Allen Wittenauer: 9 posts Huy Phan: 9 posts Jeff Zhang: 9 posts Eason.Lee: 8 posts Shwitzu: 7 posts Usman Waheed: 7 posts Yibo820217: 7 posts
show more