Search Discussions

195 discussions - 812 posts

  • Best-practices-type question: when a single cluster is being used by a team of folks to run jobs, how do people on this list handle user accounts? Many of the examples seem to show everything being ...
    Dan MilsteinDan Milstein
    May 5, 2009 at 2:45 pm
    May 8, 2009 at 2:36 pm
  • Hi, I am being confused by the protocol between mapper and reducer. When mapper emitting the (key,value) pair done, is there any signal the mapper send out to hadoop framework in protocol to indicate ...
    Jianmin WooJianmin Woo
    May 30, 2009 at 4:30 am
    Jun 4, 2009 at 7:22 am
  • Hi all, This is Grace. I am replacing Sun JVM with Jrockit JVM for Hadoop. Also I keep all the same Java options and configuration as Sun JVM. However it is very strange that the performance using ...
    May 6, 2009 at 9:03 am
    May 26, 2009 at 2:36 pm
  • Hi. I'm testing Hadoop in our lab, and started getting the following message when trying to copy a file: Could only be replicated to 0 nodes, instead of 1 I have the following setup: * 3 machines, 2 ...
    Stas OskinStas Oskin
    May 21, 2009 at 9:11 am
    Oct 13, 2009 at 1:38 pm
  • Hi. After rebooting the NameNode server, I found out the NameNode doesn't start anymore. The logs contained this error: "FSNamesystem initialization failed" I suspected filesystem corruption, so I ...
    Stas OskinStas Oskin
    May 4, 2009 at 12:51 pm
    May 10, 2009 at 8:11 am
  • Hi, I want to backup a table and then create a new empty one with following commands in Hadoop. How do I do it in java? Thanks. BEGIN; RENAME TABLE my_table TO backup_table; CREATE TABLE my_table ...
    May 20, 2009 at 6:48 am
    May 20, 2009 at 6:14 pm
  • I ran a job. In the jobtracker web interface, I found 4 maps and 1 reduce running. This is not what I set in my configuration files (hadoop-site.xml). My configuration file is set as follows: ...
    Foss UserFoss User
    May 19, 2009 at 11:50 am
    May 21, 2009 at 7:07 am
  • Hi, How robust is using hadoop with python over the streaming protocol? Any disadvantages (performance? flexibility?) ? It just strikes me that python is so much more convenient when it comes to ...
    S dS d
    May 19, 2009 at 3:37 pm
    May 21, 2009 at 5:23 pm
  • Hi. Any idea if RandomAccessFile is going to be supported in HDFS? Regards.
    Stas OskinStas Oskin
    May 24, 2009 at 12:33 pm
    Sep 23, 2009 at 5:30 pm
  • Hi, I wanna set up a cluster of 5 nodes in such a way that node1 - master node2 - secondary namenode node3 - slave node4 - slave node5 - slave How do we go about that? there is no property in ...
    Rakhi KhatwaniRakhi Khatwani
    May 14, 2009 at 1:04 pm
    May 27, 2009 at 6:22 pm
  • Hi, I am trying to sort some data with hadoop(streaming mode). The input looks like: $ cat small_numbers.txt 9971681 9686036 2592322 4518219 1467363 To send my job to the cluster I use: hadoop jar ...
    David RioDavid Rio
    May 17, 2009 at 4:11 am
    May 19, 2009 at 4:01 am
  • hi there, working through a concept at the moment and was attempting to write lots of data to few files as opposed to writing lots of data to lots of little files. what are the thoughts on this? When ...
    Sasha DolgySasha Dolgy
    May 5, 2009 at 11:34 pm
    May 12, 2009 at 4:32 pm
  • Hello Everyone, Actually I had a cluster which was up. But i stopped the cluster as i wanted to format it.But cant start it back. 1)when i give "start-dfs.sh" I get following on screen starting ...
    Pankil DoshiPankil Doshi
    May 15, 2009 at 1:44 am
    Jun 16, 2009 at 5:21 pm
  • Sorry for cross-posting, I realized I sent the following to the hbase list when it's really more a Hadoop question. ---------- Forwarded message ---------- From: Patrick Angeles ...
    Patrick AngelesPatrick Angeles
    May 27, 2009 at 2:43 pm
    May 29, 2009 at 10:56 am
  • Hello everyone, I got hint how to solve the problem where clusters have different usernames.but now other problem I face is that i can ssh a machine by using -i path/to key/ ..I cant ssh them ...
    Pankil DoshiPankil Doshi
    May 21, 2009 at 7:07 pm
    May 26, 2009 at 6:24 pm
  • Hi all, Now, if we have a large dataset to process by MapReduce. The MapReduce will take machine resources as many as possible. So when one such a big MapReduce job are running, the cluster would ...
    May 11, 2009 at 12:50 pm
    May 14, 2009 at 11:21 am
  • Hello everyone, Till now I was using same username on all my hadoop cluster machines. But now I am building my new cluster and face a situation in which I have different usernames for different ...
    Pankil DoshiPankil Doshi
    May 20, 2009 at 11:08 pm
    May 26, 2009 at 11:19 pm
  • Hi all, For the database import tool I'm writing (Sqoop; HADOOP-5815), in addition to uploading data into HDFS and using MapReduce to load/transform the data, I'd like to integrate more closely with ...
    Aaron KimballAaron Kimball
    May 15, 2009 at 9:05 pm
    May 20, 2009 at 5:44 pm
  • HI , I am trying to step up a hadoop cluster on 512 MB machine and using hadoop 0.18 and have followed procedure given in apache hadoop site for hadoop cluster. I included in conf/slaves two datanode ...
    Ashish pareekAshish pareek
    May 28, 2009 at 5:03 pm
    Jun 17, 2009 at 5:09 am
  • Hello, I'm trying hadoop for the first time and I'm just trying to create a file and append some text in it with the following code: import java.io.IOException; import org.apache.hadoop.conf. ...
    Olivier SmadjaOlivier Smadja
    May 28, 2009 at 1:47 pm
    May 28, 2009 at 7:30 pm
  • Hi. I have an issue where my application, when shutting down (at ShutdownHook level), is unable to copy files to HDFS. Each copy throws the following exception: java.lang.IllegalStateException: ...
    Stas OskinStas Oskin
    May 17, 2009 at 12:45 pm
    May 21, 2009 at 8:18 am
  • We are currently rebuilding our cluster - has anybody recommendations on the underlaying file system? Just standard Ext3? I could imagine that the block size could be larger than its default... Thx ...
    Bob SchulzeBob Schulze
    May 18, 2009 at 3:59 pm
    May 20, 2009 at 1:07 pm
  • Hi, Can someone tell about Append functionality in Hadoop. Is it available now in 0.20 ?? Regards, Wasim
    Wasim BariWasim Bari
    May 14, 2009 at 12:27 pm
    May 18, 2009 at 5:47 am
  • Hi: In my application, there are many small files. But the hadoop is designed to deal with many large files. I want to know why hadoop doesn’t support small files very well and where is the ...
    May 7, 2009 at 2:34 am
    May 13, 2009 at 5:32 am
  • Dear users, I got "ClassNotFoundException" when run the WordCount example on hadoop using Eclipse. Does anyone know where is the problem? Thank you! George
    George PangGeorge Pang
    May 8, 2009 at 7:40 am
    May 9, 2009 at 7:41 am
  • Hi, How do I convert DataInput to array of String? How do I convert ResultSet to array of String? Thanks. Following is the code: static class Record implements Writable, DBWritable { String [] ...
    May 28, 2009 at 9:48 pm
    Jun 4, 2009 at 8:16 am
  • I need to process a dataset that contains text records of fixed length in bytes. For example, each record may be 100 bytes in length, with the first field being the first 10 bytes, the second field ...
    Stuart WhiteStuart White
    May 28, 2009 at 12:16 pm
    Jun 2, 2009 at 5:32 am
  • Hi. I'm looking to move the Hadoop NameNode URL outside the hadoop-site.xml file, so I could set it at the run-time. Any idea how to do it? Or perhaps there is another configuration that can be ...
    Stas OskinStas Oskin
    May 24, 2009 at 10:03 pm
    May 26, 2009 at 7:11 am
  • Hello,everyone I am new to hama. in our project ,my team leader let me upload old code, run it on hadoop with parallel matrix computation.this is old code: public class EigenFaceGenerator { Matrix ...
    May 20, 2009 at 7:09 am
    May 25, 2009 at 3:10 am
  • Hi, all In May 9, we held the second Hadoop In China salon. About 150 people attended, 46% of them are engineers/managers from industry companies, and 38% of them are students/professors from ...
    He YongqiangHe Yongqiang
    May 16, 2009 at 1:10 am
    May 19, 2009 at 1:02 am
  • The following graphic outlines the architecture for HDFS: http://hadoop.apache.org/core/docs/current/images/hdfsarchitecture.gif If one is to write a client that adds data into HDFS, it needs to add ...
    Sasha DolgySasha Dolgy
    May 17, 2009 at 2:55 pm
    May 18, 2009 at 2:19 pm
  • Hi, I want to test how Hadoop and HBase are performing. I have a cluster with 1 namenode and 4 datanodes. I use Hadoop 0.19.1 and HBase 0.19.2. I first ran a few tests when the 4 datanodes use local ...
    Alexandra AlecuAlexandra Alecu
    May 14, 2009 at 2:44 pm
    May 15, 2009 at 6:33 pm
  • All, I have read some recommendation regarding image (binary input) processing using Hadoop-streaming which only accept text out-of-box for now. ...
    May 14, 2009 at 4:40 pm
    May 15, 2009 at 2:15 pm
  • Hi all, I have a few large files (4 that are 1.8GB+) I'm trying to copy from HDFS to S3. My micro EC2 cluster is running Hadoop 0.19.1, and has one master/two slaves. I first tried using the hadoop ...
    Ken KruglerKen Krugler
    May 7, 2009 at 11:44 pm
    May 13, 2009 at 9:01 pm
  • Hi, I just ran into something rather scary: One of my datanode processes that I¹m running with ­Xmx256M, and a maximum number of Xceiver threads of 4095 had a virtual memory size of over 7GB (!). I ...
    Stefan WillStefan Will
    May 9, 2009 at 1:12 am
    May 12, 2009 at 10:52 am
  • Hi all, I have a application want the rules of sorting and grouping use different Comparator. I had tested 0.19.1 and 0.20.0 about this function, but both do not work for Combiner. In 0.19.1, I use ...
    May 7, 2009 at 10:02 am
    May 12, 2009 at 3:58 am
  • I have two reducers running on two different machines. I ran the example word count program with some of my own System.out.println() statements to see what is going on. There were 2 slaves each ...
    Foss UserFoss User
    May 7, 2009 at 10:42 am
    May 8, 2009 at 8:22 am
  • Hey all, I'm going to be speaking at OSCON about my company's experiences with Hadoop and Friends, but I'm having a hard time coming up with a name for the entire software ecosystem. I'm thinking of ...
    Bradford StephensBradford Stephens
    May 5, 2009 at 2:45 am
    May 6, 2009 at 10:01 am
  • If I've got a sequence of streaming jobs, each of which depends on the output of the previous one, is there a good way to launch that sequence? Meaning, I want step B to only start once step A has ...
    Dan MilsteinDan Milstein
    May 1, 2009 at 4:35 pm
    May 4, 2009 at 6:09 am
  • Hi all, I am running the hadoop-0.19.1 and met strange problem in these days. Several days before, hadoop run smoothly and three nodes have been running TaskTracker and DataNode deamons. However, one ...
    Ian jonhsonIan jonhson
    May 30, 2009 at 7:23 am
    Jun 1, 2009 at 2:40 pm
  • Hi, I have a need to randomize my input file before processing. I understand I can chain Hadoop jobs together so the first could take the input file randomize it and then the second could take the ...
    John ClarkeJohn Clarke
    May 21, 2009 at 2:19 pm
    May 27, 2009 at 8:56 am
  • Version 19.1 with patches: 4780-2v19.patch (Jira 4780) closeAll3.patch (Jira 3998) I have confirmed that https://issues.apache.org/jira/browse/HADOOP-4924patch is in, so that is not the fix. We are ...
    Lance RiedelLance Riedel
    May 22, 2009 at 4:33 pm
    May 22, 2009 at 10:31 pm
  • Hi, I am want to load data in mysql using a hadoop file similar to following: LOAD DATA INFILE 'test.txt' INTO TABLE test FIELDS TERMINATED BY ',' LINES STARTING BY 'xxx'; But how do I load the hdfs ...
    May 19, 2009 at 9:44 pm
    May 20, 2009 at 12:20 am
  • Hi, I am working on a project that is suited to Hadoop and so want to create a small cluster (only 5 machines!) on our servers. The servers are however used during the day and (mostly) idle at night. ...
    John ClarkeJohn Clarke
    May 19, 2009 at 1:36 pm
    May 19, 2009 at 4:01 pm
  • Hi there, forgive the repost: Right now data is received in parallel and is written to a queue, then a single thread reads the queue and writes those messages to a FSDataOutputStream which is kept ...
    Sasha DolgySasha Dolgy
    May 15, 2009 at 1:36 pm
    May 18, 2009 at 2:00 pm
  • Hi, I am trying to do 'on demand map reduce' - something which will return in reasonable time (a few seconds). My dataset is relatively small and can fit into my datanode's memory. Is it possible to ...
    Matt BowyerMatt Bowyer
    May 10, 2009 at 9:30 pm
    May 11, 2009 at 6:20 pm
  • 1. Do the reducers of a job start only after all mappers have finished? 2. Say there are 10 slave nodes. Let us say one of the nodes is very slow as compared to other nodes. So, while the mappers in ...
    Foss UserFoss User
    May 6, 2009 at 7:22 pm
    May 7, 2009 at 8:32 am
  • Hi, I have implemented a subclass of RecordReader to handle a plain text file format where a record is multi-line and of variable length. Schematically each record is of the form some_title foo bar ...
    Rajarshi GuhaRajarshi Guha
    May 5, 2009 at 9:37 pm
    May 6, 2009 at 2:21 pm
  • Hello, I am using Hadoop on a small storage cluster (x86_64, CentOS 5.3, Hadoop-0.19.1). The hdfs is mounted using fuse and everything seemed to work just fine so far. However, I noticed that I ...
    Robert EngelRobert Engel
    May 2, 2009 at 12:57 am
    May 4, 2009 at 9:36 pm
  • Hi. I have a process that writes to file on DFS from time to time, using OutputStream. After some time of writing, I'm starting getting the exception below, and the write fails. The DFSClient retries ...
    Stas OskinStas Oskin
    May 25, 2009 at 4:27 pm
    Dec 12, 2009 at 1:46 am
Group Navigation
period‹ prev | May 2009 | next ›
Group Overview
groupcommon-user @

205 users for May 2009

Jason hadoop: 53 posts Stas Oskin: 38 posts Todd Lipcon: 30 posts Foss User: 27 posts Tom White: 27 posts Steve Loughran: 23 posts Raghu Angadi: 21 posts Sasha Dolgy: 18 posts Aaron Kimball: 15 posts Alex Loddengaard: 12 posts Dealmaker: 11 posts Edward Capriolo: 11 posts George Pang: 11 posts Lance Riedel: 11 posts Jothi Padmanabhan: 10 posts Owen O'Malley: 10 posts Pankil Doshi: 10 posts Ricky Ho: 9 posts Brian Bockelman: 8 posts John Clarke: 8 posts
show more