FAQ

Hadoop > common-user > October 2008

Search Discussions

174 discussions - 658 posts

  • I am trying to plan out my map-reduce implementation and I have some questions of where computation should be split in order to take advantage of the distributed nodes. Looking at the architecture ...
    Terrence A. PietrondiTerrence A. Pietrondi
    Oct 1, 2008 at 5:17 pm
    Oct 8, 2008 at 9:40 pm
  • I don't quite know how to write the read and write functions, but I want to write my own writable, which should have a DoubleWritable/double value followed by a list of Strings/Text. This Writable ...
    Yih Sun KhooYih Sun Khoo
    Oct 19, 2008 at 7:31 am
    Oct 21, 2008 at 6:03 am
  • Hi, I'm trying to implement NameNode failover (or at least NameNode local data backup), but it is hard since there is no official documentation. Pages on this subject are created, but still empty: ...
    Tomislav PoljakTomislav Poljak
    Oct 28, 2008 at 1:44 pm
    Nov 12, 2008 at 4:09 pm
  • I've got a simple hadoop job running on an EC2 cluster using the scripts under src/contrib/ec2. The map tasks all fail with the following error: 08/10/07 15:11:00 INFO mapred.JobClient: Task Id : ...
    Dan BenjaminDan Benjamin
    Oct 7, 2008 at 7:18 pm
    May 5, 2009 at 10:10 pm
  • Hi, when using distcp, we often find that the distcp logs are in the target dir. root. for instance, bin/hadoop dist src dest We end up with: dest/_distcp_logs_... Is there a way to (1) tell distcp ...
    Sriram RaoSriram Rao
    Oct 15, 2008 at 11:10 pm
    Oct 17, 2008 at 5:24 pm
  • Hi all, I'm working with a 30 node Hadoop cluster that has just started demonstrating some weird behavior. It's run without incident for a few weeks.. and now: The cluster will run smoothly for ...
    Aaron KimballAaron Kimball
    Oct 29, 2008 at 7:34 pm
    Oct 31, 2008 at 7:57 am
  • Hi all, This RDF proposal is a good long time ago. Now we'd like to settle down to research again. I attached our proposal, We'd love to hear your feedback & stories!! Thanks. -- Best regards, Edward ...
    Edward J. YoonEdward J. Yoon
    Oct 21, 2008 at 1:03 am
    Oct 22, 2008 at 5:36 am
  • Greetings! Hi, Am trying to modify the WordCount.java mentioned at Example: WordCount v1.0<http://hadoop.apache.org/core/docs/current/mapred_tutorial.html#Example%3A+WordCount+v1.0 at ...
    LathaLatha
    Oct 5, 2008 at 5:13 pm
    Mar 30, 2012 at 2:56 am
  • Hi, I received below message. Can anyone explain this? 08/10/09 11:53:33 INFO mapred.JobClient: Task Id : task_200810081842_0004_m_000000_0, Status : FAILED java.io.IOException: Cannot run program ...
    Edward J. YoonEdward J. Yoon
    Oct 9, 2008 at 3:00 am
    Nov 27, 2008 at 2:07 am
  • Hi folks, Anybody tried scripting Hadoop on EC2 to... 1. Launch a cluster 2. Pull data from S3 3. Run a job 4. Copy results to S3 5. Terminate the cluster ... without any user interaction? -Stuart
    Stuart SierraStuart Sierra
    Oct 23, 2008 at 2:49 pm
    Nov 26, 2008 at 5:42 pm
  • Each month the developers at my company write a short article about a Java technology we find exciting. I've just finished one about Hadoop for November and am seeking a volunteer knowledgeable about ...
    Tom WheelerTom Wheeler
    Oct 23, 2008 at 10:31 pm
    Nov 5, 2008 at 3:50 am
  • Hi Everyone! I would like to implement Nagios health monitoring of a Hadoop grid. Some of you have some experience here, do you hace any approach or advice I could use. At this time I've been only ...
    Gerardo VelezGerardo Velez
    Oct 6, 2008 at 5:05 pm
    Oct 29, 2008 at 7:14 pm
  • Hey guys, We at Linkedin are trying to run some Large Graph Analysis problems on Hadoop. The fastest way to run would be to keep a copy of whole Graph in RAM at all mappers. (Graph size is about 8G ...
    Bhupesh BansalBhupesh Bansal
    Oct 16, 2008 at 8:58 pm
    Oct 20, 2008 at 10:27 am
  • hi does hadoop support graphics packages for displaying some images..? -- Best Regards S.Chandravadana This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) ...
    ChandraChandra
    Oct 7, 2008 at 8:51 am
    Oct 15, 2008 at 12:03 pm
  • Hi, We have a cluster where we running HDFS in non-rack-aware mode. Now, we want to switch HDFS to run in rack-aware mode. Apart from the config changes (and restarting HDFS), to rackify the existing ...
    Sriram RaoSriram Rao
    Oct 14, 2008 at 7:07 pm
    Sep 16, 2010 at 12:40 am
  • Hi, could anyone tell me what the current Status of FUSE support for HDFS is? Is this something that can be expected to be usable in a few weeks/months in a production environment? We have been ...
    Robert KrügerRobert Krüger
    Oct 31, 2008 at 4:09 pm
    Nov 6, 2008 at 6:55 pm
  • Hi, Over in Mahout (lucene.a.o/mahout), we are seeing an oddity with some of our clustering code and Hadoop 0.18.1. The thread in context is at: http://mahout.markmail.org/message/vcyvlz2met7fnthr ...
    Grant IngersollGrant Ingersoll
    Oct 28, 2008 at 2:06 am
    Nov 1, 2008 at 12:24 am
  • One of my first questions about hadoop was, "How do systems outside the cluster interact with the file system?" I read several documents that described streaming data into hadoop for processing, but ...
    Edward CaprioloEdward Capriolo
    Oct 23, 2008 at 8:05 pm
    Oct 28, 2008 at 8:20 pm
  • Error initializing attempt_200810220716_0004_m_000001_0: java.lang.IllegalArgumentException: Wrong FS: hdfs://192.168.52.129:9000/tmp/hadoop-root/mapred/system/job_200810220716_0004/job.xml, ...
    David WeiDavid Wei
    Oct 22, 2008 at 8:18 am
    Jul 29, 2009 at 1:47 am
  • Hi all, I'd like to remind everyone that the Hadoop Camp & ApacheCon US is coming up in New Orleans next month. http://tinyurl.com/hadoop-camp It will be the largest gathering of Hadoop developers ...
    Owen O'MalleyOwen O'Malley
    Oct 2, 2008 at 4:06 pm
    Nov 18, 2008 at 11:00 pm
  • I'm curious to what the best method for debugging and logging in Hadoop? I put together a small cluster today and a simple application to process log files. While it worked well, I had trouble trying ...
    Scott WhitecrossScott Whitecross
    Oct 30, 2008 at 2:15 am
    Oct 31, 2008 at 6:58 am
  • Hi All, Ive been working michael nolls multi-node cluster setup example (Running_Hadoop_On_Ubuntu_Linux) for hadoop and I have a working setup. I then on my slave machine -- which is currently ...
    WmitchellWmitchell
    Oct 27, 2008 at 5:23 pm
    Oct 29, 2008 at 2:41 pm
  • hi, I am wondering how to add more datanodes to an up-and-running hadoop instance? Couldn't find instructions on this from the wiki page. Thanks!
    Ski Gh3Ski Gh3
    Oct 20, 2008 at 9:55 pm
    Oct 22, 2008 at 8:10 am
  • Hi, I have installed lzo-2.03 to my Linux box. But still my code for writing a SequenceFile using LZOcodec returns the following error: util.NativeCodeLoader: Loaded the native-hadoop library ...
    Songting ChenSongting Chen
    Oct 10, 2008 at 12:59 am
    Oct 10, 2008 at 7:46 pm
  • Dear hadoop users, I'm lucky to work in academic environment where information security is not the question. However, I'm sure that most of the hadoop users aren't. Here is the question: how secure ...
    Dmitry PushkarevDmitry Pushkarev
    Oct 5, 2008 at 6:55 am
    Oct 6, 2008 at 2:18 pm
  • I have been working with Hive for the past week. The ability to wrap an SQL like tool over HDFS as very powerful. Now that I am comfortable with the concept, I am looking at an implementation of it. ...
    Edward CaprioloEdward Capriolo
    Oct 2, 2008 at 12:04 am
    Oct 2, 2008 at 9:48 pm
  • I want to stream data from logs into the HDFS in production but I do NOT want my production machine to be apart of the computation cluster. The reason I want to do it in this way is to take advantage ...
    Shahab mehmandoustShahab mehmandoust
    Oct 31, 2008 at 6:41 pm
    Nov 3, 2008 at 2:34 am
  • Each of my datanodes has a system and a data partition, with dfs.data.dir pointed to the data partition. The data partition just filled up to 100% on all of my nodes (as evident via df), but the ...
    Stefan WillStefan Will
    Oct 27, 2008 at 5:01 pm
    Oct 30, 2008 at 9:10 pm
  • I am attempting to write a map/reduce that will sort by the key and then by the values. The output should look like: 0 0 0 1 0 5 0 123 0 89245 1 0 1 234 1 23423 My mapper is Mapper<LongWritable, ...
    David M. CoeDavid M. Coe
    Oct 28, 2008 at 2:53 pm
    Oct 30, 2008 at 3:08 pm
  • Hello: I have installed hadoop on a cluster which hava 7 nodes, one is namenode and the other 6 nodes are datanode . and At that time It runs normally, and also I runned the wordcount example, It's ...
    ZhiHong FuZhiHong Fu
    Oct 15, 2008 at 8:29 am
    Oct 20, 2008 at 1:07 am
  • Hi, I want to add a jar file (that is required by mappers and reducers) to the classpath. Initially I had copied the jar file to all the slave nodes in the $HADOOP_HOME/lib directory and it was ...
    Tarandeep SinghTarandeep Singh
    Oct 6, 2008 at 9:31 pm
    Oct 17, 2008 at 5:26 pm
  • hi, I try to use contrib/index/hadoop-0.17.1-index.jar build lucene index, but Directory NoClassDefFoundError. How solve? Thinks. Additional, hadoop run Pseudo-Distributed Mode. and can run ...
    ChenlbspaceChenlbspace
    Oct 10, 2008 at 8:36 am
    Oct 16, 2008 at 5:44 am
  • I think each mapper/reducer runs in its own JVM which makes it impossible to share objects. I need to share a large object so that I can access it at memory speeds across all the mappers. Is it ...
    Devajyoti SarkarDevajyoti Sarkar
    Oct 3, 2008 at 3:41 am
    Oct 3, 2008 at 4:48 pm
  • I like to hear some good ways of passing constants from one job to the next. These are some ways that I can think of: 1) The obvious solution is to carry the constant as part of your value from one ...
    Yih Sun KhooYih Sun Khoo
    Oct 22, 2008 at 9:53 pm
    Nov 3, 2008 at 8:37 pm
  • Hi, I implemented customized classes for InputFormat, InputSplit and RecordReader in Java and was trying to use them in a C++ pipes job. The customized InputFormat class could be included using the ...
    Zhengguo 'Mike' SUNZhengguo 'Mike' SUN
    Oct 27, 2008 at 8:37 pm
    Oct 30, 2008 at 5:37 am
  • Hello : In hadoop InputFormat are always based on the InputFileFormat , But Now I will get data from a web service application. The data will be wrapped as ResultSet type. Now I am wandering " should ...
    ZhiHong FuZhiHong Fu
    Oct 27, 2008 at 6:28 am
    Oct 29, 2008 at 2:21 am
  • Hi, I've been wondering why there are separate task limits for map and reduce. Why not a single generic task limit per node ? Thanks for any insight, Doug
    Doug BalogDoug Balog
    Oct 27, 2008 at 8:45 pm
    Oct 28, 2008 at 10:51 pm
  • I am trying to write an InputFormat and I am having some trouble understanding how my data is being broken up. My input is a previous hadoop job and I have added code to my record reader to print out ...
    Malcolm MatalkaMalcolm Matalka
    Oct 28, 2008 at 1:31 pm
    Oct 28, 2008 at 7:21 pm
  • Hi everyone, I have a problem about Hadoop startup. I failed to startup the namenode and I got the following exception in the namenode log file: 2008-10-23 21:54:51,223 INFO ...
    Woody zhouWoody zhou
    Oct 24, 2008 at 2:14 am
    Oct 27, 2008 at 6:10 pm
  • Hi - I'm a complete newbie to hadoop, and am wondering if it's appropriate for configuring a bunch of older machines that have no other use, for use as a storage cluster on an otherwise windows ...
    David C. KerberDavid C. Kerber
    Oct 24, 2008 at 4:42 pm
    Oct 27, 2008 at 5:48 pm
  • What's the port number for distcp in 0.17.2? I can't find any documentation on distcp for version 0.17.2. For version 0.18, the documentation says it's 8020. I'm using a standard install and the only ...
    BzhengBzheng
    Oct 22, 2008 at 6:58 pm
    Oct 23, 2008 at 7:05 pm
  • Hi! I've developed a Map/Reduce algorithm to analyze some logs from web application. So basically, we are ready to start QA test phase, so now, I would like to now how efficient is my application ...
    Gerardo VelezGerardo Velez
    Oct 8, 2008 at 6:28 pm
    Oct 10, 2008 at 6:22 pm
  • Hi everyone, (apologies if this gets posted on the list twice for some reason, my first attempt was denied as "suspected spam") I ran a job last night with Hadoop 0.18.0 on EC2, using the standard ...
    Per JacobssonPer Jacobsson
    Oct 1, 2008 at 6:08 pm
    Oct 2, 2008 at 10:55 pm
  • Hi, for convenience reasons, I was wondering if there is a simple way to produce one output file per key in the Reducer? Thanks, Florian
    Florian LeibertFlorian Leibert
    Oct 29, 2008 at 12:25 am
    Oct 29, 2008 at 5:46 am
  • I am trying to write a map reduce function which takes take the following types of <key,value pairs Map function -- should read floating point values (i dont really care about key) it should output ...
    Pols cutPols cut
    Oct 26, 2008 at 3:33 am
    Oct 27, 2008 at 1:25 am
  • I am using Hadoop Streaming. The input are multiple files. Is there a way to get the current filename in mapper? For example: $HADOOP_HOME/bin/hadoop \ jar $HADOOP_HOME/hadoop-streaming.jar \ -input ...
    Steve GaoSteve Gao
    Oct 23, 2008 at 1:56 am
    Oct 26, 2008 at 9:19 pm
  • Hi all, I am sure someone must have tried mysql connection using hadoop. But I am getting problem. Basically I am not getting how to inlcude classpath of jar of jdbc connector in the run command of ...
    Deepak DiwakarDeepak Diwakar
    Oct 20, 2008 at 4:43 pm
    Oct 21, 2008 at 4:20 pm
  • I'm trying to play with Chukwa, but I'm struggling to get anything going. I've been operating off of the wiki entry (< http://wiki.apache.org/hadoop/Chukwa_Quick_Start ), making revisions as I go ...
    Alex LoddengaardAlex Loddengaard
    Oct 15, 2008 at 10:27 pm
    Oct 17, 2008 at 5:29 pm
  • If we have a group blog of the hadoop user/dev group such as a Y! developer network, we can easily share/introduce our experience and outcomes from our research. So, I thought about a group blog, I ...
    Edward J. YoonEdward J. Yoon
    Oct 8, 2008 at 12:30 pm
    Oct 9, 2008 at 12:12 am
  • the input is as follows. flag a b flag c d e flag f then I used a mapper to first store values and then emit them all when met with a line contains "flag" but when the file reached its end, I have no ...
    Zhou, YunqingZhou, Yunqing
    Oct 4, 2008 at 3:36 am
    Oct 4, 2008 at 8:21 am
Oct 2008
1234
Group Navigation
period‹ prev | Oct 2008 | next ›
Group Overview
groupcommon-user @
categorieshadoop
discussions174
posts658
users192
websitehadoop.apache.org...
irc#hadoop

192 users for October 2008

Edward J. Yoon: 24 posts Steve Loughran: 21 posts Alex Loddengaard: 19 posts David Wei: 17 posts Owen O'Malley: 17 posts Alex Loddengaard: 14 posts Arun C Murthy: 14 posts Edward Capriolo: 14 posts Yih Sun Khoo: 13 posts Amit k. Saha: 12 posts Chandravadana: 11 posts Steve Gao: 10 posts Malcolm Matalka: 9 posts Terrence A. Pietrondi: 9 posts ZhiHong Fu: 9 posts Amareshwari Sriramadasu: 8 posts Raghu Angadi: 8 posts Samuel Guo: 8 posts Allen Wittenauer: 7 posts Brian Bockelman: 7 posts
show more
Archives