Search Discussions

134 discussions - 558 posts

  • Hi. I'm using the stock Ext3 as the most tested one, but I wonder, has someone ever tried, or even using there days in production another file system, like JFS, XFS or even maybe Ext4? I'm exploring ...
    Stas OskinStas Oskin
    Oct 8, 2009 at 11:38 am
    Oct 12, 2009 at 9:12 am
  • Hi, Is there a way to tell what kind of performance numbers one can expect out of their cluster given a certain set of specs. For example i have 5 nodes in my cluster that all have the following ...
    Usman WaheedUsman Waheed
    Oct 14, 2009 at 8:53 am
    Oct 20, 2009 at 12:14 am
  • Hi. I'm want to keep a checkpoint data on several separate machines for backup, and deliberating between exporting these machines disks via NFS, or actually running Secondary Name Nodes there. Can ...
    Stas OskinStas Oskin
    Oct 21, 2009 at 2:45 pm
    Dec 24, 2009 at 7:16 pm
  • As this topic comes up reasonably often on the list, I thought others might be interested in this: ...
    Todd LipconTodd Lipcon
    Oct 9, 2009 at 5:44 pm
    Dec 5, 2009 at 1:09 am
  • Hi I'm looking into Name Node high availability, but so far found only an approach using DRBD. I tried to make it work using Xen over DRBD, but it didn't quite work - in fact I received a very ...
    Stas OskinStas Oskin
    Oct 1, 2009 at 5:53 pm
    Oct 5, 2009 at 12:01 pm
  • Hi, I have a cluster setup with 3 nodes, and I'm adding hostname details (in /etc/hosts) manually in each node. Seems it is not an effective approach. How this scenario is handled in big clusters? Is ...
    Oct 19, 2009 at 1:40 pm
    Oct 21, 2009 at 9:51 am
  • Hi all, Using hadoop 0.20.1 I am seeing the following on my namenode startup. 2009-10-14 11:09:54,232 INFO org.apache.hadoop.ipc.Server: Error register getProtocolVersion ...
    Tim robertsonTim robertson
    Oct 14, 2009 at 9:33 am
    Oct 23, 2009 at 12:03 pm
  • Hi, I need to number all output records consecutively, like, 1,2,3... This is no problem with one reducer, making recordId an instance variable in the Reducer class, and setting ...
    Mark KerznerMark Kerzner
    Oct 28, 2009 at 3:55 am
    Oct 28, 2009 at 7:30 pm
  • Hi! I have quite odd Hadoop behavior. I wrote a client to my app that simply is trying to talk to HDFS and do stuff. Version of Hadoop is 20.0. I still suspect CLASSPATH, but would be nice to know ...
    Bogdan M. MaryniukBogdan M. Maryniuk
    Oct 22, 2009 at 4:24 am
    Oct 23, 2009 at 12:09 am
  • Hi, We are trying to set up a cluster (starting with 2 machines) using the new 0.20.1 version. On the master machine, just after the server starts, the name node dies off with the following ...
    Tejas LagvankarTejas Lagvankar
    Oct 13, 2009 at 2:17 pm
    Oct 13, 2009 at 6:30 pm
  • Hi, I have a setup where logs are periodically bundled up and dumped into hadoop dfs as large sequence file. It works fine for all my map reduce jobs. Now i need to handle adhoc queries for pulling ...
    Ishwar ramaniIshwar ramani
    Oct 1, 2009 at 5:49 pm
    Oct 5, 2009 at 9:32 pm
  • Hi, I am writing an M-R code using MapRunnable interface. The input format is SequenceFileInputFormat. Each Sequence-record contains a key-value pair of type <Text key,Text value (Text: ...
    Oct 29, 2009 at 12:19 pm
    Dec 22, 2009 at 3:29 pm
  • Hello Sir! I am new to hadoop. I have a project based on webservices. I have my information in 4 databases with different files in each one of them. Say, images in one, video, documents etc. My task ...
    Oct 15, 2009 at 1:50 am
    Oct 29, 2009 at 11:42 am
  • So my company is looking at only using dell or hp for our hadoop cluster and a sun thumper to backup the data. The prices are ok, after a 40% discount, but realistically I am paying twice as much as ...
    Alex NewmanAlex Newman
    Oct 15, 2009 at 3:48 pm
    Oct 15, 2009 at 7:21 pm
  • Dear Huy Phan and others, Thanks a lot for your efforts in customizing the WebDav server<http://github.com/huyphan/HDFS-over-Webdav and make it work for Hadoop-0.20.1. After setting up the WebDav ...
    Zhang Bingjun (Eddy)Zhang Bingjun (Eddy)
    Oct 27, 2009 at 10:28 am
    Oct 28, 2009 at 7:40 am
  • Hello Hadoop Users, Me and another friend of mine are looking out for some of the project ideas based on hadoop as a part of our curriculum . Can you give us some pointers please Thanks in advance ! ...
    Oct 14, 2009 at 10:09 am
    Oct 18, 2009 at 7:00 pm
  • I am using the 0.3 Cloudera scripts to start a Hadoop cluster on EC2 of 11 c1.xlarge instances (1 master, 10 slaves), that is the biggest instance available with 20 compute units and 4x 400gb disks. ...
    Chris SelineChris Seline
    Oct 13, 2009 at 4:06 pm
    Oct 15, 2009 at 7:58 pm
  • Hi, according to the API-Dokumentation of 0.20.1 JobConf is deprecated and we should use Configuration instead. However all examples on the webpage still referece JobConf. Is there a good example for ...
    Oliver B. FischerOliver B. Fischer
    Oct 21, 2009 at 12:49 pm
    Oct 29, 2009 at 4:26 am
  • Hi, We're rather proud to announce an updated beta release of Karmasphere Studio for Hadoop, a cross-platform desktop IDE for developing, debugging, deploying and monitoring applications based on ...
    Oct 10, 2009 at 12:49 am
    Oct 16, 2009 at 9:19 pm
  • Hi, the strings I am writing in my reducer have characters that may present a problem, such as char represented by decimal 254, which is hex FE. It seems that instead I see hex C3, or something else ...
    Mark KerznerMark Kerzner
    Oct 9, 2009 at 10:11 pm
    Oct 13, 2009 at 3:36 am
  • After looking at the HBaseRegionServer and its functionality, I began wondering if there is a more general use case for memory caching of HDFS blocks/files. In many use cases people wish to store ...
    Edward CaprioloEdward Capriolo
    Oct 6, 2009 at 3:17 pm
    Oct 7, 2009 at 3:21 pm
  • Hi, I have written a code to create sequence files for given text files. The program takes following input parameters: 1. Local source directory - contains all the input text files 2. Destination ...
    Oct 27, 2009 at 8:44 am
    Oct 27, 2009 at 3:19 pm
  • I installed hadoop on my workstation today (pseudo distributed mode) using the instructions: http://hadoop.apache.org/common/docs/r0.19.1/quickstart.html#Download The installation is straightforward, ...
    Stephane BrossierStephane Brossier
    Oct 16, 2009 at 3:08 pm
    Oct 17, 2009 at 2:22 am
  • Hi all, I'm getting the following on initializing my NameNode. The actual line throwing the exception is if (atime != -1) { - long inodeTime = inode.getAccessTime(); Have I corrupted the fsimage or ...
    Bryn DiveyBryn Divey
    Oct 14, 2009 at 4:24 pm
    Oct 15, 2009 at 6:32 pm
  • I'm trying to implement Writable interface. but not sure how to serialize/write/read data from nested objects in public class StorageClass implements Writable{ public String xStr; public String yStr; ...
    Oct 15, 2009 at 7:34 am
    Oct 15, 2009 at 8:32 am
  • Hi all, Given a map task, I need to know the IP address of the machine where that task is running. Is there any existing method to get that information? Thank you, Van
    Long Van Nguyen DinhLong Van Nguyen Dinh
    Oct 14, 2009 at 2:22 am
    Oct 15, 2009 at 6:17 am
  • Hey Cloudera genius guys . I read this Via Cloudera, Hadoop is currently used by most of the giants in the space including Google, Yahoo, Facebook (we wrote about Facebook’s use of Cloudera here), ...
    Smith StanSmith Stan
    Oct 2, 2009 at 11:03 pm
    Oct 7, 2009 at 9:18 am
  • Hi all, Why isn't the dfs.safemode.threshold.pct 1 by default? When dfs.replication.min=1 with dfs.safemode.threshold.pct=0.999, there might be chances for a NameNode to check in with incomplete data ...
    Manhee JoManhee Jo
    Oct 6, 2009 at 6:04 am
    Oct 7, 2009 at 1:54 am
  • Hi, I am new to Hadoop. I just configured it based on the documentation. While I was running example program wordcount.java, I am getting errors. When I gave command $ /bin/hadoop dfs -mkdir santhosh ...
    Santosh gandhamSantosh gandham
    Oct 8, 2009 at 9:24 am
    Nov 2, 2012 at 2:56 pm
  • Hi, I have input files, that contain NO carriage returns/line feeds. Each record is a fixed length (i.e. 202 bytes). Which FileInputFormat should I be using? so that each call to my Mapper receives ...
    Oct 20, 2009 at 8:53 pm
    Nov 1, 2009 at 6:44 pm
  • Hi all, I'd like to know does the map task push map output to reduce task or reduce task pull it from map task ? Which way is real in hadoop ? Thank you very much. Jeff zhang
    Jeff ZhangJeff Zhang
    Oct 27, 2009 at 1:05 am
    Oct 27, 2009 at 5:17 am
  • Hi, I currently have an app written to use 0.18.3 (cloudera ec2 dist) and it is working fine. Are there any significant advantages to move to the new stable 0.20.1? The app uses a custom MapRunnable ...
    John ClarkeJohn Clarke
    Oct 20, 2009 at 8:44 am
    Oct 23, 2009 at 8:35 am
  • I just downloaded and installed hadoop ver 0.200.1 and cygwin 1.5.25-15 and installed them (Windows XP.) I'm having trouble with ssh. When I enter "ssh localhost" I'm prompted for a password. I can ...
    Dennis DiMariaDennis DiMaria
    Oct 21, 2009 at 9:40 pm
    Oct 22, 2009 at 3:03 pm
  • I need some help with setting up a Hadoop cluster. The datanode on the slave is not coming up throwing java.net.NoRouteToHostException: No route to host. Please see the details below. I have a centos ...
    Oct 18, 2009 at 1:44 pm
    Oct 21, 2009 at 3:46 pm
  • I and running a hadoop program to perform MapReduce work on files inside a folder. My program is basically doing Map and Reduce work, each line of any file is a pair of string, and the result is a ...
    Kunsheng ChenKunsheng Chen
    Oct 19, 2009 at 2:57 am
    Oct 20, 2009 at 2:02 am
  • Hi, I am new to Hadoop so this might be an easy question for someone to help me with. I continually am getting this exception (my code follows below) java.io.IOException: Type mismatch in key from ...
    Oct 15, 2009 at 7:50 pm
    Oct 17, 2009 at 8:21 pm
  • Hi all, I have a set of map red jobs which need to be cascaded ,i.e, output of MR job1 is the input of MR job2. etc.. Can anyone point me to the corresponding classes in hadoop 0.20.0 API? I have ...
    Bharath vBharath v
    Oct 2, 2009 at 10:30 am
    Oct 17, 2009 at 8:16 pm
  • There is something wrong with network, so i killed all the hadoop thread buy "kill -9 pid" when i try to start hadoop today, it can't leave safemode automatically! the web ui shows: *Safe mode is ON. ...
    Oct 13, 2009 at 1:42 am
    Oct 13, 2009 at 5:30 am
  • I am having issues having multiple values in my value field.My desired result is <key ,<float,int or even <key ,<float,float . It seems easy in Python where I can pass a tuple as value.What is the ...
    Akshaya iyengarAkshaya iyengar
    Oct 6, 2009 at 5:55 am
    Oct 7, 2009 at 1:00 am
  • I have come across a problem. I just want to sort the num from 1 to 100, and with a maptask to map 1 to 50, with another to map 51 to 100, then how can I configure the jobconf? -- Huang Qian(黄骞) ...
    Huang QianHuang Qian
    Oct 5, 2009 at 10:31 pm
    Oct 6, 2009 at 5:24 am
  • Hello everyone, What will be easiest way to pass Dynamic value to map class?? Dynamic value are arguments given at run time. Pankil
    Pankil DoshiPankil Doshi
    Oct 5, 2009 at 11:51 pm
    Oct 6, 2009 at 5:09 am
  • Hi all, Whats the difference between the Job classes present in o.a.h.mapred.jobcontrol and o.a.h.mapreduce .. Both have different types of constructors , different functions etc.. Which one should ...
    Bharath vissapragadaBharath vissapragada
    Oct 3, 2009 at 6:04 pm
    Oct 5, 2009 at 8:21 am
  • hello everyone, i have a problem in hadoop startup ,every time i try to start hadoop name node doesnot start and when i tried to stop name node ,it gives an error :no name node to start. i tried to ...
    Oct 8, 2009 at 9:02 am
    Aug 11, 2011 at 10:28 am
  • 1. When I build hive-0.4.0, ivy would try to download hadoop, 0.18.3, 0.19.0 and 0.20.0. But always fail for 2. Then I modified shims/ivy.xml and shims/build.xml to remove ...
    Schubert ZhangSchubert Zhang
    Oct 19, 2009 at 5:03 pm
    Feb 15, 2010 at 10:15 pm
  • Hi There is 1 GB of rdf/owl files that I am executing on EC2. Execution throws the following exception ------------------- 08/11/19 16:08:27 WARN mapred.JobClient: Use GenericOptionsParser for ...
    Harshit KumarHarshit Kumar
    Oct 29, 2009 at 4:30 am
    Oct 30, 2009 at 5:15 am
  • I am using a Python script as a mapper for a Hadoop Streaming (hadoop 0.20.0) job, with reducer NONE. My jobs keep getting killed with "task failed to respond after 600 seconds." I tried sending a ...
    Ryan RosarioRyan Rosario
    Oct 25, 2009 at 7:01 pm
    Oct 27, 2009 at 2:48 pm
  • Hello, In my application I need to reduce the original reducer output keys further. I was reading about Chainreducer and Chainmappers but looks like it is for : one or more mapper - reducer - 0 or ...
    Oct 22, 2009 at 11:17 pm
    Oct 23, 2009 at 5:20 pm
  • Hi everyone. I am working on a project with hadoop and now I come across some problem. How can I deploy 100 files, with each file have one block by setting the blocksize and controling the file size, ...
    Huang QianHuang Qian
    Oct 15, 2009 at 6:40 pm
    Oct 20, 2009 at 12:06 pm
  • Hey all, While running the (latest as of Friday) Cloudera-created EC2 scripts, I noticed that running the terminate-cluster script kills ALL of your EC2 nodes, not just those associated with the ...
    Mark StetzerMark Stetzer
    Oct 19, 2009 at 3:41 pm
    Oct 19, 2009 at 4:53 pm
  • The developer's machine is Hadoop 0.20.1, Jar is compiled on the developer's machine. The server is Hadoop 0.18.3-cloudera. How can I run my mapreduce program on the server? 好玩贺卡等你发,邮箱贺卡全新上线! ...
    Oct 13, 2009 at 3:01 am
    Oct 17, 2009 at 6:56 am
Group Navigation
period‹ prev | Oct 2009 | next ›
Group Overview
groupcommon-user @

177 users for October 2009

Amandeep Khurana: 22 posts Stas Oskin: 22 posts Jason Venner: 20 posts Tim robertson: 20 posts Todd Lipcon: 19 posts Edward Capriolo: 15 posts Aaron Kimball: 14 posts Amogh Vasekar: 14 posts Mark Kerzner: 13 posts Steve Loughran: 13 posts Brian Bockelman: 12 posts Bogdan M. Maryniuk: 10 posts Sudha sadhasivam: 10 posts Allen Wittenauer: 9 posts Huy Phan: 9 posts Jeff Zhang: 9 posts Eason.Lee: 8 posts Shwitzu: 7 posts Usman Waheed: 7 posts Yibo820217: 7 posts
show more