Search Discussions

108 discussions - 389 posts

  • Cassandra sees this error with 0.21 of hadoop Exception in thread "main" java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected I see ...
    Steve LewisSteve Lewis
    Nov 11, 2010 at 9:02 pm
    Nov 16, 2010 at 12:52 pm
  • Hi, I'm a student. I'm reading the source code of Hadoop Common recently. I find a problem when I load the source code of Common into Eclipse. In the Package Explorer of Eclipse, some packages are ...
    Runtao caoRuntao cao
    Nov 5, 2010 at 9:39 am
    Nov 9, 2010 at 8:19 pm
  • i have only one machine and it's powerful. so, i want the all the slaves and master on one machine? thx in advanced
    Nov 30, 2010 at 7:12 am
    Dec 1, 2010 at 7:18 am
  • Hello, I found in Hadoop that reducers starts when a fraction of the number of mappers is complete. However, in my case, I hope reducers to start only when all mappers are complete. I searched for ...
    Da ZhengDa Zheng
    Nov 28, 2010 at 7:40 am
    Dec 2, 2010 at 9:30 am
  • Newbie alert. I have a Pig script I tested on small data and am now running it on a larger data set (85GB). My cluster is two machines right now, each with 16 cores and 32G of ram. I configured ...
    Greg LangmeadGreg Langmead
    Nov 16, 2010 at 10:50 pm
    Dec 9, 2010 at 8:46 pm
  • I'm trying to set up a Hadoop cluster. However, when I try to start the JobTracker, I get the following error (which only shows up in the logfile on the JobTracker server): 2010-11-19 17:41:15,977 ...
    Skye BerghelSkye Berghel
    Nov 20, 2010 at 1:56 am
    Nov 24, 2010 at 7:07 pm
  • I setup the cluster configuration in "masters", "slaves", "core-site.xml", "hdfs-site.xml", "mapred-site.xml" and copy to all the machines. And I login to one of the machines and use the following to ...
    Ricky HoRicky Ho
    Nov 23, 2010 at 6:13 pm
    Nov 24, 2010 at 6:27 am
  • Hi, I have set up ganglia for my cluster, and it works fine. What are the changes I need to make to make ganglia show hadoop related parameters? My gmond/gmetad config is default except for one ...
    Hari SreekumarHari Sreekumar
    Nov 22, 2010 at 12:55 pm
    Nov 23, 2010 at 5:33 am
  • Hi I have a question to you: I developed a program using Hadoop, it has one map function and one reduce function (like WordCount) and in the map function I do all the process of my data when I run ...
    Cornelio IñigoCornelio Iñigo
    Nov 17, 2010 at 9:17 am
    Nov 19, 2010 at 3:39 am
  • How are the following configs supposed to be used? mapred.cluster.map.memory.mb mapred.cluster.reduce.memory.mb mapred.cluster.max.map.memory.mb mapred.cluster.max.reduce.memory.mb ...
    Amandeep KhuranaAmandeep Khurana
    Nov 1, 2010 at 9:14 pm
    Nov 7, 2010 at 9:19 am
  • Hi , Running jadoop job from time to time I got such exception (from one of the reducers): The questions are : 1) What does this exception means for the data integrity? 2) Does it mean that part of ...
    Oleg RuchovetsOleg Ruchovets
    Nov 2, 2010 at 6:38 pm
    Nov 4, 2010 at 4:58 am
  • Hello, I'm trying to get Hadoop working on Cygwin/Windows XP in Pseudo-Distributed Mode. I downloaded version 0.21.0 and unpacked it to a cygwin directory. I'm following the quickstart directions ...
    Christopher WorleyChristopher Worley
    Nov 11, 2010 at 5:37 pm
    Nov 16, 2010 at 12:24 pm
  • I'm trying to compile a mapReduce program but it says: X conf.setInputPath( ... method setIputPath is undefined for type JobConf and when I checked the JobConf class within built-path ...
    Nov 10, 2010 at 9:45 pm
    Nov 16, 2010 at 6:03 am
  • Hello, I wrote a MapReduce program and ran it on a 3-node hadoop cluster, but its running time varies a lot, from 2 minutes to 3 minutes. I want to understand how time is used by the map phase and ...
    Da ZhengDa Zheng
    Nov 11, 2010 at 7:52 pm
    Nov 12, 2010 at 5:41 am
  • hello, I am trying to setup an Hadoop cluster. From the docs, it says I need two master: NameNode and Jobtracker and one slave: datanode, tasktracker. so, I need at least 4 machines to set up a ...
    Fabio A. MirandaFabio A. Miranda
    Nov 9, 2010 at 6:47 pm
    Nov 10, 2010 at 10:40 am
  • Hi I have cluster of 4 machines and want to configure ganglia for monitoring purpose. I have read the wiki and add the following lines to hadoop-metrics.properties on each machine. ...
    Shuja RehmanShuja Rehman
    Nov 8, 2010 at 2:34 pm
    Nov 8, 2010 at 6:51 pm
  • Hi, I have problems making namenode and jobtracker remotely accessible. It seems several people have had this problem before but I was unfortunately not able to find a solution yet. I have a hadoop ...
    Henning BlohmHenning Blohm
    Nov 5, 2010 at 11:23 am
    Nov 5, 2010 at 1:22 pm
  • Hi Everyone, What I really wish for Thanksgiving is some one giving me clarification of how the inputSplit is working. Eg. public void map(LongWritable key, Text value, OutputCollector<Text, Text ...
    Nov 26, 2010 at 10:08 pm
    Nov 28, 2010 at 7:30 pm
  • I have been looking around on some configuration parameters to improve the performance of MapReduce. Basically, I'm looking at the mapred-site.xml and so far I have set the following values: ...
    Nov 15, 2010 at 7:37 pm
    Nov 17, 2010 at 8:34 am
  • Our group made a very poorly considered decision to build out cluster using Hadoop 0.21 We discovered that a number of programs written and running properly under 0.20.2 did not work under 0.21 The ...
    Steve LewisSteve Lewis
    Nov 13, 2010 at 6:36 pm
    Nov 16, 2010 at 5:31 pm
  • Dear all, Does anyone have an experience on working Hadoop Integration with SGE ( Sun Grid Engine ). It is open -source too ( sge-6.2u5 ). Did SGE really overcomes some of the deficiencies of Hadoop. ...
    Adarsh SharmaAdarsh Sharma
    Nov 11, 2010 at 10:59 am
    Nov 11, 2010 at 3:50 pm
  • 1. I have a 512 node cluster. I need to have 32 nodes do something else. They can be datanodes but I cannot run any map or reduce jobs on them. So I see three options. 1. Stop the tasktracker on ...
    Raj VRaj V
    Nov 4, 2010 at 2:05 am
    Nov 4, 2010 at 9:27 pm
  • I am using the new mapreduce.* API in my jobs on hadoop 0.20.2. - I actually have some utilities around job scheduling and such. Now I would like to use NLineInputFormat to parallelize some data ...
    Henning BlohmHenning Blohm
    Nov 9, 2010 at 3:13 pm
    Jan 15, 2011 at 1:19 am
  • Hi, I found some erratic behavior in hadoop 0.19.2, here is a simple test program: import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.*; import org.apache.hadoop.io.*; public ...
    Qing YanQing Yan
    Nov 24, 2010 at 9:14 am
    Nov 25, 2010 at 2:06 am
  • Hello, I am new to Hadoop and I think I'm doing something silly. I sent this e-mail from another account which isn't registered to hadoop user group. I am getting the following error in my reducer. ...
    Arindam KhaledArindam Khaled
    Nov 16, 2010 at 12:07 am
    Nov 17, 2010 at 7:43 am
  • I have installed the Eclipse plugin for MapReduce by following this link: http://code.google.com/edu/parallel/tools/hadoopvm/index.html Typically on Eclipse, when I hover on a class or method, it ...
    Nov 8, 2010 at 8:15 pm
    Nov 9, 2010 at 5:44 pm
  • Hello, I have a question regarding MapRed jobs. I have 24 nodes, each node have 4 disks (mnt – mnt3), 500GB each mnt. All balanced ( I used the balancer, except mnt, which have 97% used). My question ...
    Shavit NetzerShavit Netzer
    Nov 6, 2010 at 5:34 am
    Nov 6, 2010 at 9:34 am
  • Hi, all I'm trapping in a strange problem in this evening, I have been working on hadoop for several months, including modify the source code and re-compile it, I have never met any problem, but when ...
    Nan ZhuNan Zhu
    Nov 2, 2010 at 3:17 am
    Nov 2, 2010 at 11:06 pm
  • I am building a cluster using Michael G. Noll's instructions found here: http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/ I have set up two single node ...
    Greg TroyanGreg Troyan
    Nov 30, 2010 at 8:01 pm
    Dec 8, 2010 at 9:13 pm
  • We have two Hadoop clusters in two separate buildings. Both clusters are loading the same data from the same sources (the second cluster is for DR). We're looking at how we can recover the primary ...
    Nov 30, 2010 at 4:05 am
    Nov 30, 2010 at 6:52 pm
  • Hi, When I set my inputFileFormat to take an input directory with three files in, the job is processed on all three and the output is one containing the result from all of them. Instead I want the ...
    Nov 17, 2010 at 8:36 pm
    Nov 18, 2010 at 8:20 pm
  • Hi, I have one question regarding the use of password less ssh login by the hadoop user across the hosts. I want to understand when hadoop does password less ssh, is it once or can happen any time ...
    Nov 16, 2010 at 6:04 pm
    Nov 16, 2010 at 10:24 pm
  • The JobTracker API to get completed jobs completedJobs() returns vector of org.apache.hadoop.mapred.JobInProgress This class is only visible in package. I need to develop separate application to ...
    Jaydeep AyachitJaydeep Ayachit
    Nov 12, 2010 at 6:25 am
    Nov 12, 2010 at 7:16 am
  • Hi all. I have a query about a application I have been asked to implement in Hadoop MapReduce. It is the concordance application. Specification of the Concordance application. ------------------ ...
    Rob StewartRob Stewart
    Nov 11, 2010 at 1:42 am
    Nov 11, 2010 at 10:12 pm
  • Using a copy of the Cloudera security-enabled CDH3b3, we installed vanilla hadoop in /home/www/hadoop Now when a try to run a job as me I get permission errors - I am not even sure if the error is in ...
    Steve LewisSteve Lewis
    Nov 9, 2010 at 9:29 pm
    Nov 10, 2010 at 1:07 am
  • Hi, I've MPI-BLAST application to run on HDFS and evaluate Parallel I/O. Can I submit the job by using "mpirun" command(MPICH1, which is installed on my system)....? Or do I need to convert this to ...
    Nov 8, 2010 at 5:13 pm
    Nov 9, 2010 at 8:19 am
  • Hi, I have some pretty basic stuff on replication that I am no very clear about, even after reading the online docs.. 1. My understanding is that replication factor of x means any block of data in ...
    Hari SreekumarHari Sreekumar
    Nov 4, 2010 at 5:54 pm
    Nov 4, 2010 at 8:33 pm
  • Hi all, If I only have machines with Windows OS, and don't want to re-install Linux OS, is there any means for me to use CDH3? Is there any windows package for CDH3? Thanks in advance. -- Best ...
    Yu LiYu Li
    Nov 3, 2010 at 9:27 am
    Nov 4, 2010 at 1:20 am
  • Has anyone else observed setMaxMapAttempts() not having any effect? I'm still seeing four attempts per mapper task. Keith Wiley kwiley@keithwiley.com www.keithwiley.com "It's a fine line between ...
    Keith WileyKeith Wiley
    Nov 5, 2010 at 10:55 pm
    Dec 2, 2010 at 6:32 pm
  • Hey there, I am doing some tests and wandering which are the best practices to deal with very small files which are continuously being generated(1Mb or even less). I see that if I have hundreds of ...
    Marc SturleseMarc Sturlese
    Nov 29, 2010 at 11:26 pm
    Nov 30, 2010 at 6:27 pm
  • Thurday 25 Nov 2010 Hi I would like to write program to count frequency of word in collection of text files. First, i output every word in document and calculate number of words in that documents ...
    Tri DoanTri Doan
    Nov 25, 2010 at 9:31 pm
    Nov 26, 2010 at 9:57 pm
  • Hi list, I've installed the eclipse plugin on Eclipse Helios. In order to get it to work I had to replace hadoop-core.jar in the plugin by the one shipped in CDH3, and now I can browse the ...
    Evert LammertsEvert Lammerts
    Nov 24, 2010 at 10:23 am
    Nov 25, 2010 at 10:15 am
  • I am trying to debug my map/reduce (Hadoop) app with help of the logging. When I do grep -r in $HADOOP_HOME/logs/* There is no line with debug info found. I need your help. What am I doing wrong? ...
    Tali KTali K
    Nov 24, 2010 at 1:59 am
    Nov 24, 2010 at 2:43 am
  • Dear All, Java, Hadoop rookie here coming from biology, more wet lab than dry lab so far. I'd like to build a mapper which emits (charAt(i), i+1) (in StandardStringJava not in HadoopTextJava) pairs ...
    Attila CsordasAttila Csordas
    Nov 18, 2010 at 11:46 pm
    Nov 22, 2010 at 7:58 pm
  • Hi, all, I am working on a distributed searching system. Now I have one server only. It has to crawl pages from the Web, generate indexes locally and respond users' queries. I think this is too busy ...
    Bing LiBing Li
    Nov 19, 2010 at 4:26 pm
    Nov 19, 2010 at 6:01 pm
  • Hi, I'm using the MapFileOutputFormat to lookup values in MapFiles and keep getting "Could not obtain block" errors. I'm thinking it might be because ulimit is not set high enough. Has anyone else ...
    Kim VogtKim Vogt
    Nov 18, 2010 at 8:45 pm
    Nov 19, 2010 at 12:27 am
  • Hi all , I have been working with MapReduce and HDFS for sometime. So the procedure what I normally follow is : 1) copy in the input file from Local File System to HDFS 2) run the map reduce module ...
    Matthew JohnMatthew John
    Nov 15, 2010 at 5:37 am
    Nov 17, 2010 at 1:51 pm
  • I'm having a problem with a custom WritableComparable that I created to use as a Key object. I basically have a number of identifier's with a timestamp, and I'm wanting to group the Identifier's ...
    Aaron BaffAaron Baff
    Nov 12, 2010 at 12:29 am
    Nov 12, 2010 at 6:57 pm
  • Hi, What are the changes I need to make to run ganglia 3.1.7 on hadoop 0.20.2? I have used GangliaContext31, but I think I'll also need to apply the 4675 patch ...
    Hari SreekumarHari Sreekumar
    Nov 12, 2010 at 4:36 am
    Nov 12, 2010 at 2:16 pm
  • Hello, given: fabio@nodo1:~/hadoop$ cat conf/hdfs-site.xml <?xml version="1.0"? <?xml-stylesheet type="text/xsl" href="configuration.xsl"? <!-- Put site-specific property overrides in this file. -- ...
    Fabio A. MirandaFabio A. Miranda
    Nov 10, 2010 at 6:38 am
    Nov 10, 2010 at 2:17 pm
Group Navigation
period‹ prev | Nov 2010 | next ›
Group Overview
groupcommon-user @

146 users for November 2010

Hari Sreekumar: 25 posts Harsh J: 21 posts Steve Loughran: 17 posts Maha: 11 posts Steve Lewis: 11 posts Allen Wittenauer: 10 posts Da Zheng: 10 posts Henning Blohm: 8 posts Adarsh Sharma: 6 posts Alex Baranau: 6 posts Arun C Murthy: 6 posts Brian Bockelman: 6 posts Michael Segel: 6 posts Shavit Netzer: 6 posts Aaron Eng: 5 posts ANKITBHATNAGAR: 5 posts Fabio A. Miranda: 5 posts Konstantin Boudnik: 5 posts Shuja Rehman: 5 posts Amandeep Khurana: 4 posts
show more