Search Discussions

126 discussions - 462 posts

  • Regards to all the list. There are many people that use the Hadoop Tutorial released by Yahoo at http://developer.yahoo.com/hadoop/tutorial/ ...
    Marcos OrtizMarcos Ortiz
    Apr 4, 2012 at 12:32 pm
    Jun 19, 2012 at 4:04 am
  • Hi, I m trying to find out best way to add debugging in map- red code. I have System.out.println() statements that I keep on commenting and uncommenting so as not to increase stdout size But problem ...
    Mapred LearnMapred Learn
    Apr 20, 2012 at 3:18 pm
    May 8, 2012 at 12:24 am
  • Hi, Do hadoop have any web service or other interface so I can submit jobs from remote machine? Thanks, Arindam
    Arindam ChoudhuryArindam Choudhury
    Apr 20, 2012 at 12:08 pm
    Apr 21, 2012 at 3:22 pm
  • Strangely isee the tmp folder has enough space. What else could be the problem ? How much should my tmp space be ? Error: java.io.IOException: No space left on device at ...
    Nuthalapati, RameshNuthalapati, Ramesh
    Apr 25, 2012 at 7:13 pm
    Apr 26, 2012 at 1:59 pm
  • I am investigating automated methods of moving our data from the web tier into HDFS for processing, a process that's performed periodically. I am looking for feedback from anyone who has actually ...
    Karl HennigKarl Hennig
    Apr 20, 2012 at 10:14 pm
    Apr 22, 2012 at 4:24 pm
  • Hi All, Just wanted if hadoop supports more than one data centre. This is basically for DR purposes and High Availability where one centre goes down other can bring up. Regards, Abhishek
    Abhishek Pratap SinghAbhishek Pratap Singh
    Apr 11, 2012 at 6:45 pm
    Apr 19, 2012 at 11:44 pm
  • Hi, Is it possible to get the 'id' of the currently executing split or block from within the mapper? Using this block Id / split id, I want to be able to query the namenode to get the names of hosts ...
    Deepak NettemDeepak Nettem
    Apr 8, 2012 at 4:17 pm
    Apr 11, 2012 at 3:01 pm
  • Hello all, My map-reduce operation on Hadoop (running on Debian) is correctly starting and finding the input file. However, just after starting the map reduce, Hadoop tells me that it cannot find a ...
    Bas HickendorffBas Hickendorff
    Apr 3, 2012 at 1:28 pm
    Apr 3, 2012 at 8:01 pm
  • I had 20 mappers in parallel reading 20 gz files and each file around 30-40MB data over 5 hadoop nodes and then writing to the analytics database. Almost midway it started to get this error ...
    Mohit AnchliaMohit Anchlia
    Apr 26, 2012 at 11:50 pm
    Apr 29, 2012 at 8:26 pm
  • Hello everyone ! I have a problem with MapReduce [:(] like that : I have 4 file input with 3 fields : teacherId, classId, numberOfStudent (numberOfStudent is ordered by desc for each teach) Output is ...
    Lac TrungLac Trung
    Apr 24, 2012 at 2:39 am
    Apr 24, 2012 at 12:38 pm
  • Hey guys, I've been stuck with HCE installation for two days now and can't figure out the problem. Errors I get from running (sh build.sh) is "can not execute binary file" . I tried setting my ...
    Mark questionMark question
    Apr 18, 2012 at 9:26 pm
    Apr 20, 2012 at 3:17 pm
  • Hello, When I start a map-reduce job, it starts, and after a short while, fails with the error below (SnappyCodec not found). I am currently starting the job from other Java code (so the Hadoop ...
    Bas HickendorffBas Hickendorff
    Apr 14, 2012 at 11:36 am
    Apr 16, 2012 at 9:24 am
  • Hi, I wanted to know if there are any existing API's within Hadoop for us to do some text analysis like sentiment analysis, etc. OR are we to rely on tools like R, etc. for this. Regards, Karanveer ...
    Karanveer SinghKaranveer Singh
    Apr 25, 2012 at 12:51 pm
    Apr 26, 2012 at 8:08 am
  • I am going through the chapter "How mapreduce works" and have some confusion: 1) Below description of Mapper says that reducers get the output file using HTTP call. But the description under "The ...
    Mohit AnchliaMohit Anchlia
    Apr 4, 2012 at 11:56 pm
    Apr 5, 2012 at 5:57 pm
  • Dear All, I have installed *Hadoop-fuse* to mount the HDFS filesystem locally . I could mount the HDFS without any issues.But I am not able to do any file operations like *delete, copy, move* etc ...
    Manu SManu S
    Apr 26, 2012 at 10:51 am
    Apr 26, 2012 at 2:24 pm
  • I require each input file to be processed by each mapper as a whole. I subclass c.o.a.h.mapreduce.lib.input.TextInputFormat and override isSplitable() to invariably return false. The job is ...
    Dan DrewDan Drew
    Apr 23, 2012 at 11:40 am
    Apr 24, 2012 at 1:54 pm
  • Hi All, Is there a way for me to set global counters in Mapper and access them from reducer? Could you suggest how I can acheve this? Thanks Gayatri
    Gayatri RaoGayatri Rao
    Apr 20, 2012 at 4:14 pm
    Apr 21, 2012 at 6:25 am
  • Hello, I would like to ask you if it is possible to create and work with a temporary file while in a map function. I suppose that map function is running on a single node in Hadoop cluster. So what ...
    Ondřej KlimperaOndřej Klimpera
    Apr 7, 2012 at 8:45 pm
    Apr 8, 2012 at 7:33 pm
  • Hi, Is it possible to get the execution time of the constituent map/reduce tasks of a MapReduce job (say sort) at the end of a job run? Preferably, can we obtain this programatically? Thanks, Bikash
    Bikash sharmaBikash sharma
    Apr 4, 2012 at 9:20 pm
    Apr 6, 2012 at 9:24 am
  • we need to connect to HIVE from Microstrategy reports, and it requires the Hive Thrift server. But I tried to start it, and it just hangs as below. # hive --service hiveserver Starting Hive Thrift ...
    Michael WangMichael Wang
    Apr 16, 2012 at 8:54 pm
    Apr 20, 2012 at 7:55 pm
  • 国际著名大型IT企业(排名前3位)开发中心招聘Hadoop技术专家(北京)-非猎头 职位描述: Hadoop系统和平台开发(架构师,资深开发人员) 职位要求: 1.有设计开发大型分布式系统的经验(工作年限3年以上,架构师5年以上),hadoop大型实际应用经验优先 2.良好的编程和调试经验(java or c++/c),扎实的计算机理论基础,快速的学习能力 3 ...
    Bing LiBing Li
    Apr 9, 2012 at 3:00 pm
    Apr 9, 2012 at 4:09 pm
  • Hi guys, quick question: Are there any performance gains from hadoop streaming or pipes over Java? From what I've read, it's only to ease testing by using your favorite language. So I guess it is ...
    Mark questionMark question
    Apr 5, 2012 at 6:54 pm
    Apr 7, 2012 at 6:38 am
  • hi all, i'm just started to play around with hdfs+mapred. i'm currently playing with teragen/sort/validate to see if i understand all. the test setup involves 5 nodes that all are tasktracker and ...
    Stijn De WeirdtStijn De Weirdt
    Apr 2, 2012 at 4:50 pm
    Apr 2, 2012 at 8:00 pm
  • Hi, Can someone point me to some info on Image processing using Hadoop? Regards, Shreya This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain ...
    Shreya PalShreya Pal
    Apr 2, 2012 at 9:32 am
    Apr 2, 2012 at 11:21 am
  • Hi, I am pretty a newbie and i am following the quick start guide for single node set up on windows using cygwin. In this step, $ bin/hadoop fs -put conf input I am getting the following errors. I ...
    Onder SEZGINOnder SEZGIN
    Apr 27, 2012 at 11:06 am
    Apr 29, 2012 at 12:20 am
  • Within my small 2 node cluster I set up my 4 core slave node to have 4 task trackers and I also limited my java heap size to -Xmx1024m Is there a possibility that when the data gets broken up that it ...
    Barry, Sean FBarry, Sean F
    Apr 26, 2012 at 8:46 pm
    Apr 27, 2012 at 4:57 am
  • Hi, I am new to hadoop and I am trying to understand hadoop job submission. We submit the job using: hadoop jar some.jar name input output this in turn invoke the RunJar . But in RunJar I can not ...
    Arindam ChoudhuryArindam Choudhury
    Apr 25, 2012 at 8:44 am
    Apr 25, 2012 at 10:26 am
  • Hello folks, I have a job that processes text files from hdfs on local fs (temp directory) and then copies those back to hdfs. I added another drive to each server to have better io performance, but ...
    Apr 22, 2012 at 7:21 am
    Apr 23, 2012 at 7:40 am
  • I am looking to test hadoop 0.23 or CDH4 beta on my local VM. I am looking to execute the sample example codes in new architecture, play around with the containers/resource managers. Is there any ...
    Praveenesh kumarPraveenesh kumar
    Apr 17, 2012 at 3:22 pm
    Apr 19, 2012 at 8:19 pm
  • I want to know if there is any way of reading a file from HDFS using a servlet . Suppose I have filename of a valid file situated over HDFS . How do I generate a URL to display that file on a jsp ...
    Sushil sontakkeSushil sontakke
    Apr 13, 2012 at 7:47 am
    Apr 15, 2012 at 9:13 am
  • i have a map reduce job that is generating a lot of intermediate key-value pairs. for example, when i am 1/3 complete with my map phase, i may have generated over 130,000,000 output records (which is ...
    Jane WayneJane Wayne
    Apr 3, 2012 at 8:39 am
    Apr 5, 2012 at 3:16 am
  • Hi guys: I have a map reduce job that runs normally on local file system from eclipse, *but* it fails on HDFS running in psuedo distributed mode. The exception I see is ...
    Jay VyasJay Vyas
    Apr 2, 2012 at 1:40 pm
    Apr 2, 2012 at 3:25 pm
  • Hi all, I want to use oozie to submit different workflows from different users. These users are able to submit hadoop jobs. I am using hadoop 0.20.205 and oozie 3.1.3 I have a hadoop user as a ...
    Praveenesh kumarPraveenesh kumar
    Apr 2, 2012 at 12:15 pm
    Apr 2, 2012 at 12:46 pm
  • Hi guys : 1) Does anybody know if there is a VM out there which runs EMR hadoop ? I would like to have a local vm for dev purposes that mirrored the EMR hadoop instances. 2) How does EMR's hadoop ...
    Jay VyasJay Vyas
    Apr 30, 2012 at 4:58 am
    May 1, 2012 at 9:03 am
  • Hello I'd like to ask you what is the preferred way of getting running jobs progress from Java application, that has executed them. Im using Hadoop 0.20.203, tried job.end.notification.url property ...
    Ondřej KlimperaOndřej Klimpera
    Apr 29, 2012 at 8:33 pm
    Apr 30, 2012 at 9:29 am
  • hduser@master:~ /usr/java/jdk1.7.0/bin/jps 20907 TaskTracker 20629 SecondaryNameNode 25863 Jps 20777 JobTracker 20383 NameNode 20507 DataNode hduser@master:~ stop- stop-all.sh stop-balancer.sh ...
    Barry, Sean FBarry, Sean F
    Apr 28, 2012 at 7:46 pm
    Apr 29, 2012 at 7:43 pm
  • hello everyone: i have a problem.we knew map() has it's own parameter such as map(Object key, Text value, Context context). the following is my structure. public class aaa{ public static class bbb { ...
    Apr 26, 2012 at 8:58 am
    Apr 26, 2012 at 3:28 pm
  • Hi Everybody, I am a newbie to hadoop. I have about 40K .tgz files each of approximately 3MB . I would like to process this as if it were a single large file formed by "cat list-of-files | ...
    Sunil S NandihalliSunil S Nandihalli
    Apr 24, 2012 at 4:07 pm
    Apr 24, 2012 at 5:37 pm
  • Hello, I'd like to ask you if there is a possibility of setting a timeout for processing one input line of text input in mapper function. The idea is, that if processing of one line is too long, ...
    Ondřej KlimperaOndřej Klimpera
    Apr 18, 2012 at 12:39 pm
    Apr 18, 2012 at 2:12 pm
  • I am a newbie to Unix/Hadoop and have basic questions about CDH3 setup. I installed CDH3 on Ubuntu 11.0 Unix box. I want to setup a sudo cluster where I can run my pig jobs under mapreduce mode. How ...
    Shan sShan s
    Apr 15, 2012 at 9:20 pm
    Apr 16, 2012 at 4:29 am
  • Hi we are doing some benchmarking of some of our infrastructure and are using TeraGen/TeraSort to do the benchmarking. I am wondering if the data generated by TeraGen is deterministic, in that if I ...
    David EricksonDavid Erickson
    Apr 14, 2012 at 8:53 pm
    Apr 14, 2012 at 10:15 pm
  • *FYI this is a proof of concept cluster* In my two node cluster that consists of Master - Jobtracker, Datanode, Namenode, tasktracker, Secondarynamenode And Slave - Datenode , tasktraker I have no ...
    Barry, Sean FBarry, Sean F
    Apr 13, 2012 at 9:18 pm
    Apr 13, 2012 at 9:48 pm
  • I will be out of the office starting 04/02/2012 and will not return until 04/05/2012. I am out of office, and will reply you when I am back.
    Yuan JinYuan Jin
    Apr 2, 2012 at 10:10 am
    Apr 12, 2012 at 11:24 pm
  • hi all , i did all Hadoop installation and Configuration , but while Executing Word count program , Mapping is not happening not getting result getting Below Result while executing program ...
    Sujit DhamaleSujit Dhamale
    Apr 8, 2012 at 3:43 pm
    Apr 10, 2012 at 6:37 am
  • Hi all, I currently have a 2 node cluster up and running. But now I face a new issue, one of my nodes is running a Datanode and a Tasktracker on a 4 core machine and in order to do a bit of proof of ...
    Barry, Sean FBarry, Sean F
    Apr 9, 2012 at 4:23 pm
    Apr 9, 2012 at 8:27 pm
  • I am noticing something strange with JobTracker history logs on my cluster. I see configuration files (*_conf.xml) under /logs/history/ but none of the actual job logs. Anyone has ideas on what might ...
    Prashant KommireddiPrashant Kommireddi
    Apr 5, 2012 at 8:57 am
    Apr 9, 2012 at 5:51 pm
  • Hello, I'm very new to Hadoop and I am trying to carry out of proof of concept for processing some trading data. I am from a .net background, so I am trying to prove whether it can be done primarily ...
    Tom FergusonTom Ferguson
    Apr 9, 2012 at 4:40 pm
    Apr 9, 2012 at 5:28 pm
  • Hi all, my DataNode is not started . even after deleting hadoop*.pid file from /tmp , But still Data node is not started , Hadoop Version: hadoop-1.0.1.tar.gz Java version : java version "1.6.0_26 ...
    Sujit DhamaleSujit Dhamale
    Apr 6, 2012 at 6:13 pm
    Apr 6, 2012 at 6:33 pm
  • Hello, I am using BinSedesTuple as a mapper key to emit a tuple of values. But somehow same keys do not go to the same reducer and I do not get aggregates. Is it not suggested to use it as a mapper ...
    Gayatri RaoGayatri Rao
    Apr 23, 2012 at 5:31 am
    May 3, 2012 at 9:30 am
  • Here's an error I've never seen before. I rebooted my machine sometime last week, so obviously when I tried to run a hadoop job this morning, the first thing I was quickly reminded of was that the ...
    Keith WileyKeith Wiley
    Apr 30, 2012 at 5:49 pm
    Apr 30, 2012 at 6:19 pm
Group Navigation
period‹ prev | Apr 2012 | next ›
Group Overview
groupcommon-user @

135 users for April 2012

Harsh J: 57 posts Jay Vyas: 26 posts Robert Evans: 16 posts Mohit Anchlia: 13 posts Barry, Sean F: 11 posts Madhu phatak: 11 posts Bas Hickendorff: 10 posts Sujit Dhamale: 10 posts Sky USC: 9 posts Alo alt: 8 posts Edward Capriolo: 8 posts Michel Segel: 8 posts Ondřej Klimpera: 8 posts Raj Vishwanathan: 8 posts Jane Wayne: 7 posts Prashant Kommireddi: 7 posts Praveenesh kumar: 7 posts Devaraj k: 6 posts John George: 6 posts Lac Trung: 6 posts
show more