FAQ

Search Discussions

132 discussions - 492 posts

  • I've tried what it shows in the examples, but those don't seem to be working. Aside from that, they also complain about deprecated interface when I compile. Any help you guys can give would be ...
    David HawthorneDavid Hawthorne
    Feb 10, 2010 at 7:09 pm
    Feb 13, 2010 at 5:42 pm
  • Hi. Just wondering - does anyone know what framework Hadoop uses for daemonizing? Any chance it's jsvc from Apache? Regards.
    Stas OskinStas Oskin
    Feb 4, 2010 at 9:04 pm
    Feb 12, 2010 at 7:29 pm
  • Ye currently am using jobclient to read these counters. But We are not able to use *webservices *because the jar which is used to read the counters from running hadoop job is itself a Hadoop program ...
    Mark NMark N
    Feb 4, 2010 at 11:00 am
    Feb 24, 2010 at 2:49 am
  • Hey all, Just a note that you should avoid upgrading your clusters to 1.6.0u18. We've seen a lot of segfaults or bus errors on the DN when running with this JVM - Stack found the ame thing on one of ...
    Todd LipconTodd Lipcon
    Feb 16, 2010 at 5:55 am
    Mar 2, 2010 at 10:24 am
  • I was (seemingly) able to build the native libraries, but still get the "unable to load native-hadoop library" message at run-time (with a simple app of my own to read/write a gzip-compressed ...
    Derek BrownDerek Brown
    Feb 22, 2010 at 8:52 pm
    Feb 23, 2010 at 7:14 pm
  • Can anybody point me how to use JNI calls in a map reduce program. My .so files have other dependencies also , is there a way to load the LD_LIBRARY_PATH for child processes . Should all the native ...
    Utkarsh AgarwalUtkarsh Agarwal
    Feb 12, 2010 at 5:12 pm
    Feb 19, 2010 at 6:24 pm
  • Hi, I was wondering if it is possible to write each key-value pair produced by the reduce function to a different file. How could I open a new file in the reduce function of the reducer? I know its ...
    Udaya LakshmiUdaya Lakshmi
    Feb 5, 2010 at 5:11 am
    Feb 24, 2010 at 3:49 pm
  • Hi there, can anybody help me out on a (most likely) simple unclarity. I am wondering how intermediate key/value pairs are materialized. I have a job where the map phase produces 600,000 records and ...
    Tim KieferTim Kiefer
    Feb 23, 2010 at 11:45 am
    Feb 24, 2010 at 8:45 am
  • Hello Everyone, We often find many child processes in datanodes, which have already finished for long time. And following are the jstack log: Full thread dump Java HotSpot(TM) 64-Bit Server VM ...
    Zheng LvZheng Lv
    Feb 11, 2010 at 3:59 am
    Feb 24, 2010 at 2:47 am
  • Sometimes for the same task I see that a duplicate task gets run on a different machine and gets killed later. Not always but sometimes. Any reason why duplicate tasks get run. I thought tasks are ...
    Prasenjit mukherjeePrasenjit mukherjee
    Feb 9, 2010 at 2:04 pm
    Feb 11, 2010 at 6:38 am
  • 8

    aws

    Hi, Can anybody tell me if there aws/amazon has any kind of hadoop sandbox to play in for free? Thanks Brian
    Brian WolfBrian Wolf
    Feb 3, 2010 at 7:09 am
    Feb 8, 2010 at 10:08 am
  • I am trying to run the wordcount example in c/c++ given on http://wiki.apache.org/hadoop/C%2B%2BWordCount with Hadoop 0.18.3 but when I run Ant using the specified command "ant -Dcompile.c++=yes ...
    Ratner, Alan S (IS)Ratner, Alan S (IS)
    Feb 24, 2010 at 4:03 pm
    Mar 1, 2010 at 5:17 pm
  • Hi, yesterday I read the documentation of zookeeper and the zk contrib bookkeeper. From what I read, I thought, that bookkeeper would be the ideal enhancement for the namenode, to make it distributed ...
    Thomas KochThomas Koch
    Feb 19, 2010 at 8:41 am
    Feb 23, 2010 at 11:03 am
  • -- View this message in context: http://old.nabble.com/Developing-cross-component-patches-post-split-tp27634796p27634796.html Sent from the Hadoop core-user mailing list archive at Nabble.com.
    TiruTiru
    Feb 18, 2010 at 6:30 am
    Feb 23, 2010 at 3:48 am
  • I am running a hadoop job that combines daily results with results with previous days. The reduce output is lzo compressed and growing daily in size. - DistributedLzoIndexer is used to index lzo ...
    Steve KuoSteve Kuo
    Feb 14, 2010 at 8:11 pm
    Feb 22, 2010 at 7:10 pm
  • Hi, I have a question about HEARTBEAT_INTERVAL. Why does the default HEARTBEAT_INTERVAL value is 3 rather than 2 or 1? any resources? Thanks. Shen
    ChingShenChingShen
    Feb 9, 2010 at 4:52 pm
    Feb 10, 2010 at 6:36 am
  • Im using maven to run all my unit tests, and i have a unit test that creates a mini mr cluster. When i create this cluster, i get classdefnotfound errors for the core hadoop libs (Caused by: ...
    Michael BasnightMichael Basnight
    Feb 3, 2010 at 9:09 pm
    Feb 4, 2010 at 6:33 pm
  • i have read about basic stuff about hadoop..err i have a few doubts...mind u am a begginer 1:so is hadoop a file sytem only? 2:can hbase be used instead of other databases in other platforms(eg ...
    NijilNijil
    Feb 2, 2010 at 4:43 pm
    Feb 3, 2010 at 5:12 pm
  • Reducer starts to execute starts to execute before all Maps are finished and close method is called. Locally this programs works fine as only reducer is present However on hadoop-0.18.3 clustered ...
    ANKITBHATNAGARANKITBHATNAGAR
    Feb 20, 2010 at 12:51 am
    Jun 21, 2010 at 3:01 am
  • Hi. We run hadoop-0.18.3 and it seems that the jobcache does not get cleaned out properly. Would this cron script be to any harm to hadoop ? # Clean all files which are two or more days old ...
    Marcus HerouMarcus Herou
    Feb 10, 2010 at 8:16 am
    Mar 9, 2010 at 4:51 pm
  • Just wanted to get the groups general feelings on what the preferred distro is and why? Obviously assuming one didn't have a service agreement with cloudera. Ananth T Sarathy
    Ananth SarathyAnanth Sarathy
    Feb 23, 2010 at 11:13 pm
    Feb 24, 2010 at 6:12 pm
  • Hi all, I'm getting this error [hadoop@master01 hadoop-0.20.1 ]$ ./bin/hadoop jar hadoop-0.20.1-examples.jar pi 1 1 Number of Maps = 1 Samples per Map = 1 Wrote input for Map #0 Starting Job ...
    Edson RamiroEdson Ramiro
    Feb 22, 2010 at 1:18 pm
    Feb 24, 2010 at 12:46 pm
  • I have a pig script as follows (see far below). It loads 2 data sets, perform some filtering, then join the two sets. Lastly count occurrences of a combination of fields and writes results to hdfs. ...
    Jiang lichtJiang licht
    Feb 21, 2010 at 12:42 am
    Feb 22, 2010 at 8:37 am
  • Hey all, I'm trying to get Hadoop up and running as a proof of concept to make an argument for moving away from a big RDBMS. I'm having some challenges just getting a really simple demo mapreduce to ...
    Cory BergCory Berg
    Feb 18, 2010 at 5:08 pm
    Feb 18, 2010 at 11:41 pm
  • Hi Folks Currently we use distCp to transfer files between two hadoop clusters. I have a perl script which calls a system command “hadoop distcp....” to achieve this. Is there a Java Api to do ...
    Balu VellankiBalu Vellanki
    Feb 18, 2010 at 6:42 am
    Feb 18, 2010 at 6:40 pm
  • New to Hadoop (now using 0.20.1), I want to do the following: Automatic status check and notification of hadoop jobs such that e.g. when a job is finished, a script can be trigged so that job results ...
    Jiang lichtJiang licht
    Feb 17, 2010 at 5:32 am
    Feb 17, 2010 at 7:01 pm
  • I'm using streaming hadoop, installed vua cloudera on ec2. My job should be straightforward: 1) Map task, emits 2 keys and 1 VALUE <WORD <FLAG, 0 or 1 <TEXT eg AA 0 QUICK BROWN FOX AA 1 QUICK BROWN ...
    Winton DaviesWinton Davies
    Feb 10, 2010 at 11:39 pm
    Feb 11, 2010 at 5:46 am
  • Hi everyone, I am new to mapReduce. I am trying to run a very basic mapReduce application. I encountered the following problem. Can someone help me about it: 1) I have 3 files, namely ...
    Prateek JindalPrateek Jindal
    Feb 5, 2010 at 10:59 pm
    Feb 6, 2010 at 11:07 pm
  • I've been trying to run a fairly small input file (300MB) on Cloudera Hadoop 0.20.1. The job I'm using probably writes to on the order of over 1000 part-files at once, across the whole grid. The grid ...
    Meng MaoMeng Mao
    Feb 3, 2010 at 12:30 am
    Feb 5, 2010 at 10:19 pm
  • I am running a hadoop job written in PIG. It fails from out of memory because a UDF function consumes a lot of memory, it loads a big file. What are the settings to avoid the following ...
    Jiang lichtJiang licht
    Feb 23, 2010 at 1:44 am
    Feb 23, 2010 at 5:31 pm
  • I have a pig script. If I don't set any codec for Map output for hadoop cluster, no problem. Now I made the following compression settings, the job failed and the error message is shown below. I ...
    Jiang lichtJiang licht
    Feb 23, 2010 at 2:47 am
    Feb 23, 2010 at 5:29 pm
  • Hi, Hadoop/HDFS newbie. Been struggling with getting the streaming example working with -archives. c.f. ...
    Michael KintzerMichael Kintzer
    Feb 19, 2010 at 10:36 pm
    Feb 22, 2010 at 6:07 pm
  • Hi, I was working on a scenario where in I am generating a file in close() function of my Map implementation. Since Map execution are worked concurrently, this file is overwritten. I was wondering ...
    ANKITBHATNAGARANKITBHATNAGAR
    Feb 13, 2010 at 2:34 pm
    Feb 18, 2010 at 7:51 pm
  • Hi, I've tried posting this to Cloudera's community support site, but the community website getsatisfaction.com returns various server errors at the moment. I believe the following is an issue ...
    Dan StarrDan Starr
    Feb 18, 2010 at 3:46 am
    Feb 18, 2010 at 5:59 am
  • Is it possible to access a MiniDFSCluster via an hdfs:// URL? I ask because it seems to not work...
    Jason RutherglenJason Rutherglen
    Feb 17, 2010 at 1:30 am
    Feb 18, 2010 at 1:32 am
  • New to Hadoop (now using 0.20.1), I want to know how to choose and set up compression methods for Map output, especially how to configure and use LZO compression? Specifically, please share your ...
    Jiang lichtJiang licht
    Feb 17, 2010 at 5:27 am
    Feb 17, 2010 at 9:02 pm
  • Hi, do you know and would maybe like to recommend hosting providers which have hadoop-friendly offerings? I'm new to hadoop, but what I've read is: - virtualization makes no sense and only costs - no ...
    Thomas KochThomas Koch
    Feb 16, 2010 at 8:29 am
    Feb 16, 2010 at 6:59 pm
  • Hi all, I m trying to Write a program that performs some simple Datamining on a certain DataSet. I was told that an Identity Reducer should be written. public class Reduce extends MapReduceBase ...
    Prabhu Hari DhanapalPrabhu Hari Dhanapal
    Feb 12, 2010 at 1:11 am
    Feb 12, 2010 at 6:43 am
  • Hi, I am trying to submit many independent jobs in paralllel (same user). This works for up to 16 jobs, but after that I only get 16 jobs in parallel no matter how many I try to submit. I am using ...
    Vasilis LiaskovitisVasilis Liaskovitis
    Feb 8, 2010 at 7:42 pm
    Feb 10, 2010 at 12:57 am
  • Hello, I have a question about mapred.Child processes. Even though a mapper is finished I see that the process (from ps) stays around longer than reported on the hadoop MR webpage. What is the mapper ...
    Navraj S. ChohanNavraj S. Chohan
    Feb 4, 2010 at 7:52 pm
    Feb 8, 2010 at 6:24 pm
  • Dear all, I want to build a Hadoop cluster, when I finish the hadoop installation, it seems work. However, after starting the DFS, the datanode in slave server will shut down with the following error ...
    GillggxGillggx
    Feb 5, 2010 at 4:44 pm
    Feb 6, 2010 at 2:11 pm
  • Dear All, Please help with the NullPointerException in the WordCount example. Sorry it’s the simple code because I am new to hadoop. :-) I am running v0.20.1 in Ubuntu 9.10. The map task works ...
    Frank DuFrank Du
    Feb 3, 2010 at 8:35 pm
    Feb 4, 2010 at 11:52 pm
  • Hi all: I need to set up a hadoop cluster. The cluster is based on CentOS 5.4, and I already have all the base OSes installed. I saw that Cloudera had a repo for hadoop CentOS, so I set up that repo, ...
    Jim KusznirJim Kusznir
    Feb 3, 2010 at 7:08 pm
    Feb 4, 2010 at 12:23 am
  • Hi, As a newbie to hadoop, I am not able to figure out how to use DistributedCache class. Can someone give me a small code which distributes file to the cluster and the show how to open and use the ...
    Udaya LakshmiUdaya Lakshmi
    Feb 3, 2010 at 1:31 pm
    Feb 3, 2010 at 7:27 pm
  • Hi I'm a beginner in working with hadoop. I want to know if we have to physically connect the machines using LAN cable before setting up the cluster. Urgently needed to clarify this and start my ...
    Janani venkatJanani venkat
    Feb 3, 2010 at 1:51 pm
    Feb 3, 2010 at 6:01 pm
  • Hi, there's an attempt, to get hadoop into the Debian Linux distribution. For now, this is more a pre-announce, since the package still has to pass some review. But you may already want to add your ...
    Thomas KochThomas Koch
    Feb 2, 2010 at 4:51 pm
    Feb 3, 2010 at 12:44 pm
  • Hi, We are using the streaming API. We are trying to understand what hadoop uses as a threshold or trigger to involve more TaskTracker nodes in a given Map-Reduce execution. With default settings ...
    Michael KintzerMichael Kintzer
    Feb 25, 2010 at 5:46 pm
    Mar 1, 2010 at 6:09 pm
  • While running example programe ('hadoop jar *example*jar pi 2 2'), I encounter 'Network is unreachable' problem (at $HADOOP_HOME/logs/userlogs/.../stderr), as below: Exception in thread "main" ...
    Neo andersonNeo anderson
    Feb 24, 2010 at 5:17 pm
    Feb 25, 2010 at 2:40 pm
  • Hadoop is great. Almost every day I live gives me more reasons to like it. My story for today: We have a system running a file system with a 48 TB Disk array on 4 shelves. Today I got this ...
    Edward CaprioloEdward Capriolo
    Feb 19, 2010 at 7:08 pm
    Feb 22, 2010 at 3:13 pm
  • Hi all, I recently have me t a problem that sometimes, reducer hang up at pending state, with 0% complete. It seems all the mappers are completely done, and when it just about to start the reducer, ...
    Song LiuSong Liu
    Feb 16, 2010 at 11:51 pm
    Feb 17, 2010 at 1:16 pm
Group Navigation
period‹ prev | Feb 2010 | next ›
Group Overview
groupcommon-user @
categorieshadoop
discussions132
posts492
users149
websitehadoop.apache.org...
irc#hadoop

149 users for February 2010

Todd Lipcon: 26 posts Jiang licht: 17 posts Allen Wittenauer: 15 posts E. Sammer: 14 posts Jeff Zhang: 14 posts Steve Loughran: 14 posts Edward Capriolo: 13 posts Meng Mao: 13 posts Amogh Vasekar: 12 posts Thomas Koch: 9 posts Gang Luo: 8 posts Prasenjit mukherjee: 8 posts Ted Yu: 8 posts Udaya Lakshmi: 8 posts Alex Kozlov: 7 posts Amareshwari Sri Ramadasu: 7 posts ANKITBHATNAGAR: 7 posts Brian Wolf: 7 posts Edson Ramiro: 7 posts Mark N: 7 posts
show more