Search Discussions

129 discussions - 474 posts

  • Here is a tutorial on some handy Hadoop classes - with sample source code. http://sujee.net/tech/articles/hadoop-useful-classes/ Would appreciate any feedback / suggestions. thanks all Sujee Maniyam ...
    Sujee ManiyamSujee Maniyam
    Aug 31, 2011 at 11:58 pm
    Sep 2, 2011 at 5:54 pm
  • I'm running Hadoop with the default io.compression.codec value. When I try to do "hadoop fs -text" on a "*.deflate" file on HDFS it prints garbage to STDOUT. Presumably the file is not ...
    W.P. McNeillW.P. McNeill
    Aug 31, 2011 at 11:49 pm
    Aug 31, 2011 at 11:49 pm
  • Does map-reduce work well with binary contents in the file? This binary content is basically some CAD files and map reduce program need to read these files using some proprietry tool extract values ...
    Mohit AnchliaMohit Anchlia
    Aug 31, 2011 at 3:45 pm
    Sep 2, 2011 at 3:14 pm
  • Hadoop newbie here. I wrapped my company's entity extraction product in a Hadoop task, and give it a large file of the magnitude of 100MB. I have 4 VMs running on a 24-core CPU server, and made two ...
    Teruhiko KurosakaTeruhiko Kurosaka
    Aug 31, 2011 at 8:49 am
    Aug 31, 2011 at 11:30 pm
  • Hi ! I wanted to build and run the hadoop code itself from eclipse . I followed the instructions in http://wiki.apache.org/hadoop/EclipseEnvironment as : 1) I have checked out the hadoop release from ...
    Arun kArun k
    Aug 31, 2011 at 6:10 am
    Aug 31, 2011 at 6:10 am
  • I'm configuring a local hadoop cluster in secure mode for development/experimental purposes on Ubuntu 11.04 with the hadoop- distribution from apache mirror. I have the basic Kerberos setup ...
    Thomas WeiseThomas Weise
    Aug 31, 2011 at 1:09 am
    Aug 31, 2011 at 8:33 pm
  • Hello, i've setted up a 3 nodes cluster running Hadoop 0.21. Everything works fine . but In the web-view of the HDFS (Port 60070) I see that I have 3 LiveNodes. If I click this link, I see a listing ...
    Ralf HeydeRalf Heyde
    Aug 30, 2011 at 4:12 pm
    Aug 30, 2011 at 5:47 pm
  • Hi all, I'm running Hadoop on a single node mode in SLES 10 and everything ran smoothly for about a week until 3 days ago, where for no reason the web ui 404's for every daemon. Hadoop runs with no ...
    Joao pedro canilhoJoao pedro canilho
    Aug 30, 2011 at 12:07 pm
    Aug 30, 2011 at 12:08 pm
  • Hi All, While discussing about Hadoop backup & restore plan with my team I thought about a scenario I wanted to ask you about: I wonder what will happen following the steps below: 1. Backup the ...
    Avi VakninAvi Vaknin
    Aug 30, 2011 at 12:06 pm
    Aug 30, 2011 at 8:13 pm
  • Hi All, First, I really enjoy writing you and I'm thankful for your help. I have Oozie installed on dedicated server and I want to monitor it using my Nagios server. Can you please suggest some ...
    Avi VakninAvi Vaknin
    Aug 30, 2011 at 9:29 am
    Aug 30, 2011 at 1:10 pm
  • Ok, this is a first, but we're to see if we can do a simulcast by broadcasting the meeting live over LiveMeeting. This month's meeting: A tale of two derivatives features Ted Dunning of MapR and ...
    Michael SegelMichael Segel
    Aug 30, 2011 at 2:55 am
    Aug 30, 2011 at 3:11 am
  • Dear hadoop users, Sorry for the off-topic. We're slowly migrating our hadoop cluster to EC2, and one thing that I'm trying to explore is whether we can use alternative scheduling systems like SGE ...
    Dmitry PushkarevDmitry Pushkarev
    Aug 29, 2011 at 9:57 pm
    Aug 31, 2011 at 7:03 pm
  • Is it possible to turn off all the Hadoop logs simultaneously? In my unit tests, I don’t want to see the myriad “INFO” logs spewed out by various Hadoop components. I’m using: ((Log4JLogger) ...
    Frank AstierFrank Astier
    Aug 29, 2011 at 7:32 pm
    Sep 1, 2011 at 9:54 am
  • Hi - I’m working with Maven inside IntelliJ, using Hadoop, and I get the following error message when trying to run my own unit tests, that use MiniDFSCluster. I copied how to use ...
    Frank AstierFrank Astier
    Aug 29, 2011 at 7:01 pm
    Aug 29, 2011 at 7:01 pm
  • I installed hadoop-0.20.2 in Eucalyptus VM environment. The file system is based on glusterfs, so it is a shared NAS. Though the nodes are much powerful (8 cores + 15G memory), I found the response ...
    Shi YuShi Yu
    Aug 29, 2011 at 3:34 pm
    Sep 1, 2011 at 9:51 am
  • Hi, I am trying to run the following code sample: http://pages.cs.brandeis.edu/~cs147a/lab/hadoop-example/ I am on Windows XP, Cygwin and am using hadoop-0.20.2. I am getting stuck on an error while ...
    Aug 29, 2011 at 2:15 pm
    Aug 29, 2011 at 2:18 pm
  • Hi All, I want to install Oozie and I wonder if it is OK to install it on the name node or maybe I need to install dedicated server to it. I have a very small Hadoop cluster (4 datanodes + namenode + ...
    Avi VakninAvi Vaknin
    Aug 29, 2011 at 9:51 am
    Aug 29, 2011 at 2:10 pm
  • How do i take a machine down that is a datanode without causing the namenode to start fixing the under-replication? Say i need to fix something on the machine that will take 30 mins, and after that i ...
    Koert KuipersKoert Kuipers
    Aug 27, 2011 at 11:05 pm
    Aug 28, 2011 at 2:19 am
  • ----------------------------------------------------------------------- Senthil wants to stay in better touch using some of Google's coolest new products. If you already have Gmail or Google Talk, ...
    Aug 27, 2011 at 4:06 am
    Aug 27, 2011 at 10:07 pm
  • Hi - Is there a way I can start HDFS (the namenode) from a Java main and run unit tests against that? I need to integrate my Java/HDFS program into unit tests, and the unit test machine might not ...
    Frank AstierFrank Astier
    Aug 26, 2011 at 6:34 pm
    Aug 29, 2011 at 4:58 am
  • Using hadoop single node setup root directory is writable for everybody despite I've set it's mode to 755 and then even 000 (verified using -ls) What could be the problem? Best regards, -- ...
    Stanislaw AdaszewskiStanislaw Adaszewski
    Aug 26, 2011 at 4:13 pm
    Aug 26, 2011 at 4:56 pm
  • Hi, I know this question has been asked before, but I could not find the right solution. Maybe because I use hadoop 0.20.2, some posts assumed older versions. My code (relevant chunk): import ...
    Dieter PlaetinckDieter Plaetinck
    Aug 26, 2011 at 3:41 pm
    Aug 29, 2011 at 5:04 am
  • One of my colleagues has noticed this problem for a while, and now it's biting me. Jobs seem to be failing before every really starting. It seems to be limited (so far) to running in ...
    John ArmstrongJohn Armstrong
    Aug 26, 2011 at 2:51 pm
    Aug 26, 2011 at 8:19 pm
  • Can anyone offer me some insight. It may have been due to me trying to run the start-all.sh script instead of starting the services. Not sure. Thanks Sean ...
    Sean wagnerSean wagner
    Aug 26, 2011 at 12:49 pm
    Aug 29, 2011 at 5:29 pm
  • Hi All, The following patches in the https://issues.apache.org/jira/browse/HDFS-976 link are belongs to which Version of hadoop ? 0001-0.20.3_rc2-AvatarNode.patch AvatarNode.20.patch Thanks in ...
    Aug 26, 2011 at 9:17 am
    Aug 26, 2011 at 10:50 am
  • Hi All, I have the doubt in avatar node setup . I configure the avatarnode using the patch https://issues.apache.org/jira/browse/HDFS-976 Am I need to configure the NFS filer for share the FSimage ...
    Aug 26, 2011 at 8:27 am
    Aug 26, 2011 at 8:29 am
  • Hi guys, We are currently in the process of writing a paper regarding hadoop and we would like to reference any attempt to remove the single point of failure of the Namenode. We have found in various ...
    George KousiourisGeorge Kousiouris
    Aug 25, 2011 at 9:45 am
    Aug 25, 2011 at 6:46 pm
  • Hi everyone, We are going to create a new Hadoop cluster in our company, i have to get some advises from you: 1. Does anyone have stored whole Hadoop data not on local disks but on Netapp or other ...
    Hakan İlterHakan İlter
    Aug 25, 2011 at 6:58 am
    Sep 1, 2011 at 9:49 am
  • Hi, We are organizing a Hadoop meetup to talk about Hadoop and big data in Delhi/NCR. We have good presenters from different organizations and the topic looks very interesting. Suggestions and ...
    Shahnawaz SaifiShahnawaz Saifi
    Aug 24, 2011 at 5:19 pm
    Aug 24, 2011 at 5:19 pm
  • Does anyone have any experience using HP Proliant SL hardware for hadoop? We are currently using DL 160 and DL 180 servers, and the SL hardware seems to fit the bill for our new servers in many ways. ...
    Koert KuipersKoert Kuipers
    Aug 24, 2011 at 2:43 pm
    Aug 24, 2011 at 6:23 pm
  • Hello guys, I am looking to use MultipleInputs.addInputPath() method in hadoop 0.20.2. But when I am looking to its signature in the API, its like this : * public static void addInputPath(JobConf ...
    Praveenesh kumarPraveenesh kumar
    Aug 24, 2011 at 9:11 am
    Aug 26, 2011 at 10:43 am
  • For those of you who has the book, on page 49 there is a custom partitioner example. It basically describes a situation where the map emits <K,V , but the key is a compound key like (K1,K2), and we ...
    Mehmet TepedelenliogluMehmet Tepedelenlioglu
    Aug 23, 2011 at 11:26 pm
    Aug 24, 2011 at 12:10 am
  • Anyone had worked on Hadoop data integration with SAS? Does SAS have a connector to HDFS? Can it use data directly on HDFS? Any link or samples or tools? Thanks! Jonathan This message is for the ...
    Jonathan HwangJonathan Hwang
    Aug 23, 2011 at 9:22 pm
    Aug 23, 2011 at 9:51 pm
  • Hi, I am newbie in Python I was looking in to the Python example of running map reduce job of Michael Noll's article. I was trying to run this example in CDH3. Map tasks is running in a loop and the ...
    Jagaran dasJagaran das
    Aug 23, 2011 at 9:20 pm
    Aug 23, 2011 at 9:20 pm
  • Hi All, I have some query regarding task re-scheduling.Can it possible to make Job tracker wait for some time before re-scheduling of failed tracker's tasks. Thanks and regards, Vaibhav Pol National ...
    Vaibhav PolVaibhav Pol
    Aug 23, 2011 at 9:44 am
    Aug 23, 2011 at 2:53 pm
  • Hadoop fans, The September Hadoop meetup will be held Wednesday, September 14, from 6pm to 8pm. This meetup will be hosted by ngmoco at their office on Brannan St. As usual, we will use the ...
    Aaron KimballAaron Kimball
    Aug 22, 2011 at 9:35 pm
    Aug 22, 2011 at 9:35 pm
  • Hadoop developers, Arun will be cutting a branch for Hadoop 0.23 as soon the trunk has a successful build. I'd like Hoop (https://issues.apache.org/jira/browse/HDFS-2178) to be part of 0.23 (Nicholas ...
    Alejandro AbdelnurAlejandro Abdelnur
    Aug 22, 2011 at 9:18 pm
    Aug 22, 2011 at 11:10 pm
  • What does HADOOP_CLASSPATH set in $HADOOP/conf/hadoop-env.sh do? This isn't clear to me from documentation and books, so I did some experimenting. Here's the conclusion I came to: the paths in ...
    W.P. McNeillW.P. McNeill
    Aug 22, 2011 at 6:01 pm
    Aug 22, 2011 at 6:49 pm
  • Hi. I'm running a 2 Node Hadoop (v 20.203). I'm trying to do a distributed image processing. And now, I'm facing an issue, The job it's a Hadoop pipes job, with custom RecordReader, Maper and Reducer ...
    Xian BoullosaXian Boullosa
    Aug 22, 2011 at 2:23 pm
    Aug 22, 2011 at 3:48 pm
  • Hi all, I would like to force locality for the tasks in hadoop using FairScheduler and schedule the tasks locally than on non-local nodes. I am using CDH3 and tried setting the property in ...
    Aug 22, 2011 at 12:43 pm
    Aug 22, 2011 at 12:43 pm
  • Hey to everyone! ;) Anyone knows, how do I pass null or empty -param through to pig script? Following *does not* work: /pig/bin/pig -param *rootPath=* MapProfile.pig /pig/bin/pig -param *rootPath=''* ...
    Aug 22, 2011 at 12:41 pm
    Aug 22, 2011 at 12:41 pm
  • 发送自 HTC ----- Reply message ----- 发件人: "lulynn_2008" <lulynn_2008@163.com 收件人: "user@pig.apache.org" <user@pig.apache.org , "common-user@hadoop.apache.org" <common-user@hadoop.apache.org 主题: Can ...
    Aug 22, 2011 at 12:02 pm
    Aug 22, 2011 at 7:46 pm
  • Hi I just upgraded from 0.19 to 0.20 everything seems fine however the web monitoring tool doesn't work any more: neither http://mydomain.com:50070/webapps/hdfs/dfshealth.jsp nore ...
    Aug 21, 2011 at 6:04 pm
    Aug 21, 2011 at 6:04 pm
  • Hi all ! How are you? My name is Avi and I have been fascinated by Apache Hadoop for the last few months. I am spending the last two weeks trying to optimize my configuration files and environment. I ...
    Avi VakninAvi Vaknin
    Aug 21, 2011 at 11:58 am
    Aug 22, 2011 at 4:19 pm
  • http://on.fb.me/p7Og4F -- Thanks, Shah
    Shahnawaz SaifiShahnawaz Saifi
    Aug 20, 2011 at 2:49 pm
    Aug 20, 2011 at 2:49 pm
  • I would like my streaming job to receive the names of files stored on HDFS, but not the actual contents of the files, and I would like data locality to be honored (I want mappers to run on nodes ...
    Keith WileyKeith Wiley
    Aug 19, 2011 at 9:46 pm
    Aug 19, 2011 at 9:46 pm
  • Hi all, I've tried to make a rack topology script. I've written it in python and it works if I call it with the following arguments: ...
    Aug 19, 2011 at 4:46 pm
    Aug 21, 2011 at 11:21 pm
  • Hello all, I have a Hadoop related application that is integrated with HDFS and is started via the command line with "hadoop jar ..." The amount of data used by the application changes from use case ...
    Adam ShookAdam Shook
    Aug 19, 2011 at 3:35 pm
    Aug 19, 2011 at 4:20 pm
  • I updated my hadoop cluster from 0.20.2 to higher version 0.21.0 because of MAPREDUCE-1286, and now I have problem running a Hbase on it. I saw the 0.21.0 version is marked as "unstable, unsupported, ...
    Steven zhuangSteven zhuang
    Aug 19, 2011 at 7:40 am
    Aug 22, 2011 at 4:21 pm
  • Hi, http://wiki.apache.org/hadoop/HowToContribute is a great resource detailing the steps needed to build jars and tars from the source code. However, I am still not sure what the best way to run ...
    Ravi PrakashRavi Prakash
    Aug 18, 2011 at 9:19 pm
    Aug 23, 2011 at 12:32 am
Group Navigation
period‹ prev | Aug 2011 | next ›
Group Overview
groupcommon-user @

165 users for August 2011

Harsh J: 36 posts A Df: 14 posts W.P. McNeill: 14 posts Allen Wittenauer: 13 posts Jagaran das: 12 posts John Armstrong: 12 posts Daniel,Wu: 10 posts Michel Segel: 10 posts Shanmuganathan.r: 10 posts אבי ווקנין: 10 posts GOEKE, MATTHEW (AG/1000): 9 posts Joey Echeverria: 9 posts Steve Loughran: 9 posts Keith Wiley: 8 posts Stan Rosenberg: 8 posts Kai Voigt: 7 posts Shahnawaz Saifi: 7 posts Jianxin Wang: 6 posts Madhu phatak: 6 posts Adi: 5 posts
show more