Search Discussions
-
Hi guys, I have some quick questions regarding to Hadoop counter, - Hadoop counter (customer defined) is global accessible (for both read and write) for all Mappers and Reducers in a job? - What is ...
Lin Ma
Oct 19, 2012 at 10:51 am
Oct 23, 2012 at 7:13 am -
Hi I have read scattered documentation across the net which mostly say HDFS doesn't go well with SAN being used to store data. While some say, it is an emerging trend. I would love to know if there ...
Pamecha, Abhishek
Oct 16, 2012 at 8:14 pm
Oct 19, 2012 at 8:07 am -
Hi, I am a new Hadoop user, and would really appreciate your opinions on whether Hadoop is the right tool for what I'm thinking of using it for. I am investigating options for scaling an archive of ...
Matt Painter
Oct 15, 2012 at 7:48 pm
Oct 18, 2012 at 12:22 pm -
Hi, I have a complex Hadoop job that iterates over large graph data multiple times until some convergence condition is met. I know that the map output goes to the local disk of each particular mapper ...
Jim Twensky
Oct 5, 2012 at 4:31 pm
Oct 8, 2012 at 11:03 pm -
Hi, I had made Hadoop setup & set all the configuration files. when go to this url through browser http://localhost:50070, it opened well but when i clicked on BrowseTheFileSystem , its redirecting ...
Murthy nvvs
Oct 11, 2012 at 4:54 am
Oct 18, 2012 at 7:49 am -
Is that anyway to control who can submit job to a pool.? Eg. Pool1, can run jobs submitted from any users except userx. Userx can submit jobs to poolx only. Can't submit to pool1. Hope this make ...
Patai Sangbutsarakum
Oct 14, 2012 at 12:33 am
Oct 18, 2012 at 10:12 am -
Hello Hadoopers, I was reading the hardware recommendation doc. from Cloudera/HP, and this is one of the recommendation about CPU. "To remove the bottleneck for CPU bound workloads, for the best ...
Patai Sangbutsarakum
Oct 11, 2012 at 4:23 pm
Oct 13, 2012 at 7:22 am -
Hi, I have a file which has some financial transaction data. Each transaction will have amount and a credit/debit indicator. I want to write a mapreduce program which computes cumulative credit & ...
Sarath
Oct 4, 2012 at 1:59 pm
Oct 19, 2012 at 6:03 am -
Hello, can we create symlinks within hadoop is ther any shell commands or can we do it thru java....
Visioner Sadak
Oct 8, 2012 at 6:44 am
Oct 9, 2012 at 1:24 pm -
I am running Hadoop 1.0.3 in Pseudo distributed mode. When I submit a map/reduce job to process a file of size about 16 GB, in job.xml, I have the following mapred.map.tasks =242 ...
Shing Hing Man
Oct 2, 2012 at 4:35 pm
Oct 3, 2012 at 1:50 pm -
Hi, Does anyone know any (opensource) project that builds a rules engine (based on RETE) on top Hadoop? Searching a bit on the net, I have only seen a small reference to Concord/IBM but there is ...
Luangsay Sourygna
Oct 19, 2012 at 7:25 pm
Oct 21, 2012 at 1:50 pm -
Hi, Imagine I have a very fast hard drive that I want to use for the NameNode. That is, I want the NameNode to store its blocks information on this hard drive instead of in memory. Why would I do it? ...
Mark Kerzner
Oct 12, 2012 at 4:00 am
Oct 17, 2012 at 11:28 pm -
Hi, I have an issues with hadoop dfs, I have 3 servers (24Gb RAM on each). The servers are not overloaded, they just have hadoop installed. One have datanode and namenode, second - datanode only, ...
Alexey
Oct 9, 2012 at 10:13 am
Oct 16, 2012 at 2:39 am -
I have read around about the hardware recommendation for hadoop cluster. One of them is recommend 1:1 ratio between spindle per core. Intel CPU come with Hyperthread which will double the number ...
Patai Sangbutsarakum
Oct 12, 2012 at 5:46 pm
Oct 13, 2012 at 5:53 am -
Hi, if I create a Lucene index in each mapper, locally, then copy them to under /jobid/mapid1, /jodid/mapid2, and then in the reducers copy them to some Solr machine (perhaps even merging), does such ...
Mark Kerzner
Oct 10, 2012 at 2:48 am
Oct 12, 2012 at 5:24 am -
Hello all, I'm looking for a reference architecture for hadoop. The only result I found is Lambda architecture from Nathan Marz[0]. With architecture I mean answers to question like: - How should I ...
Daniel Käfer
Oct 25, 2012 at 7:24 pm
Oct 29, 2012 at 11:26 pm -
How can we manage cluster-wide atomic operations? Such as maintaining an auto-increment counter. Does Hadoop provide native support for these kinds of operations? An in case ultimate answer involves ...
David Parks
Oct 27, 2012 at 3:08 am
Oct 29, 2012 at 10:16 am -
To all, I have a few questions regarding YARN (with respect to Hadoop): Are YARN and Hadoop separate, or is YARN the successor to Hadoop? What are the major conceptual differences between YARN and ...
Tom Brown
Oct 18, 2012 at 8:33 pm
Oct 24, 2012 at 4:32 am -
Hi Users, I am trying to install Hadoop 0.20.2 on a cluster on two virtual machines. One acting as master other as slave. I am able to ssh from master to slave and vice verse. But when I run ...
Sundeep Kambhmapati
Oct 17, 2012 at 9:05 pm
Oct 22, 2012 at 3:20 am -
Hi All, Had anyone tried installing Hadoop on mac pc..if yes can u please share the installation steps.. Thanks in advance.. Thanks, Suneel Sent from my iphone
Suneel hadoop
Oct 16, 2012 at 10:51 am
Oct 16, 2012 at 4:08 pm -
Hi all, I installed Hadoop HDFS on 3 nodes, a namenode and 2 datanodes, when i want to start dfs processes, Only secondaryNameNode is launched but the namenode datanodes processes doesn't work there ...
Khaled Ben Bahri
Oct 16, 2012 at 9:19 am
Oct 16, 2012 at 10:04 am -
I would like to be able to resize a set of inputs, already in SequenceFile format, to be larger. I have tried 'hadoop distcp -Ddfs.block.size=$[64*1024*1024]' and did not get what I expected. The ...
Anna Lahoud
Oct 1, 2012 at 6:31 pm
Oct 9, 2012 at 4:28 pm -
Hi, Can anyone give the clear idea about these comparisons on same hardware & software configuration. Sql server hbase hadoop+hbase data compression ? ? ? (yes/no,if all yes where it is more ...
Iwannaplay games
Oct 18, 2012 at 10:49 am
Oct 26, 2012 at 11:07 am -
Hi all, I am trying to run the command ssh Master it runs and shows after entering password. Password: abc Last login: Thu Oct 25 13:51:06 2012 from master But ssh for Slave through error. ssh Slave ...
Yogesh Kumar13
Oct 25, 2012 at 8:29 am
Oct 25, 2012 at 5:37 pm -
Hi Hadoopers, I have <property <name dfs.replication</name <value 2</value <final true</final </property set in hdfs-site.xml in staging environment cluster. while the staging cluster is running the ...
Patai Sangbutsarakum
Oct 15, 2012 at 7:02 pm
Oct 16, 2012 at 7:28 am -
Hi , i install Hadoop on window with the help of Cygwin . *data node and Task tracker is not starting .* can some one help me in this ? i added Log File with this mail . is any one have ...
Sujit Dhamale
Oct 9, 2012 at 5:31 am
Oct 11, 2012 at 9:55 am -
hi all When I run "hadoop job -status xxx",Output the following some list. Rack-local map tasks=124 Data-local map tasks=6 What is the difference between Rack-local map tasks and Data-local map ...
Centerqi hu
Oct 7, 2012 at 1:57 pm
Oct 8, 2012 at 5:45 am -
I am working on some samples where I want to write to HDFS running on another machine (different OS etc.) The identity of my client process is just whatever my OS says it is (e.g., 'oleg') hence ...
Oleg Zhurakousky
Oct 5, 2012 at 1:20 pm
Oct 5, 2012 at 11:58 pm -
I want to use HDFS HA function, I find the IO Fencing function is complex in hadoop2.0. I think we can use file lock to implement the IO Fencing function, I think that is simple. Thanks, LiuLei
Lei liu
Oct 25, 2012 at 8:28 am
Oct 27, 2012 at 5:35 pm -
I want to create a MapReduce job which reads many multi-gigabyte input files from various HTTP sources & processes them nightly. Is there a reasonably flexible way to do this in the Hadoop job its ...
David Parks
Oct 22, 2012 at 8:41 am
Oct 24, 2012 at 4:25 pm -
Anyone using Hadoop running on Isilon NAS? I am trying to submit a job with a user other than the one running Hadoop and I'm getting the following error: Exception in thread "main" ...
Artem Ervits
Oct 17, 2012 at 7:20 pm
Oct 18, 2012 at 3:25 pm -
Hi all, a very strange thing is happening with my hadoop program. My map simply emits tuples with a custom object as key (which implement WritableComparable). The object is made of 2 fields, and I ...
Alberto Cordioli
Oct 15, 2012 at 7:12 pm
Oct 16, 2012 at 6:45 pm -
Hello, I am trying to run hadoop on s3 using distributed mode. However I am having issues running my job successfully on it. I get the following error I followed the instructions provided in this ...
Parth Savani
Oct 15, 2012 at 5:13 pm
Oct 16, 2012 at 3:11 pm -
I am bringing up a Hadoop cluster for the first time (but am an experienced sysadmin with lots of cluster experience) and running into an issue with permissions on mapred.system.dir. It has generally ...
Goldstone, Robin J.
Oct 9, 2012 at 11:45 pm
Oct 14, 2012 at 8:47 am -
Hi, all I'm modifying the source code of hadoop, specifically, source code of FairScheduler In last weeks, no problem during my development, but last night, when i add some code on ...
Nan Zhu
Oct 13, 2012 at 3:37 pm
Oct 13, 2012 at 4:36 pm -
Guys, have been stretching my head for the past couple of days. Why are my tags duplicated while the content they wrap around i.e.my StringBuilder sb is not? My Reduce code is: while ...
Kartashov, Andy
Oct 2, 2012 at 1:32 pm
Oct 2, 2012 at 7:54 pm -
Hello friends, I m runnning hadoop in psuedo distr mode at a remote linux ip ,Thru the namenode WEB UI i m able to access the directory structure(thru data node) by clicking on browse the directory ...
Visioner Sadak
Oct 19, 2012 at 6:28 am
Oct 31, 2012 at 12:05 am -
Hi, on our cluster our jobs usually satisfied with less than 2 GB of heap space. so we have on our 8 GB computers 3 maps maximum and on our 16 GB computers 4 maps maximum (we only have quad core CPUs ...
Marco Zühlke
Oct 30, 2012 at 3:50 pm
Oct 30, 2012 at 5:51 pm -
Hi All, I run this command hadoop fsck -Ddfs.http.address=localhost:50070 / and found that some blocks are missing and corrupted results comes like.. /user/hive/warehouse/tt_report_htcount/000000_0 ...
Yogesh Kumar13
Oct 29, 2012 at 8:34 am
Oct 29, 2012 at 12:31 pm -
Hi All, I am trying to copy the public key by this command. Master:~ mediaadmin$ ssh-copy -id -i $HOME/.ssh/id_rsa.pub [email protected] I have two machines Master Name is pluto and same name is of Slave ...
Yogesh Kumar13
Oct 25, 2012 at 9:55 am
Oct 25, 2012 at 8:38 pm -
Hi, I try to install cdh free manager 4 but it always failed on java installation section. So I tried to install oracle jdk1.6.0_31 first and tried to install the cdh free manager 4 again, but it's ...
Martinus Martinus
Oct 12, 2012 at 7:03 am
Oct 14, 2012 at 6:50 pm -
Hi all, I have understood the Hadoop and Hadoop Ecosystem(Pig as ETL, Hive as DataWare house, Sqoop as importing tool). I worked and learned on single node cluster with demo data. As Hadoop suits ...
Yogesh dhari
Oct 1, 2012 at 1:37 pm
Oct 3, 2012 at 3:02 am -
Hi, i am kind of unsure where to post this problem, but i think it is more related to hadoop than to pig. By successfully executing a pig script i created a new file in my hdfs. Sadly though, i ...
Björn-Elmar Macek
Oct 1, 2012 at 4:12 pm
Oct 3, 2012 at 12:26 am -
Hi list, Are the any tools for parsing and extracting data from Hadoop's Job Logs? I want to do stuff like .. 1. Getting run time of each map/reduce task 2. Total map/reduce tasks ran on a particular ...
Bharath vissapragada
Oct 30, 2012 at 1:49 am
Oct 30, 2012 at 5:22 am -
Hi all, I am using last stable Hadoop version (1.0.3) and I am implementing right now my first MR jobs. I read about the presence of 2 API: the old and the new one. I read some stuff about them, but ...
Alberto Cordioli
Oct 22, 2012 at 1:23 pm
Oct 25, 2012 at 3:12 pm -
With secure hadoop the user name is authenticated by the kerberos server. But what about the groups that the user is a member of? Are these simple the groups that the user is a member of on the ...
Koert Kuipers
Oct 8, 2012 at 9:01 pm
Oct 19, 2012 at 2:20 am -
Hi all, I'm experimenting with hadoop streaming on build 1.0.3. To give background info, i'm streaming a text file into mapper written in C. Using the default settings, streaming uses TextInputFormat ...
Jason Wang
Oct 18, 2012 at 4:03 am
Oct 18, 2012 at 8:03 pm -
have configured webhdfs in hdfs-site.xml as <property <name dfs.webhdfs.enabled</name <value true</value </property but unable to access my file at ...
Visioner Sadak
Oct 17, 2012 at 9:08 am
Oct 18, 2012 at 11:53 am -
I have some encrypted data in an HDFS csv, that I've created a Hive table for, and I want to run a Hive query that first encrypts the query param, then does the lookup. I have a UDF that does ...
Sam Mohamed
Oct 18, 2012 at 1:04 am
Oct 18, 2012 at 3:21 am -
Hello I'm trying to test the Hadoop archive functionality under 0.23 and I can't get it working. I have in HDFS a /test folder with several text files. I created a hadoop archive using hadoop archive ...
Alexander Hristov
Oct 2, 2012 at 6:12 am
Oct 12, 2012 at 5:59 am
Group Overview
| group | hdfs-user |
| categories | hadoop |
| discussions | 196 |
| posts | 861 |
| users | 207 |
| website | hadoop.apache.org... |
| irc | #hadoop |
207 users for October 2012
Archives
- February 2013 (245)
- January 2013 (838)
- December 2012 (590)
- November 2012 (723)
- October 2012 (861)
- September 2012 (710)
- August 2012 (1,046)
- July 2012 (151)
- June 2012 (91)
- May 2012 (126)
- April 2012 (95)
- March 2012 (64)
- February 2012 (128)
- January 2012 (258)
- December 2011 (110)
- November 2011 (164)
- October 2011 (83)
- September 2011 (101)
- August 2011 (58)
- July 2011 (73)
- June 2011 (101)
- May 2011 (184)
- April 2011 (51)
- March 2011 (110)
- February 2011 (100)
- January 2011 (101)
- December 2010 (44)
- November 2010 (49)
- October 2010 (48)
- September 2010 (26)
- August 2010 (52)
- July 2010 (50)
- June 2010 (64)
- May 2010 (57)
- April 2010 (45)
- March 2010 (38)
- February 2010 (10)
- January 2010 (84)
- December 2009 (3)
- November 2009 (38)
- October 2009 (43)
- September 2009 (32)
- August 2009 (35)
- July 2009 (5)