FAQ

Search Discussions

212 discussions - 955 posts

  • Hi guys, I have some quick questions regarding to Hadoop counter, - Hadoop counter (customer defined) is global accessible (for both read and write) for all Mappers and Reducers in a job? - What is ...
    Lin MaLin Ma
    Oct 19, 2012 at 10:51 am
    Oct 23, 2012 at 7:13 am
  • Hi I have read scattered documentation across the net which mostly say HDFS doesn't go well with SAN being used to store data. While some say, it is an emerging trend. I would love to know if there ...
    Pamecha, AbhishekPamecha, Abhishek
    Oct 16, 2012 at 8:14 pm
    Oct 19, 2012 at 8:07 am
  • Hi, I am a new Hadoop user, and would really appreciate your opinions on whether Hadoop is the right tool for what I'm thinking of using it for. I am investigating options for scaling an archive of ...
    Matt PainterMatt Painter
    Oct 15, 2012 at 7:48 pm
    Oct 18, 2012 at 12:22 pm
  • Hi, I have a complex Hadoop job that iterates over large graph data multiple times until some convergence condition is met. I know that the map output goes to the local disk of each particular mapper ...
    Jim TwenskyJim Twensky
    Oct 5, 2012 at 4:31 pm
    Oct 8, 2012 at 11:03 pm
  • Hi, I had made Hadoop setup & set all the configuration files. when go to this url through browser http://localhost:50070, it opened well but when i clicked on BrowseTheFileSystem , its redirecting ...
    Murthy nvvsMurthy nvvs
    Oct 11, 2012 at 4:54 am
    Oct 18, 2012 at 7:49 am
  • Is that anyway to control who can submit job to a pool.? Eg. Pool1, can run jobs submitted from any users except userx. Userx can submit jobs to poolx only. Can't submit to pool1. Hope this make ...
    Patai SangbutsarakumPatai Sangbutsarakum
    Oct 14, 2012 at 12:33 am
    Oct 18, 2012 at 10:12 am
  • Hello Hadoopers, I was reading the hardware recommendation doc. from Cloudera/HP, and this is one of the recommendation about CPU. "To remove the bottleneck for CPU bound workloads, for the best ...
    Patai SangbutsarakumPatai Sangbutsarakum
    Oct 11, 2012 at 4:23 pm
    Oct 13, 2012 at 7:22 am
  • Hi, I have a file which has some financial transaction data. Each transaction will have amount and a credit/debit indicator. I want to write a mapreduce program which computes cumulative credit & ...
    SarathSarath
    Oct 4, 2012 at 1:59 pm
    Oct 19, 2012 at 6:03 am
  • Hi , We are on a very early stage of our hadoop project and want to do a POC. We have ~ 5-6 terabytes of row data and we are going to execute some aggregations. We plan to use 8 - 10 machines ...
    Oleg RuchovetsOleg Ruchovets
    Oct 1, 2012 at 10:02 pm
    Oct 3, 2012 at 5:21 pm
  • Hi All, Had anyone tried installing Hadoop on mac pc..if yes can u please share the installation steps.. Thanks in advance.. Thanks, Suneel Sent from my iphone
    Suneel hadoopSuneel hadoop
    Oct 16, 2012 at 10:51 am
    Oct 16, 2012 at 4:08 pm
  • Hi , i install Hadoop on window with the help of Cygwin . *data node and Task tracker is not starting .* can some one help me in this ? i added Log File with this mail . is any one have ...
    Sujit DhamaleSujit Dhamale
    Oct 9, 2012 at 5:31 am
    Oct 11, 2012 at 9:55 am
  • Hello, can we create symlinks within hadoop is ther any shell commands or can we do it thru java....
    Visioner SadakVisioner Sadak
    Oct 8, 2012 at 6:44 am
    Oct 9, 2012 at 1:24 pm
  • I am running Hadoop 1.0.3 in Pseudo distributed mode. When I submit a map/reduce job to process a file of size about 16 GB, in job.xml, I have the following mapred.map.tasks =242 ...
    Shing Hing ManShing Hing Man
    Oct 2, 2012 at 4:35 pm
    Oct 3, 2012 at 1:50 pm
  • Hi, Does anyone know any (opensource) project that builds a rules engine (based on RETE) on top Hadoop? Searching a bit on the net, I have only seen a small reference to Concord/IBM but there is ...
    Luangsay SourygnaLuangsay Sourygna
    Oct 19, 2012 at 7:25 pm
    Oct 21, 2012 at 1:50 pm
  • Hi, Imagine I have a very fast hard drive that I want to use for the NameNode. That is, I want the NameNode to store its blocks information on this hard drive instead of in memory. Why would I do it? ...
    Mark KerznerMark Kerzner
    Oct 12, 2012 at 4:00 am
    Oct 17, 2012 at 11:28 pm
  • Hi, I have an issues with hadoop dfs, I have 3 servers (24Gb RAM on each). The servers are not overloaded, they just have hadoop installed. One have datanode and namenode, second - datanode only, ...
    AlexeyAlexey
    Oct 9, 2012 at 10:13 am
    Oct 16, 2012 at 2:39 am
  • I have read around about the hardware recommendation for hadoop cluster. One of them is recommend 1:1 ratio between spindle per core. Intel CPU come with Hyperthread which will double the number ...
    Patai SangbutsarakumPatai Sangbutsarakum
    Oct 12, 2012 at 5:46 pm
    Oct 13, 2012 at 5:53 am
  • Hi, if I create a Lucene index in each mapper, locally, then copy them to under /jobid/mapid1, /jodid/mapid2, and then in the reducers copy them to some Solr machine (perhaps even merging), does such ...
    Mark KerznerMark Kerzner
    Oct 10, 2012 at 2:48 am
    Oct 12, 2012 at 5:24 am
  • Hello all, I'm looking for a reference architecture for hadoop. The only result I found is Lambda architecture from Nathan Marz[0]. With architecture I mean answers to question like: - How should I ...
    Daniel KäferDaniel Käfer
    Oct 25, 2012 at 7:24 pm
    Oct 29, 2012 at 11:26 pm
  • How can we manage cluster-wide atomic operations? Such as maintaining an auto-increment counter. Does Hadoop provide native support for these kinds of operations? An in case ultimate answer involves ...
    David ParksDavid Parks
    Oct 27, 2012 at 3:08 am
    Oct 29, 2012 at 10:16 am
  • To all, I have a few questions regarding YARN (with respect to Hadoop): Are YARN and Hadoop separate, or is YARN the successor to Hadoop? What are the major conceptual differences between YARN and ...
    Tom BrownTom Brown
    Oct 18, 2012 at 8:33 pm
    Oct 24, 2012 at 4:32 am
  • Hi Users, I am trying to install Hadoop 0.20.2 on a cluster on two virtual machines. One acting as master other as slave. I am able to ssh from master to slave and vice verse. But when I run ...
    Sundeep KambhmapatiSundeep Kambhmapati
    Oct 17, 2012 at 9:05 pm
    Oct 22, 2012 at 3:20 am
  • Hi all, I installed Hadoop HDFS on 3 nodes, a namenode and 2 datanodes, when i want to start dfs processes, Only secondaryNameNode is launched but the namenode datanodes processes doesn't work there ...
    Khaled Ben BahriKhaled Ben Bahri
    Oct 16, 2012 at 9:19 am
    Oct 16, 2012 at 10:04 am
  • I would like to be able to resize a set of inputs, already in SequenceFile format, to be larger. I have tried 'hadoop distcp -Ddfs.block.size=$[64*1024*1024]' and did not get what I expected. The ...
    Anna LahoudAnna Lahoud
    Oct 1, 2012 at 6:31 pm
    Oct 9, 2012 at 4:28 pm
  • Hi, Was curious if there was a method to measure the total number of IOPS (I/O operations per second) on a HDFS cluster. -- --- Get your facts first, then you can distort them as you please.--
    RitaRita
    Oct 21, 2012 at 12:31 pm
    Oct 26, 2012 at 11:57 am
  • Hi all, I am trying to run the command ssh Master it runs and shows after entering password. Password: abc Last login: Thu Oct 25 13:51:06 2012 from master But ssh for Slave through error. ssh Slave ...
    Yogesh Kumar13Yogesh Kumar13
    Oct 25, 2012 at 8:29 am
    Oct 25, 2012 at 5:37 pm
  • Hi Hadoopers, I have <property <name dfs.replication</name <value 2</value <final true</final </property set in hdfs-site.xml in staging environment cluster. while the staging cluster is running the ...
    Patai SangbutsarakumPatai Sangbutsarakum
    Oct 15, 2012 at 7:02 pm
    Oct 16, 2012 at 7:28 am
  • hi all When I run "hadoop job -status xxx",Output the following some list. Rack-local map tasks=124 Data-local map tasks=6 What is the difference between Rack-local map tasks and Data-local map ...
    Centerqi huCenterqi hu
    Oct 7, 2012 at 1:57 pm
    Oct 8, 2012 at 5:45 am
  • I am working on some samples where I want to write to HDFS running on another machine (different OS etc.) The identity of my client process is just whatever my OS says it is (e.g., 'oleg') hence ...
    Oleg ZhurakouskyOleg Zhurakousky
    Oct 5, 2012 at 1:20 pm
    Oct 5, 2012 at 11:58 pm
  • I want to use HDFS HA function, I find the IO Fencing function is complex in hadoop2.0. I think we can use file lock to implement the IO Fencing function, I think that is simple. Thanks, LiuLei
    Lei liuLei liu
    Oct 25, 2012 at 8:28 am
    Oct 27, 2012 at 5:35 pm
  • I want to create a MapReduce job which reads many multi-gigabyte input files from various HTTP sources & processes them nightly. Is there a reasonably flexible way to do this in the Hadoop job its ...
    David ParksDavid Parks
    Oct 22, 2012 at 8:41 am
    Oct 24, 2012 at 4:25 pm
  • Anyone using Hadoop running on Isilon NAS? I am trying to submit a job with a user other than the one running Hadoop and I'm getting the following error: Exception in thread "main" ...
    Artem ErvitsArtem Ervits
    Oct 17, 2012 at 7:20 pm
    Oct 18, 2012 at 3:25 pm
  • Hi all, a very strange thing is happening with my hadoop program. My map simply emits tuples with a custom object as key (which implement WritableComparable). The object is made of 2 fields, and I ...
    Alberto CordioliAlberto Cordioli
    Oct 15, 2012 at 7:12 pm
    Oct 16, 2012 at 6:45 pm
  • Hello, I am trying to run hadoop on s3 using distributed mode. However I am having issues running my job successfully on it. I get the following error I followed the instructions provided in this ...
    Parth SavaniParth Savani
    Oct 15, 2012 at 5:13 pm
    Oct 16, 2012 at 3:11 pm
  • I am bringing up a Hadoop cluster for the first time (but am an experienced sysadmin with lots of cluster experience) and running into an issue with permissions on mapred.system.dir. It has generally ...
    Goldstone, Robin J.Goldstone, Robin J.
    Oct 9, 2012 at 11:45 pm
    Oct 14, 2012 at 8:47 am
  • Hi, all I'm modifying the source code of hadoop, specifically, source code of FairScheduler In last weeks, no problem during my development, but last night, when i add some code on ...
    Nan ZhuNan Zhu
    Oct 13, 2012 at 3:37 pm
    Oct 13, 2012 at 4:36 pm
  • Guys, have been stretching my head for the past couple of days. Why are my tags duplicated while the content they wrap around i.e.my StringBuilder sb is not? My Reduce code is: while ...
    Kartashov, AndyKartashov, Andy
    Oct 2, 2012 at 1:32 pm
    Oct 2, 2012 at 7:54 pm
  • I think these methods should are idempotent, these methods should be repeated calls to be harmless by same client. Thanks, LiuLei
    Lei liuLei liu
    Oct 28, 2012 at 6:41 am
    Nov 5, 2012 at 1:59 am
  • Hello friends, I m runnning hadoop in psuedo distr mode at a remote linux ip ,Thru the namenode WEB UI i m able to access the directory structure(thru data node) by clicking on browse the directory ...
    Visioner SadakVisioner Sadak
    Oct 19, 2012 at 6:28 am
    Oct 31, 2012 at 12:05 am
  • Hi, on our cluster our jobs usually satisfied with less than 2 GB of heap space. so we have on our 8 GB computers 3 maps maximum and on our 16 GB computers 4 maps maximum (we only have quad core CPUs ...
    Marco ZühlkeMarco Zühlke
    Oct 30, 2012 at 3:50 pm
    Oct 30, 2012 at 5:51 pm
  • Hi, I have a data on remote machine accessible over ssh. I have Hadoop CDH4 installed on RHEL. I am planning to load quite a few Petabytes of Data onto HDFS. Which will be the fastest method to use ...
    Sumit ghoshSumit ghosh
    Oct 30, 2012 at 10:07 am
    Oct 30, 2012 at 2:24 pm
  • Hi All, I run this command hadoop fsck -Ddfs.http.address=localhost:50070 / and found that some blocks are missing and corrupted results comes like.. /user/hive/warehouse/tt_report_htcount/000000_0 ...
    Yogesh Kumar13Yogesh Kumar13
    Oct 29, 2012 at 8:34 am
    Oct 29, 2012 at 12:31 pm
  • Hi, Can anyone give the clear idea about these comparisons on same hardware & software configuration. Sql server hbase hadoop+hbase data compression ? ? ? (yes/no,if all yes where it is more ...
    Iwannaplay gamesIwannaplay games
    Oct 18, 2012 at 10:49 am
    Oct 26, 2012 at 11:07 am
  • Hi All, I am trying to copy the public key by this command. Master:~ mediaadmin$ ssh-copy -id -i $HOME/.ssh/id_rsa.pub pluto@Slave I have two machines Master Name is pluto and same name is of Slave ...
    Yogesh Kumar13Yogesh Kumar13
    Oct 25, 2012 at 9:55 am
    Oct 25, 2012 at 8:38 pm
  • Hi all, Can we discuss performance of pig vs hive 1) what hive is good at? 2) what pig is good at? 3) Hive optimizer vs pig optimizer 4) hive limitations vs pig limitations Regards Abhi Sent from my ...
    AbhishekAbhishek
    Oct 3, 2012 at 9:53 pm
    Oct 4, 2012 at 4:17 am
  • Hi all, I have understood the Hadoop and Hadoop Ecosystem(Pig as ETL, Hive as DataWare house, Sqoop as importing tool). I worked and learned on single node cluster with demo data. As Hadoop suits ...
    Yogesh dhariYogesh dhari
    Oct 1, 2012 at 1:37 pm
    Oct 3, 2012 at 3:02 am
  • Hi, i am kind of unsure where to post this problem, but i think it is more related to hadoop than to pig. By successfully executing a pig script i created a new file in my hdfs. Sadly though, i ...
    Björn-Elmar MacekBjörn-Elmar Macek
    Oct 1, 2012 at 4:12 pm
    Oct 3, 2012 at 12:26 am
  • Hi guys, I've been trying to figure out whether a map-side join using the join-package does anything clever regarding data locality with respect to at least one of the partitions to join. To be more ...
    Sigurd SpieckermannSigurd Spieckermann
    Oct 22, 2012 at 8:29 pm
    Nov 6, 2012 at 9:00 am
  • Hi list, Are the any tools for parsing and extracting data from Hadoop's Job Logs? I want to do stuff like .. 1. Getting run time of each map/reduce task 2. Total map/reduce tasks ran on a particular ...
    Bharath vissapragadaBharath vissapragada
    Oct 30, 2012 at 1:49 am
    Oct 30, 2012 at 5:22 am
  • Hi all, I am using last stable Hadoop version (1.0.3) and I am implementing right now my first MR jobs. I read about the presence of 2 API: the old and the new one. I read some stuff about them, but ...
    Alberto CordioliAlberto Cordioli
    Oct 22, 2012 at 1:23 pm
    Oct 25, 2012 at 3:12 pm
Group Navigation
period‹ prev | Oct 2012 | next ›
Group Overview
groupcommon-user @
categorieshadoop
discussions212
posts955
users218
websitehadoop.apache.org...
irc#hadoop

218 users for October 2012

Harsh J: 87 posts Kartashov, Andy: 39 posts Visioner Sadak: 39 posts Michael Segel: 33 posts Bertrand Dechoux: 25 posts Bejoy KS: 20 posts Ted Dunning: 20 posts Steve Loughran: 18 posts Andy Isaacson: 15 posts Jay Vyas: 15 posts Patai Sangbutsarakum: 15 posts Abhishek dodda: 11 posts Chris Nauroth: 11 posts Lei liu: 11 posts Vinod Kumar Vavilapalli: 11 posts Yogesh Kumar13: 10 posts Alberto Cordioli: 10 posts David Parks: 10 posts Mohammad Tariq: 10 posts Sudha sadhasivam: 10 posts
show more