Search Discussions
-
Hi All, I am following http://wiki.apache.org/hadoop/MountableHDFS for HDFS mount. I have successfully followed the steps till "Installing" and I am able mount it properly. After that I am trying ...
Stuti Awasthi
Jan 4, 2012 at 10:59 am
Jan 11, 2012 at 4:10 pm -
17
no namenode
Hi, I start-all.sh my hadoop master node, but I can't find any namenode on it. Would anyone be so kindly to tell me how to fix this problem? Thanks.Martinus Martinus
Jan 13, 2012 at 10:36 am
Feb 1, 2012 at 11:31 am -
Hi, I have setup a hadoop clusters with 4 nodes and I have start-all.sh and checked in every node, there are tasktracker and datanode run, but when I run hadoop dfsadmin -report it's said like this : ...
Martinus Martinus
Jan 2, 2012 at 3:24 am
Jan 5, 2012 at 8:06 am -
Hi All, I was wondering if there was an easy way to combing multiple .avro files efficiently. e.g. combining multiple hours of logs into a daily aggregate Note that our Avro schema might evolve to ...
Frank Grimes
Jan 6, 2012 at 3:56 pm
Jan 12, 2012 at 3:43 pm -
Hi, I am relatively new to Hadoop and I am trying to utilize HDFS for own application where I want to take advantage of data partitioning HDFS performs. The idea is that I get list of individual ...
David Pavlis
Jan 6, 2012 at 11:03 am
Jan 11, 2012 at 4:34 pm -
Hi all, Wish you all a very Happy New Year. As I know that files once written in Hadoop are immutable. I can't edit the files in Apache distribution of Hadoop. I tried with MapR distribution and I am ...
Stuti Awasthi
Jan 3, 2012 at 12:12 pm
Jan 4, 2012 at 7:08 pm -
Hi, I've been searching on how to configure the maximum capacity of a datanode. I've added big volumes to one of my datanodes, but the configured capacity doesn't get bigger than the default 5GB. If ...
Hamed Ghavamnia
Jan 1, 2012 at 6:04 am
Jan 2, 2012 at 11:58 am -
Does anyone know of any work/ideas to encrypt data stored on hdfs? Ideally both temporary files and final files would be encrypted. Or there would have to be a mechanism in hdfs to securely wipe ...
Koert Kuipers
Jan 20, 2012 at 9:09 pm
Feb 12, 2012 at 5:25 pm -
Hi All, I wanted to apply ACL on per file level in Hadoop Cluster i.e. I want to apply ACL's on every file which is present in Hadoop Cluster. I tried mounting HDFS using fuse_dfs , that works fine. ...
Stuti Awasthi
Jan 18, 2012 at 12:26 pm
Jan 20, 2012 at 7:43 am -
Good day, I’m guessing this question been asked a myriad of times, but we’re about to get serious with some of our Hadoop implementations so I wanted to re-ask to see if I’m missing anything, or if ...
Mac Noland
Jan 3, 2012 at 8:56 pm
Jan 5, 2012 at 2:34 pm -
Hi, Does anyone have a link for a good tutorial regarding how to setup a fully distributed hbase on top of a multi-node hadoop cluster? Thanks, Hamed
Hamed Ghavamnia
Jan 2, 2012 at 12:23 pm
Jan 3, 2012 at 11:28 am -
Hi guys, we are running Hadoop 0.20.2+228, the namenode process's memory grows overtime to occupy over 18GB. However, if I restart the namenode, it only occupies about 10GB after it is stable. I am ...
Felix gao
Jan 18, 2012 at 11:23 pm
Jan 20, 2012 at 7:57 pm -
Hey guys, Sorry for the typo in my last message.I have corrected it. I would like to perform some additional processing on the data which is streamed to DFSClient. To my knowledge the class ...
Sesha Kumar
Jan 16, 2012 at 2:20 pm
Jan 20, 2012 at 12:55 am -
Hi All, I am able to successfully built fuse_dfs shipped with Hadoop-1.0.0 but when I try to mount HDFS using fuse_dfs_wrapper.sh I get the following error .When I used fuse_dfs built from ...
Stuti Awasthi
Jan 18, 2012 at 11:03 am
Jan 19, 2012 at 12:40 pm -
Hi All, I am able to mount HDFS using fuse-dfs. I am using http://wiki.apache.org/hadoop/MountableHDFS for the reference. Currently able to HDFS mount using fuse_dfs_wrapper.sh and also directly ...
Stuti Awasthi
Jan 12, 2012 at 12:03 pm
Jan 16, 2012 at 11:17 am -
Hi, I am trying to setup a HDFS federation and getting the below error. Also, pasted the core-site.xml and hdfs-site.xml at the bottom of the mail. Did I miss something in the configuration files? ...
Praveen Sripati
Jan 11, 2012 at 6:55 am
Jan 11, 2012 at 4:58 pm -
Hi- How does the NameNode handle load balancing of non-local reads with multiple block locations when locality is equal? IE, if the client is equidistant (same rack) from 2 DataNodes hosting the same ...
Ben Clay
Jan 5, 2012 at 9:40 pm
Jan 6, 2012 at 11:42 pm -
I have two questions regarding creation of replicas. - When a user uploads a file to HDFS, it returns whenever the first replica is created? or the client needs wait until all replicas are created? - ...
Zhenhua (Gerald) Guo
Jan 26, 2012 at 6:57 pm
Jan 27, 2012 at 7:52 am -
Hello list, I have a situation wherein I have to move large binary files(~TB) from remote machines into the HDFS.While looking for some way to do this I came across Hoop.Could anyone tell me whether ...
Mohammad Tariq
Jan 24, 2012 at 3:31 pm
Jan 24, 2012 at 7:29 pm -
Also wanted to confirm that is Hadoop 0.20.205 is same as Hadoop 1.0.0 . I have read it at some places that Hadoop0.20.205 is renamed to Hadoop 1.0.0 -----Original Message----- From: Stuti Awasthi ...
Stuti Awasthi
Jan 18, 2012 at 7:41 am
Jan 19, 2012 at 5:02 am -
Hi All, We're trying to speed up an M/R job which combines multiple .avro files. We've noticed that when it spills to disk, it's in uncompressed format. Is there a way to make it spill temporary ...
Frank Grimes
Jan 12, 2012 at 3:41 pm
Jan 12, 2012 at 11:25 pm -
hi. i have a small hadoop grid connected with a 1g network. when servers are configured to use the local dns server the jobs are running without a problem and copy speed during reduce is tens on MB. ...
Oren
Jan 4, 2012 at 3:09 pm
Jan 5, 2012 at 3:10 pm -
Hi folks, I'm trying to connect to a NameNode using Hadoop's RPC ClientProtocol class. Code below: InetSocketAddress address = new InetSocketAddress("namenode", 9000); Configuration conf = new ...
Michael Lok
Jan 17, 2012 at 3:21 am
Feb 1, 2012 at 2:30 pm -
Hi, I have my hadoop clusters running with 1 master and 6 slaves, and when I run bin/hadoop jar hadoop-examples-1.0.0.jar wordcount input output, I got the following error message from my slave ...
Martinus Martinus
Jan 31, 2012 at 5:55 am
Feb 1, 2012 at 12:03 pm -
Team, I would like to add additional directory under "dfs.name.dir" property in order to have one more copy of namenode metadata. My HDFS is already running and in production. Is there a clean way to ...
Jain, Prem
Jan 31, 2012 at 10:27 pm
Feb 1, 2012 at 3:39 am -
When I do an fsck / this is what I get and my jobs are failing with *Could not obtain block* /user/www/testoutput/part-00000: Under replicated blk_4592416107180235380_1833133. Target Replicas is 3 ...
Steve Lewis
Jan 24, 2012 at 6:15 pm
Jan 24, 2012 at 9:12 pm -
Hi all, I want to test Backup Node in Hadoop 1.0.0 for my 2 node cluster. Say machine A is master and machine B I want to setup as Backup node. I removed entry from "masters" file so that no SNN is ...
Stuti Awasthi
Jan 20, 2012 at 9:38 am
Jan 20, 2012 at 10:32 am -
When I run this command, I get an error message. This problem started first when I first accessed HDFS programatically. I am on a single node cluster in ubuntu. hadoop dfs -copyFromLocal ...
Apurv Verma
Jan 11, 2012 at 2:46 am
Jan 13, 2012 at 5:47 am -
Hello Yes this is quite trivial and for example ibm did that in their big insights flavor of hadoop The hadoop alrzady has quite a lot of instrumentation regarding monitoring and you can extend the ...
Franck Besnard
Jan 11, 2012 at 7:24 am
Jan 11, 2012 at 11:15 am -
Hi All, I am new to Hadoop. I was able to 3 datanode running and working. I purposefully shutdown one datanode and execute "bin/hadoop fs -copyFromLocal ../hadoop.sh ...
TS chia
Jan 5, 2012 at 4:24 pm
Jan 8, 2012 at 10:03 am -
When the checkpointing starts, the primary namenode starts a new edits file. During the checkpointing process will the namenode go into safe mode? According to the Hadoop - The Definitive Guide ...
Praveen Sripati
Jan 8, 2012 at 3:44 am
Jan 8, 2012 at 7:02 am -
I'm getting this recurring exception on my secondary NameNode. 2012-01-31 21:46:12,710 INFO org.apache.hadoop.hdfs.util.GSet: recommended=2097152, actual=2097152 2012-01-31 21:46:12,712 INFO ...
Gabriel Rosendorf
Jan 31, 2012 at 9:50 pm
Feb 1, 2012 at 3:26 pm -
Hi, Is there any DN metric for xceivers count? I'm using Hadoop 0.20.2-cdh3u2, so i checked the source of class org.apache.hadoop.hdfs.server.datanode.metrics.DataNodeMetrics and i didn't find ...
Leonardo Gamas
Jan 25, 2012 at 2:26 pm
Feb 1, 2012 at 1:26 pm -
Hi all, I am using hadoop 1.0.0 running in Cygwin. I'm getting the following error message when I try to run jobtracker. 12/01/30 17:18:21 FATAL mapred.JobTracker: java.lang.IllegalArgumentException: ...
Ina M
Jan 30, 2012 at 10:37 pm
Jan 31, 2012 at 1:36 am -
Sorry for the spam: I am attempting to install CDH3 using yum and getting the following error on few hosts. Not sure why just few hosts are throwing this error while others went fine. [email protected]# ...
Jain, Prem
Jan 19, 2012 at 12:18 am
Jan 19, 2012 at 6:02 am -
When using v0.20.2, the script command echo "test result is $( bin/hadoop dfs -test -d $FILEPATH )<" gives the following output whether FILEPATH is a directory or a file: test result is < Is this a ...
Pundurs, Mark
Jan 13, 2012 at 3:50 pm
Jan 13, 2012 at 5:23 pm -
Hi, I have tried the webhdfs functionality of Hadoop-1.0.0 and it is working fine. Just a small change is required in the documentation : Make a Directory declaration in documentation: curl -i -X PUT ...
Stuti Awasthi
Jan 19, 2012 at 12:18 pm
Feb 1, 2012 at 1:25 pm -
Hi, Quoted from *HDFS Architecture Guide<http://hadoop.apache.org/common/docs/current/hdfs_design.html * "HDFS relaxes a few POSIX requirements to enable streaming access to file I wanted to get a ...
Wasif Riaz Malik
Jan 23, 2012 at 10:53 am
Feb 1, 2012 at 12:13 pm -
Greetings to all- How does the W3C protocols play a role in distributed data model through Hadoop? Web Services: Simple Access Object Protocol: SOAP XML: Extensible Mark Up Language UDDI: Universal ...
Anthony Ghosn
Jan 30, 2012 at 11:51 pm
Feb 1, 2012 at 11:47 am -
Hi guys, I'm having a problem running map reduce jobs in hadoop. Whenever I try to run a map reduce job, I get the following exception: Caused by: org.apache.hadoop.ipc.RemoteException: ...
Sesha Kumar
Jan 31, 2012 at 7:49 am
Feb 1, 2012 at 11:42 am -
As I read in Hadoop tutorial, Hadoop replicate file blocks by a factor (default 3), in other words, it replicate each block file 3 times. Does Hadoop do it for all files? I mean files written by ...
Alieh Saeedi
Jan 31, 2012 at 12:20 pm
Jan 31, 2012 at 12:34 pm -
Given a HDFS slave node setup of 3 disks per node, should I have 3 filesystems (one file system per disk) in my dfs.data.dir listing, or should I have a single filesystem on a JBOD setup of 3 disks? ...
Aaron Tokhy
Jan 30, 2012 at 9:58 pm
Jan 30, 2012 at 10:11 pm -
Hey guys, I'm trying to use the TestDFSIO tool program given in hadoop. The problem i'm facing is that it doesn't provide any option to create files that have size less than 1MB. I'm using hadoop ...
Sesha Kumar
Jan 24, 2012 at 6:19 am
Jan 27, 2012 at 6:26 am -
I have a Hadoop cluster on which I have generated some data using Teragen. But while running Terasort on this data, it gives following error. java.lang.RuntimeException: Error in configuring object ...
Utkarsh Rathore
Jan 24, 2012 at 11:00 am
Jan 25, 2012 at 4:28 am -
Hii all, 1. I am using eclipse for development. The problem is that I have to create a jar everytime and copy the jar to the other user and then run it. Is there a faster way of doing development. I ...
Apurv Verma
Jan 13, 2012 at 5:47 am
Jan 13, 2012 at 7:25 am -
Has anyone had any experience building high-performance datanodes using SSD for storage? Any gotchas or notes to contribute? We were talking out a crazy idea for some stupid fast hadoop nodes using ...
Jonathan Disher
Jan 12, 2012 at 9:49 pm
Jan 12, 2012 at 9:52 pm -
can someone point me to the exact rules? thanks! for example, i want to know if i can take down a slave for the afternoon to fix something on the machine, without causing the cluster to start ...
Koert Kuipers
Jan 8, 2012 at 3:28 am
Jan 8, 2012 at 6:19 am -
Hi guys, Am trying to implement some solutions for small file problem in hdfs as part of my project work. I got my own set of files stored in my hadoop cluster. I need a tool or method to test and ...
Sesha Kumar
Jan 5, 2012 at 3:21 pm
Jan 5, 2012 at 6:25 pm -
Hi All, I am new to Hadoop and started learning the HDFS . I am unable to watch the video from the below link though I have access tohttps://Vimeo.com<https://vimeo.com/ . It says I don't have ...
krishnan N
Jan 4, 2012 at 12:59 am
Jan 4, 2012 at 4:51 am -
Hello, I've just posted a work done for few months now: http://sourceforge.net/projects/jxtadoop/ My goal is to be able to run Hadoop on desktop computers which are spread across the entreprise or ...
Jxtadoop
Jan 26, 2012 at 10:12 pm
Jan 26, 2012 at 10:12 pm
Group Overview
| group | hdfs-user |
| categories | hadoop |
| discussions | 61 |
| posts | 258 |
| users | 72 |
| website | hadoop.apache.org... |
| irc | #hadoop |
72 users for January 2012
Archives
- February 2013 (245)
- January 2013 (838)
- December 2012 (590)
- November 2012 (723)
- October 2012 (861)
- September 2012 (710)
- August 2012 (1,046)
- July 2012 (151)
- June 2012 (91)
- May 2012 (126)
- April 2012 (95)
- March 2012 (64)
- February 2012 (128)
- January 2012 (258)
- December 2011 (110)
- November 2011 (164)
- October 2011 (83)
- September 2011 (101)
- August 2011 (58)
- July 2011 (73)
- June 2011 (101)
- May 2011 (184)
- April 2011 (51)
- March 2011 (110)
- February 2011 (100)
- January 2011 (101)
- December 2010 (44)
- November 2010 (49)
- October 2010 (48)
- September 2010 (26)
- August 2010 (52)
- July 2010 (50)
- June 2010 (64)
- May 2010 (57)
- April 2010 (45)
- March 2010 (38)
- February 2010 (10)
- January 2010 (84)
- December 2009 (3)
- November 2009 (38)
- October 2009 (43)
- September 2009 (32)
- August 2009 (35)
- July 2009 (5)