FAQ

Search Discussions

61 discussions - 258 posts

  • Hi All, I am following http://wiki.apache.org/hadoop/MountableHDFS for HDFS mount. I have successfully followed the steps till "Installing" and I am able mount it properly. After that I am trying ...
    Stuti AwasthiStuti Awasthi
    Jan 4, 2012 at 10:59 am
    Jan 11, 2012 at 4:10 pm
  • Hi, I start-all.sh my hadoop master node, but I can't find any namenode on it. Would anyone be so kindly to tell me how to fix this problem? Thanks.
    Martinus MartinusMartinus Martinus
    Jan 13, 2012 at 10:36 am
    Feb 1, 2012 at 11:31 am
  • Hi, I have setup a hadoop clusters with 4 nodes and I have start-all.sh and checked in every node, there are tasktracker and datanode run, but when I run hadoop dfsadmin -report it's said like this : ...
    Martinus MartinusMartinus Martinus
    Jan 2, 2012 at 3:24 am
    Jan 5, 2012 at 8:06 am
  • Hi All, I was wondering if there was an easy way to combing multiple .avro files efficiently. e.g. combining multiple hours of logs into a daily aggregate Note that our Avro schema might evolve to ...
    Frank GrimesFrank Grimes
    Jan 6, 2012 at 3:56 pm
    Jan 12, 2012 at 3:43 pm
  • Hi, I am relatively new to Hadoop and I am trying to utilize HDFS for own application where I want to take advantage of data partitioning HDFS performs. The idea is that I get list of individual ...
    David PavlisDavid Pavlis
    Jan 6, 2012 at 11:03 am
    Jan 11, 2012 at 4:34 pm
  • Hi all, Wish you all a very Happy New Year. As I know that files once written in Hadoop are immutable. I can't edit the files in Apache distribution of Hadoop. I tried with MapR distribution and I am ...
    Stuti AwasthiStuti Awasthi
    Jan 3, 2012 at 12:12 pm
    Jan 4, 2012 at 7:08 pm
  • Hi, I've been searching on how to configure the maximum capacity of a datanode. I've added big volumes to one of my datanodes, but the configured capacity doesn't get bigger than the default 5GB. If ...
    Hamed GhavamniaHamed Ghavamnia
    Jan 1, 2012 at 6:04 am
    Jan 2, 2012 at 11:58 am
  • Does anyone know of any work/ideas to encrypt data stored on hdfs? Ideally both temporary files and final files would be encrypted. Or there would have to be a mechanism in hdfs to securely wipe ...
    Koert KuipersKoert Kuipers
    Jan 20, 2012 at 9:09 pm
    Feb 12, 2012 at 5:25 pm
  • Hi All, I wanted to apply ACL on per file level in Hadoop Cluster i.e. I want to apply ACL's on every file which is present in Hadoop Cluster. I tried mounting HDFS using fuse_dfs , that works fine. ...
    Stuti AwasthiStuti Awasthi
    Jan 18, 2012 at 12:26 pm
    Jan 20, 2012 at 7:43 am
  • Good day, I’m guessing this question been asked a myriad of times, but we’re about to get serious with some of our Hadoop implementations so I wanted to re-ask to see if I’m missing anything, or if ...
    Mac NolandMac Noland
    Jan 3, 2012 at 8:56 pm
    Jan 5, 2012 at 2:34 pm
  • Hi, Does anyone have a link for a good tutorial regarding how to setup a fully distributed hbase on top of a multi-node hadoop cluster? Thanks, Hamed
    Hamed GhavamniaHamed Ghavamnia
    Jan 2, 2012 at 12:23 pm
    Jan 3, 2012 at 11:28 am
  • Hi guys, we are running Hadoop 0.20.2+228, the namenode process's memory grows overtime to occupy over 18GB. However, if I restart the namenode, it only occupies about 10GB after it is stable. I am ...
    Felix gaoFelix gao
    Jan 18, 2012 at 11:23 pm
    Jan 20, 2012 at 7:57 pm
  • Hey guys, Sorry for the typo in my last message.I have corrected it. I would like to perform some additional processing on the data which is streamed to DFSClient. To my knowledge the class ...
    Sesha KumarSesha Kumar
    Jan 16, 2012 at 2:20 pm
    Jan 20, 2012 at 12:55 am
  • Hi All, I am able to successfully built fuse_dfs shipped with Hadoop-1.0.0 but when I try to mount HDFS using fuse_dfs_wrapper.sh I get the following error .When I used fuse_dfs built from ...
    Stuti AwasthiStuti Awasthi
    Jan 18, 2012 at 11:03 am
    Jan 19, 2012 at 12:40 pm
  • Hi All, I am able to mount HDFS using fuse-dfs. I am using http://wiki.apache.org/hadoop/MountableHDFS for the reference. Currently able to HDFS mount using fuse_dfs_wrapper.sh and also directly ...
    Stuti AwasthiStuti Awasthi
    Jan 12, 2012 at 12:03 pm
    Jan 16, 2012 at 11:17 am
  • Hi, I am trying to setup a HDFS federation and getting the below error. Also, pasted the core-site.xml and hdfs-site.xml at the bottom of the mail. Did I miss something in the configuration files? ...
    Praveen SripatiPraveen Sripati
    Jan 11, 2012 at 6:55 am
    Jan 11, 2012 at 4:58 pm
  • Hi- How does the NameNode handle load balancing of non-local reads with multiple block locations when locality is equal? IE, if the client is equidistant (same rack) from 2 DataNodes hosting the same ...
    Ben ClayBen Clay
    Jan 5, 2012 at 9:40 pm
    Jan 6, 2012 at 11:42 pm
  • I have two questions regarding creation of replicas. - When a user uploads a file to HDFS, it returns whenever the first replica is created? or the client needs wait until all replicas are created? - ...
    Zhenhua (Gerald) GuoZhenhua (Gerald) Guo
    Jan 26, 2012 at 6:57 pm
    Jan 27, 2012 at 7:52 am
  • Hello list, I have a situation wherein I have to move large binary files(~TB) from remote machines into the HDFS.While looking for some way to do this I came across Hoop.Could anyone tell me whether ...
    Mohammad TariqMohammad Tariq
    Jan 24, 2012 at 3:31 pm
    Jan 24, 2012 at 7:29 pm
  • Also wanted to confirm that is Hadoop 0.20.205 is same as Hadoop 1.0.0 . I have read it at some places that Hadoop0.20.205 is renamed to Hadoop 1.0.0 -----Original Message----- From: Stuti Awasthi ...
    Stuti AwasthiStuti Awasthi
    Jan 18, 2012 at 7:41 am
    Jan 19, 2012 at 5:02 am
  • Hi All, We're trying to speed up an M/R job which combines multiple .avro files. We've noticed that when it spills to disk, it's in uncompressed format. Is there a way to make it spill temporary ...
    Frank GrimesFrank Grimes
    Jan 12, 2012 at 3:41 pm
    Jan 12, 2012 at 11:25 pm
  • hi. i have a small hadoop grid connected with a 1g network. when servers are configured to use the local dns server the jobs are running without a problem and copy speed during reduce is tens on MB. ...
    OrenOren
    Jan 4, 2012 at 3:09 pm
    Jan 5, 2012 at 3:10 pm
  • Hi folks, I'm trying to connect to a NameNode using Hadoop's RPC ClientProtocol class. Code below: InetSocketAddress address = new InetSocketAddress("namenode", 9000); Configuration conf = new ...
    Michael LokMichael Lok
    Jan 17, 2012 at 3:21 am
    Feb 1, 2012 at 2:30 pm
  • Hi, I have my hadoop clusters running with 1 master and 6 slaves, and when I run bin/hadoop jar hadoop-examples-1.0.0.jar wordcount input output, I got the following error message from my slave ...
    Martinus MartinusMartinus Martinus
    Jan 31, 2012 at 5:55 am
    Feb 1, 2012 at 12:03 pm
  • Team, I would like to add additional directory under "dfs.name.dir" property in order to have one more copy of namenode metadata. My HDFS is already running and in production. Is there a clean way to ...
    Jain, PremJain, Prem
    Jan 31, 2012 at 10:27 pm
    Feb 1, 2012 at 3:39 am
  • When I do an fsck / this is what I get and my jobs are failing with *Could not obtain block* /user/www/testoutput/part-00000: Under replicated blk_4592416107180235380_1833133. Target Replicas is 3 ...
    Steve LewisSteve Lewis
    Jan 24, 2012 at 6:15 pm
    Jan 24, 2012 at 9:12 pm
  • Hi all, I want to test Backup Node in Hadoop 1.0.0 for my 2 node cluster. Say machine A is master and machine B I want to setup as Backup node. I removed entry from "masters" file so that no SNN is ...
    Stuti AwasthiStuti Awasthi
    Jan 20, 2012 at 9:38 am
    Jan 20, 2012 at 10:32 am
  • When I run this command, I get an error message. This problem started first when I first accessed HDFS programatically. I am on a single node cluster in ubuntu. hadoop dfs -copyFromLocal ...
    Apurv VermaApurv Verma
    Jan 11, 2012 at 2:46 am
    Jan 13, 2012 at 5:47 am
  • Hello Yes this is quite trivial and for example ibm did that in their big insights flavor of hadoop The hadoop alrzady has quite a lot of instrumentation regarding monitoring and you can extend the ...
    Franck BesnardFranck Besnard
    Jan 11, 2012 at 7:24 am
    Jan 11, 2012 at 11:15 am
  • Hi All, I am new to Hadoop. I was able to 3 datanode running and working. I purposefully shutdown one datanode and execute "bin/hadoop fs -copyFromLocal ../hadoop.sh ...
    TS chiaTS chia
    Jan 5, 2012 at 4:24 pm
    Jan 8, 2012 at 10:03 am
  • When the checkpointing starts, the primary namenode starts a new edits file. During the checkpointing process will the namenode go into safe mode? According to the Hadoop - The Definitive Guide ...
    Praveen SripatiPraveen Sripati
    Jan 8, 2012 at 3:44 am
    Jan 8, 2012 at 7:02 am
  • I'm getting this recurring exception on my secondary NameNode. 2012-01-31 21:46:12,710 INFO org.apache.hadoop.hdfs.util.GSet: recommended=2097152, actual=2097152 2012-01-31 21:46:12,712 INFO ...
    Gabriel RosendorfGabriel Rosendorf
    Jan 31, 2012 at 9:50 pm
    Feb 1, 2012 at 3:26 pm
  • Hi, Is there any DN metric for xceivers count? I'm using Hadoop 0.20.2-cdh3u2, so i checked the source of class org.apache.hadoop.hdfs.server.datanode.metrics.DataNodeMetrics and i didn't find ...
    Leonardo GamasLeonardo Gamas
    Jan 25, 2012 at 2:26 pm
    Feb 1, 2012 at 1:26 pm
  • Hi all, I am using hadoop 1.0.0 running in Cygwin. I'm getting the following error message when I try to run jobtracker. 12/01/30 17:18:21 FATAL mapred.JobTracker: java.lang.IllegalArgumentException: ...
    Ina MIna M
    Jan 30, 2012 at 10:37 pm
    Jan 31, 2012 at 1:36 am
  • Sorry for the spam: I am attempting to install CDH3 using yum and getting the following error on few hosts. Not sure why just few hosts are throwing this error while others went fine. root@namenode1# ...
    Jain, PremJain, Prem
    Jan 19, 2012 at 12:18 am
    Jan 19, 2012 at 6:02 am
  • When using v0.20.2, the script command echo "test result is $( bin/hadoop dfs -test -d $FILEPATH )<" gives the following output whether FILEPATH is a directory or a file: test result is < Is this a ...
    Pundurs, MarkPundurs, Mark
    Jan 13, 2012 at 3:50 pm
    Jan 13, 2012 at 5:23 pm
  • Hi, I have tried the webhdfs functionality of Hadoop-1.0.0 and it is working fine. Just a small change is required in the documentation : Make a Directory declaration in documentation: curl -i -X PUT ...
    Stuti AwasthiStuti Awasthi
    Jan 19, 2012 at 12:18 pm
    Feb 1, 2012 at 1:25 pm
  • Hi, Quoted from *HDFS Architecture Guide<http://hadoop.apache.org/common/docs/current/hdfs_design.html * "HDFS relaxes a few POSIX requirements to enable streaming access to file I wanted to get a ...
    Wasif Riaz MalikWasif Riaz Malik
    Jan 23, 2012 at 10:53 am
    Feb 1, 2012 at 12:13 pm
  • Greetings to all- How does the W3C protocols play a role in distributed data model through Hadoop? Web Services: Simple Access Object Protocol: SOAP XML: Extensible Mark Up Language UDDI: Universal ...
    Anthony GhosnAnthony Ghosn
    Jan 30, 2012 at 11:51 pm
    Feb 1, 2012 at 11:47 am
  • Hi guys, I'm having a problem running map reduce jobs in hadoop. Whenever I try to run a map reduce job, I get the following exception: Caused by: org.apache.hadoop.ipc.RemoteException: ...
    Sesha KumarSesha Kumar
    Jan 31, 2012 at 7:49 am
    Feb 1, 2012 at 11:42 am
  • As I read in Hadoop tutorial, Hadoop replicate file blocks by a factor (default 3), in other words, it replicate each block file 3 times. Does Hadoop do it for all files? I mean files written by ...
    Alieh SaeediAlieh Saeedi
    Jan 31, 2012 at 12:20 pm
    Jan 31, 2012 at 12:34 pm
  • Given a HDFS slave node setup of 3 disks per node, should I have 3 filesystems (one file system per disk) in my dfs.data.dir listing, or should I have a single filesystem on a JBOD setup of 3 disks? ...
    Aaron TokhyAaron Tokhy
    Jan 30, 2012 at 9:58 pm
    Jan 30, 2012 at 10:11 pm
  • Hey guys, I'm trying to use the TestDFSIO tool program given in hadoop. The problem i'm facing is that it doesn't provide any option to create files that have size less than 1MB. I'm using hadoop ...
    Sesha KumarSesha Kumar
    Jan 24, 2012 at 6:19 am
    Jan 27, 2012 at 6:26 am
  • I have a Hadoop cluster on which I have generated some data using Teragen. But while running Terasort on this data, it gives following error. java.lang.RuntimeException: Error in configuring object ...
    Utkarsh RathoreUtkarsh Rathore
    Jan 24, 2012 at 11:00 am
    Jan 25, 2012 at 4:28 am
  • Hii all, 1. I am using eclipse for development. The problem is that I have to create a jar everytime and copy the jar to the other user and then run it. Is there a faster way of doing development. I ...
    Apurv VermaApurv Verma
    Jan 13, 2012 at 5:47 am
    Jan 13, 2012 at 7:25 am
  • Has anyone had any experience building high-performance datanodes using SSD for storage? Any gotchas or notes to contribute? We were talking out a crazy idea for some stupid fast hadoop nodes using ...
    Jonathan DisherJonathan Disher
    Jan 12, 2012 at 9:49 pm
    Jan 12, 2012 at 9:52 pm
  • can someone point me to the exact rules? thanks! for example, i want to know if i can take down a slave for the afternoon to fix something on the machine, without causing the cluster to start ...
    Koert KuipersKoert Kuipers
    Jan 8, 2012 at 3:28 am
    Jan 8, 2012 at 6:19 am
  • Hi guys, Am trying to implement some solutions for small file problem in hdfs as part of my project work. I got my own set of files stored in my hadoop cluster. I need a tool or method to test and ...
    Sesha KumarSesha Kumar
    Jan 5, 2012 at 3:21 pm
    Jan 5, 2012 at 6:25 pm
  • Hi All, I am new to Hadoop and started learning the HDFS . I am unable to watch the video from the below link though I have access tohttps://Vimeo.com<https://vimeo.com/ . It says I don't have ...
    krishnan Nkrishnan N
    Jan 4, 2012 at 12:59 am
    Jan 4, 2012 at 4:51 am
  • Hello, I've just posted a work done for few months now: http://sourceforge.net/projects/jxtadoop/ My goal is to be able to run Hadoop on desktop computers which are spread across the entreprise or ...
    JxtadoopJxtadoop
    Jan 26, 2012 at 10:12 pm
    Jan 26, 2012 at 10:12 pm
Group Navigation
period‹ prev | Jan 2012 | next ›
Group Overview
grouphdfs-user @
categorieshadoop
discussions61
posts258
users72
websitehadoop.apache.org...
irc#hadoop

72 users for January 2012

Stuti Awasthi: 32 posts Harsh J: 19 posts Alo.alt: 18 posts Martinus Martinus: 18 posts Joey Echeverria: 11 posts Hamed Ghavamnia: 10 posts Sesha Kumar: 8 posts Frank Grimes: 7 posts Praveen Sripati: 6 posts Arun C Murthy: 5 posts David Pavlis: 5 posts Mohamed Elsayed: 5 posts Todd Lipcon: 5 posts Uma Maheswara Rao G: 5 posts Apurv Verma: 4 posts Bejoy Ks: 4 posts Michael Lok: 4 posts Rajiv Chittajallu: 4 posts Tim Broberg: 4 posts Ben Clay: 3 posts
show more