FAQ

Search Discussions

29 discussions - 101 posts

  • Hi, If I define HDFS to use blocks of 64 MB, and I store in HDFS a 1KB file, this file will ocupy 64MB in the HDFS? Thanks,
    Pedro CostaPedro Costa
    Jun 10, 2011 at 3:06 pm
    Jun 13, 2011 at 9:08 pm
  • Hello, So I have a question about changing dfs.block.size in $HADOOP_HOME/conf/hdfs-site.xml. I understand that when files are created, blocksizes can be modified from default. What happens if you ...
    J. Ryan EarlJ. Ryan Earl
    Jun 6, 2011 at 7:10 pm
    Jun 6, 2011 at 10:05 pm
  • Hello! I'm using the hadoop version from cloudera hadoop-core-0.20.2-cdh3u1-SNAPSHOT.jar. Today I've made a mistake. I have deleted my user from HDFS with the command hadoop fs -rmr /user/my_user No ...
    Florin PFlorin P
    Jun 8, 2011 at 8:51 am
    Jun 10, 2011 at 6:49 am
  • Synopsis: * After shutting down a datanode in a cluster, fsck declares CORRUPT with missing blocks, * I restore/restart the datanode and fsck soon declares things healthy * But dfsadmin -report says ...
    Robert J BergerRobert J Berger
    Jun 8, 2011 at 5:39 pm
    Jul 9, 2011 at 3:00 am
  • Is there a way to unzip a gzip file within HDFS where source & target both live on HDFS? I don't want to pull a large file to local and put it back. -Ayon See My Photos on Flickr Also check out my ...
    Ayon SinhaAyon Sinha
    Jun 17, 2011 at 7:30 am
    Jun 17, 2011 at 5:29 pm
  • I have a datanode with a ~900GB hard drive in it: Filesystem Size Used Avail Use% Mounted on /dev/hda1 878G 384G 450G 47% / But the NameNode GUI shows 2.57TB: Node Last Contact Admin State Configured ...
    Time LessTime Less
    Jun 13, 2011 at 7:02 pm
    Jun 16, 2011 at 11:34 pm
  • Dear all, I'm looking for ways to improve the namenode heap size usage of a 800-node 10PB testing Hadoop cluster that stores around 30 million files. Here's some info: 1 x namenode: 32GB RAM, 24GB ...
    SiuonSiuon
    Jun 10, 2011 at 11:46 am
    Jun 10, 2011 at 8:16 pm
  • Hi list, How does the NN place blocks on the disks within a single node? Does it spread out adjecent blocks of a single file horizontally over the disks? For example, lets say I have four DN's and ...
    Evert LammertsEvert Lammerts
    Jun 30, 2011 at 1:04 pm
    Jul 1, 2011 at 8:30 am
  • Sorry for sending this email again but I got no answers from the first one. Anyone please help or forward it to mail-list that would help. 2011-06-15 *********************************************** * ...
    Hailong.yang1115Hailong.yang1115
    Jun 15, 2011 at 3:00 am
    Jun 29, 2011 at 7:20 am
  • Are there any glaring culprits to check for errors like this: receiving incremental file list rsync: failed to set times on "/mnt/hdfs/user/hadoop/reports_new": Input/output error (5) rsync: failed ...
    J. Ryan EarlJ. Ryan Earl
    Jun 10, 2011 at 2:07 am
    Jun 10, 2011 at 3:00 am
  • Hello, I am trying to start a DataNodeCluster using the following command: bin/hadoop-daemon.sh start org.apache.hadoop.hdfs.DataNodeCluster -simulated -n $DATANODE_PER_HOST -inject ...
    Elise GaleElise Gale
    Jun 1, 2011 at 7:25 pm
    Jun 3, 2011 at 7:25 pm
  • Hi I want to use nfs to save a copy of the data in namenode, and the nfs dir /mnt/nfs/dfs/ can be read and written. Things seem to be right, but when i format the hdfs, "ERROR common.Storage: Cannot ...
    吕鹏吕鹏
    Jun 26, 2011 at 8:34 am
    Jun 26, 2011 at 5:47 pm
  • Hi all, I wanna ask question, Is there a reason why block size should be set to some 2^N, for some integer N ? Does it help with block defragmentation etc. ? Thanks in advance..
    SnedixSnedix
    Jun 17, 2011 at 7:55 pm
    Jun 18, 2011 at 1:56 am
  • Hello, I'm trying to nail down a process for converting existing Apache-hadoop clusters with significant amounts of pre-existing data to CDH3. While I've found documentation for upgrading between CDH ...
    J. Ryan EarlJ. Ryan Earl
    Jun 17, 2011 at 9:09 pm
    Jun 17, 2011 at 9:23 pm
  • Hello! I've read about mapred.tasktracker.map.tasks.maximum that represents the maximum number of map tasks that will be run simultaneously by a task tracker. Given the following scenario: 1. You ...
    Florin PFlorin P
    Jun 13, 2011 at 6:50 pm
    Jun 13, 2011 at 7:16 pm
  • What is the connection between the throughput value from TestDFSIO and the performance of hadoop cluster? And how to explain the write and read value? Is the high throughput can guarantee that the ...
    SnedixSnedix
    Jun 4, 2011 at 10:04 pm
    Jun 4, 2011 at 10:11 pm
  • Hello I'm trying to compile HDFS/FUSE for mountable HDFS. When I go through the documented process @ http://wiki.apache.org/hadoop/MountableHDFS I get: BUILD FAILED ...
    J. Ryan EarlJ. Ryan Earl
    Jun 2, 2011 at 9:34 pm
    Jun 3, 2011 at 9:09 am
  • Hi, Every day each map/reduce processes I schedule on my cluster leave files behind on all the DataNodes in a directory named blocksBeingWritten. After 1 week the amount of files left behind reach 70 ...
    Jean-Pierre OCALANJean-Pierre OCALAN
    Jun 30, 2011 at 6:38 pm
    Jun 30, 2011 at 6:38 pm
  • Aloha, We are experimenting with Hadoop and I have setup a basic cluster and now trying to integrate it into our environment. Our cluster is up and accessible via the web tools and command line. ...
    Herb WoodruffHerb Woodruff
    Jun 29, 2011 at 10:27 pm
    Jun 29, 2011 at 10:27 pm
  • I am trying to start up the facebook version of hadoop at here https://github.com/facebook/hadoop-20-warehouse#readme where the AvartaNode code is. I notice that the AvartaNode is trying to connect ...
    W S ChungW S Chung
    Jun 29, 2011 at 9:35 pm
    Jun 29, 2011 at 9:35 pm
  • Hello All, As you all know, tomorrow is the Hadoop Summit 2011. There will be many interesting talks tomorrow. Don't miss any talk if you want to see how long Hadoop progressed. Link: ...
    Bharath MundlapudiBharath Mundlapudi
    Jun 28, 2011 at 7:44 pm
    Jun 28, 2011 at 7:44 pm
  • Hello. I am using Hadoop 0.21 I can see that if data node receives some IO error, this can cause checkDir storm. What I mean: 1) any error produces DataNode.checkDiskError call 2) this call locks ...
    Віталій ТимчишинВіталій Тимчишин
    Jun 20, 2011 at 10:50 am
    Jun 20, 2011 at 10:50 am
  • Hi, I'm running an MR application that produces an output that is saved in HDFS. My application has 5 slave nodes (so it has also 5 data nodes). The hdfs file replication factor is 1. I want from my ...
    Pedro CostaPedro Costa
    Jun 15, 2011 at 7:22 pm
    Jun 15, 2011 at 7:22 pm
  • hi,all I'd like to try avatarnode, and I have download facebook-hadoop from https://github.com/facebook/hadoop-20-warehouse using git clone git://github.com/facebook/hadoop-20-warehouse.git How can I ...
    MattDeansMattDeans
    Jun 14, 2011 at 3:35 am
    Jun 14, 2011 at 3:35 am
  • Hi,all I have a namenode and a backupnode ( two seperate host), and Datanodes using VIP to communicate with namenode. When namenode failed , backupnode will get the vip. But how to make BackupNode ...
    MattDeansMattDeans
    Jun 10, 2011 at 5:59 am
    Jun 10, 2011 at 5:59 am
  • Hi,all I read hadoop 0.21.0/hdfs_user_guide.htm , and did a little google, but I cannot found how to setup and configure a BackupNode. Can anybody tell me how to setup and configure a BackupNode? ...
    MattDeansMattDeans
    Jun 10, 2011 at 2:51 am
    Jun 10, 2011 at 2:51 am
  • Hello! I've been looking around for secondary index support in HBASE. I've found two requests in Jira: https://issues.apache.org/jira/browse/HBASE-3340 ...
    Florin PFlorin P
    Jun 9, 2011 at 9:32 am
    Jun 9, 2011 at 9:32 am
  • Hi, Im a postgraduate student and as a part of a project i'm having at the university I'm trying to run an additional datanode in cloudera distribution, but it seems that no matter what i do, i only ...
    Andreas KondylisAndreas Kondylis
    Jun 4, 2011 at 5:36 pm
    Jun 4, 2011 at 5:36 pm
  • Hi, I have a question regarding using sequence file input format in hadoop streaing jar with mappers and reducers written in python. If i use sequence file as input format for streaming jar and use ...
    Mapred LearnMapred Learn
    Jun 2, 2011 at 7:06 am
    Jun 2, 2011 at 7:06 am
Group Navigation
period‹ prev | Jun 2011 | next ›
Group Overview
grouphdfs-user @
categorieshadoop
discussions29
posts101
users41
websitehadoop.apache.org...
irc#hadoop

41 users for June 2011

Harsh J: 11 posts Florin P: 9 posts Joey Echeverria: 7 posts J. Ryan Earl: 7 posts Marcos Ortiz: 6 posts Ayon Sinha: 5 posts Pedro Costa: 5 posts Robert J Berger: 4 posts Allen Wittenauer: 3 posts Jain, Prem: 3 posts MattDeans: 3 posts Snedix: 3 posts Elise Gale: 2 posts Hailong.yang1115: 2 posts Jeff Bean: 2 posts Philip Zeyliger: 2 posts Sridhar basam: 2 posts Time Less: 2 posts Aaron Eng: 1 post Andreas Kondylis: 1 post
show more