FAQ

Hadoop > hdfs-user > December 2012

Search Discussions

133 discussions - 590 posts

  • Hi, Someone wants to give me (OK, sell, but for the cheap) a 48-node cluster of decent blades, a few years old. Should I even thy to think of what to do with them, given that one can use dedicated ...
    Mark KerznerMark Kerzner
    Dec 20, 2012 at 3:17 am
    Dec 23, 2012 at 2:49 am
  • Hello community, I am trying to use hadoop 1.1.0 on a SLES 11 (zLinux) running on IBM S390. The java provided is "java-s390x-60" 64Bit. While trying to format the namenode I got the following error ...
    Emile KaoEmile Kao
    Dec 11, 2012 at 2:23 pm
    Dec 13, 2012 at 1:00 pm
  • Hi guys, I want to confirm when on each task node either mapper or reducer access distributed cache file, it resides on disk, not resides in memory. Just want to make sure distributed cache file does ...
    Lin MaLin Ma
    Dec 22, 2012 at 12:04 pm
    Dec 28, 2012 at 10:02 am
  • Hello, I'm Steve who need some raw big data for studying mapreduce programming. Where can i find them? especially those about weblog, traffic info etc. My English is not so well, if you can give me a ...
    Yin SteveYin Steve
    Dec 7, 2012 at 3:43 pm
    Dec 11, 2012 at 4:04 am
  • Some reasons lead to my name node data error, but the error data also overwrite the second name node data, also the NFS backup. I want to recover the name node data a day ago or even a week ago,but I ...
    周梦想周梦想
    Dec 20, 2012 at 9:48 am
    Dec 26, 2012 at 2:17 pm
  • Which one is current .. What is the preferred way to pass a small number of configuration parameters to a mapper or reducer? *A. *As key-value pairs in the jobconf object. * * *B. *As a custom input ...
    Kshiva KpsKshiva Kps
    Dec 28, 2012 at 6:12 pm
    Dec 30, 2012 at 7:39 pm
  • Hi All, I´m going to test a hadoop cluster and I have a doubt about HA and Federation. With federation I Have a NameNode per namespace and with HA I have an Active NameNode and a standby NameNode ...
    ESGLinuxESGLinux
    Dec 20, 2012 at 4:31 pm
    Dec 21, 2012 at 8:38 am
  • Has anyone experienced a TaskTracker/DataNode behaving like the attached image? This was during a MR job (which runs often). Note the extremely high System CPU time. Upon investigating I saw that out ...
    Robert DyerRobert Dyer
    Dec 9, 2012 at 12:09 am
    Dec 18, 2012 at 8:43 am
  • I've got a job that reads in 167 files from S3, but 2 of the files are being mapped twice and 1 of the files is mapped 3 times. This is the code I use to set up the mapper: Path lsDir = new ...
    David ParksDavid Parks
    Dec 6, 2012 at 6:16 am
    Dec 7, 2012 at 4:41 am
  • What's the best way to trigger alert when jobs run for too long or have many failures? Is there a hadoop command that can be used to perform this activity?
    Mohit AnchliaMohit Anchlia
    Dec 22, 2012 at 8:31 pm
    Dec 24, 2012 at 7:59 am
  • Hi , I want to use the distributed cache to allow my mappers to access data. In main, I'm using the command DistributedCache.addCacheFile(new URI("/user/peter/cacheFile/testCache1"), conf); Where ...
    Peter CoganPeter Cogan
    Dec 6, 2012 at 5:00 pm
    Dec 11, 2012 at 10:09 am
  • Hiii, I am new in hadoop. Trying to configure it in fully distributed mode. But after the command bin/start-all.sh or bin/start-mapred.sh or bin/hadoop-daemon.sh start tasktracker, TASKTRACKER IS ...
    Dibyendu KarmakarDibyendu Karmakar
    Dec 4, 2012 at 10:25 am
    Dec 6, 2012 at 2:33 am
  • Hello list, I don't know if this question makes any sense, but I would like to ask, does it make sense to store 500TB (or more) data in a single DN?If yes, then what should be the spec of other ...
    Mohammad TariqMohammad Tariq
    Dec 12, 2012 at 3:03 pm
    Dec 13, 2012 at 3:19 pm
  • Hi All, I am getting the exception as below but the job continues to running how to avoid this and whats the cause for the issue? 12/12/06 23:30:56 INFO mapred.JobClient: Task Id ...
    Manoj BabuManoj Babu
    Dec 7, 2012 at 5:39 am
    Dec 7, 2012 at 3:37 pm
  • Hi everybody, I have a M/R job which does a bulk import to hbase. I have to process many gzip files (2800 x ~ 100mb) I don't understand why my job instanciates 80 maps but runs each map sequentialy ...
    X6i4uybz labsX6i4uybz labs
    Dec 5, 2012 at 4:24 pm
    Dec 6, 2012 at 4:53 pm
  • Hi all, I have a doubt about how to deploy the Zookeeper in a NN HA cluster, As far as I know, I need at least three nodes to run three ZooKeeper FailOver Controller (ZKFC). I plan to put these 3 ...
    ESGLinuxESGLinux
    Dec 27, 2012 at 12:04 pm
    Dec 28, 2012 at 11:02 am
  • Is it possible to merge files from different locations from HDFS location into one file into HDFS location?
    Mohit AnchliaMohit Anchlia
    Dec 22, 2012 at 6:41 am
    Dec 23, 2012 at 3:31 pm
  • Hi all, is it necessary to run secondary namenode when starting HDFS? I am dealing with Hadoop 1.1.1. Looking at script $HADOOP_HOME/bin/start_dfs.sh There are next lines in this file: # start dfs ...
    Ivan RyndinIvan Ryndin
    Dec 17, 2012 at 5:05 pm
    Dec 17, 2012 at 7:00 pm
  • Hello list, I have a tool (written in C) that performs some different types of operations and can be used as a command line utility. I had to write a similar tool, as we have moved towards Hadoop ...
    Mohammad TariqMohammad Tariq
    Dec 3, 2012 at 12:07 pm
    Dec 3, 2012 at 3:57 pm
  • I'm pretty consistently seeing a few reduce tasks fail with OutOfMemoryError (below). It doesn't kill the job, but it slows it down. In my current case the reducer is pretty darn simple, the ...
    David ParksDavid Parks
    Dec 22, 2012 at 4:34 am
    Dec 25, 2012 at 7:32 am
  • Why does one need to build an app from source if one can download gzip file and gunzip and use the app. Why git, why check out... What's considered "building"? I have been exposed to bits and pieces ...
    Kartashov, AndyKartashov, Andy
    Dec 18, 2012 at 3:56 pm
    Dec 21, 2012 at 11:17 pm
  • Hello, I run “mvn test” on hadoop-2.02-alpha’s source code checked out from the SVN. It could get through the compilation, but there are some test failures. I thought there should be no test failures ...
    Mark YangMark Yang
    Dec 18, 2012 at 9:12 am
    Dec 21, 2012 at 11:12 pm
  • Hey, everone. Just got finished reading about all of the unsubscribe messages in Sept-Oct, and was hoping someone has a clue about what my system is doing wrong. I suspect that this is a ...
    Robert RappleanRobert Rapplean
    Dec 18, 2012 at 10:43 pm
    Dec 19, 2012 at 4:39 pm
  • I am going through the tutorial presented @ http://developer.yahoo.com/hadoop/tutorial/module3.html#vm-jobs I have installed vmware and hadoop virtual machine. This tutorial talks about hadoop 0.18.0 ...
    Jennifer LopezJennifer Lopez
    Dec 9, 2012 at 5:24 pm
    Dec 11, 2012 at 10:28 am
  • Hi all, Hadoop have different file system(HDFS) and Cassandra have different file system(CFS). As Hadoop have great Eco-System (Hive{Dataware House}, Hbase{Data Base} n etc..) and Cassandra(Database) ...
    Yogesh dhariYogesh dhari
    Dec 6, 2012 at 4:55 pm
    Dec 6, 2012 at 7:43 pm
  • Hello guys, I need a counter for shuffled bytes to the mappers. Is there existing one or should I define one myself ? How can I implement such a counter? Thank you and happy Christmas time, Eduard
    Eduard SkaleyEduard Skaley
    Dec 25, 2012 at 12:34 pm
    Dec 26, 2012 at 12:56 pm
  • Hi : Do data nodes, when they receive write requests, forward the request to the next datanode in the chain after appending to a file? Or rather, do the write and forward asynchronously and at the ...
    Jay VyasJay Vyas
    Dec 18, 2012 at 8:57 pm
    Dec 18, 2012 at 10:32 pm
  • After all my R&D, I have setup hadoop 0.22.0 succesfully. Right now, I am using Eclipse Indigo Service Release 2 and hadoop 0.22.0 on win 7. Trying to use the eclipse plugin provided in the Hadoop ...
    Jennifer LopezJennifer Lopez
    Dec 13, 2012 at 10:16 am
    Dec 14, 2012 at 6:07 am
  • Hi all, I downloaded Hadoop-1.1.1 tar ball from one of the mirrors and configured it in psuedo-distributed mode. Namenode starts fine but datanode fails to start because of version mismatch. The ...
    Mark GroverMark Grover
    Dec 13, 2012 at 6:33 am
    Dec 14, 2012 at 1:12 am
  • Hi, I would like to check if there is any information about release dates for new versions, we are starting a new project that would be ready in some months and we need to decide which hadoop version ...
    Hernán LeoniHernán Leoni
    Dec 12, 2012 at 1:29 pm
    Dec 13, 2012 at 12:32 pm
  • Thank you all for the comments. that don't require such strictness I just want to make sure one thing. If I write my own scheduler, is it possible to do "strict" scheduling ? Thanks
    Hiroyuki YamadaHiroyuki Yamada
    Dec 1, 2012 at 10:41 am
    Dec 8, 2012 at 1:19 pm
  • http://hadoop.apache.org/docs/r1.0.4/hdfs_user_guide.html#Backup+Node the document write: The Backup node is configured in the same manner as the Checkpoint node. It is started with bin/hdfs namenode ...
    周梦想周梦想
    Dec 29, 2012 at 3:14 am
    Dec 29, 2012 at 7:15 am
  • Hi guys, Supposing in a Hadoop job, there are both mappers and reducers. My question is, reducer tasks cannot begin until all mapper tasks complete? If so, why designed in this way? thanks in ...
    Lin MaLin Ma
    Dec 22, 2012 at 1:26 pm
    Dec 23, 2012 at 3:10 pm
  • Hi The command, bin/hadoop jar -v hadoop-examples-0.20.203.0.jar grep input output 'dfs[a-z]+' returns without error in Cygwin. However the output directory is not created....Any help in identifying ...
    Ramachandran VilayannurRamachandran Vilayannur
    Dec 21, 2012 at 5:49 am
    Dec 22, 2012 at 9:34 am
  • Hi Jon, Thanks for sharing these insights! Can't agree with you more! Recently we released a tool called Starfish Hadoop Log Analyzer for analyzing the job histories. I believe it can quickly point ...
    Jie LiJie Li
    Dec 14, 2012 at 1:46 am
    Dec 21, 2012 at 1:22 am
  • Hi i am following this tutorial for installing hadoop alpha 2.0.2 on ubuntu 12.10 and am sure home variables for java and hadoop are properly configured ...
    Anand sharmaAnand sharma
    Dec 14, 2012 at 3:48 pm
    Dec 15, 2012 at 12:56 am
  • I'm submitting unrelated jobs programmatically (using AWS EMR) so they run in parallel. I'd like to run an s3distcp job in parallel as well, but the interface to that job is a Tool, e.g ...
    David ParksDavid Parks
    Dec 14, 2012 at 4:39 am
    Dec 14, 2012 at 6:32 am
  • Stupid question for the day… I have a file created by a mahout job of the form: 0 [356:0.3481597,359:0.3481597,358:0.3481597,361:0.3481597,360:0.3481597] 8 ...
    Pat FerrelPat Ferrel
    Dec 12, 2012 at 1:01 am
    Dec 13, 2012 at 5:11 am
  • Hi All I got the below exception, Is the issue related to https://issues.apache.org/jira/browse/MAPREDUCE-1182 ? Am using CDH3U1 2012-12-10 06:22:39,688 FATAL org.apache.hadoop.mapred.Task ...
    Manoj BabuManoj Babu
    Dec 10, 2012 at 1:10 pm
    Dec 12, 2012 at 6:16 am
  • Hi, Is there a way for force the tasks from a MR job to run ONLY on the taskservers where the input split location is? I mean, on the taskdetails UI, I can see all my tasks (25), and some of them ...
    Jean-Marc SpaggiariJean-Marc Spaggiari
    Dec 7, 2012 at 6:08 pm
    Dec 9, 2012 at 1:49 am
  • Hi guys: Where and how does a Hadoop's record reader decide wether or not it needs to copy a file to local disk ? Clearly, since the InputSplit (which has meta data about file inputs) is the input to ...
    Jay VyasJay Vyas
    Dec 6, 2012 at 9:38 pm
    Dec 7, 2012 at 3:34 am
  • hi, all I's using Hadoop 1.2.0 , java version "1.7.0_05" When running my pig script , the worker always report this error, and the MR jobs run very slow. Increase the dfs.socket.timeout value does ...
    Haitao YaoHaitao Yao
    Dec 4, 2012 at 7:10 am
    Dec 4, 2012 at 7:55 pm
  • Hi, Is it possible to change hadoop configuration files such as core-site.xml and get the changes take effect without having to restart hadoop services? Thanks, Pankaj
    Pankaj GuptaPankaj Gupta
    Dec 3, 2012 at 11:59 pm
    Dec 4, 2012 at 11:13 am
  • Hi, I am getting following errors - 12/12/03 17:11:42 INFO mapred.JobClient: Task Id : attempt_201212031650_0001_m_000011_0, Status : FAILED Too many fetch-failures ...
    Ajay SrivastavaAjay Srivastava
    Dec 3, 2012 at 5:22 pm
    Dec 4, 2012 at 1:43 am
  • Hello, I was reading on the relationship between input splits and HDFS blocks and a question came up to me: If a logical record crosses HDFS block boundary, let's say block#1 and block#2, does the ...
    Jeff LIJeff LI
    Dec 2, 2012 at 10:03 pm
    Dec 3, 2012 at 6:51 am
  • Hi All, I'm trying to setup Hadoop Cluster using 4 machines[4 x Ubuntu 12.04 x_64]. Using the following doc: 1. http://titan.softnet.tuc.gr:8082/User:xenia/Page_Title/Hadoop_Cluster_Setup_Tutorial ...
    A GeekA Geek
    Dec 2, 2012 at 3:55 pm
    Dec 3, 2012 at 3:41 am
  • Hi, Is there a way to ask Hadoop to display its parameters? I have updated the property as followed: <property <name dfs.name.dir</name <value ${hadoop.tmp.dir}/dfs/name,/media/usb0/</value ...
    Jean-Marc SpaggiariJean-Marc Spaggiari
    Dec 1, 2012 at 2:12 am
    Dec 1, 2012 at 4:30 am
  • Hi, I have two different hdfs cluster. I need to transfer files between these environments. What's the fastest way to transfer files for that situation? I've researched about it. I found distcp ...
    BurakkkBurakkk
    Dec 28, 2012 at 8:01 pm
    Dec 29, 2012 at 10:29 am
  • please unsubscribe me from the mailing list. thanks.
    Harun Raşit ErHarun Raşit Er
    Dec 19, 2012 at 4:45 pm
    Dec 28, 2012 at 11:45 pm
  • Hi, So I am still in process of learning hadoop. I tried to run wordcount.java (by writing my own mapper reducer.. creating jar and then running it in a pseudo distributed mode). At that time I got ...
    Jamal sashaJamal sasha
    Dec 27, 2012 at 8:03 pm
    Dec 27, 2012 at 11:11 pm
Dec 2012
123
Group Navigation
period‹ prev | Dec 2012 | next ›
Group Overview
grouphdfs-user @
categorieshadoop
discussions133
posts590
users179
websitehadoop.apache.org...
irc#hadoop

179 users for December 2012

Harsh J: 71 posts Mohammad Tariq: 36 posts David Parks: 21 posts Jean-Marc Spaggiari: 16 posts ESGLinux: 13 posts Manoj Babu: 12 posts 周梦想: 12 posts Lin Ma: 11 posts Hemanth Yamijala: 10 posts Ted Dunning: 10 posts Mark Kerzner: 9 posts Michel Segel: 9 posts Nitin Pawar: 9 posts Jay Vyas: 8 posts Jennifer Lopez: 8 posts Mohit Anchlia: 8 posts Rishi Yadav: 8 posts Anand sharma: 7 posts Robert Molina: 7 posts Ivan Ryndin: 6 posts
show more
Archives