FAQ

Hadoop > hdfs-user > January 2013

Search Discussions

207 discussions - 838 posts

  • I am sure you will suffer from severe stomach ache after watching this :) http://www.youtube.com/watch?v=hEqQMLSXQlY Warm Regards, Tariq https://mtariq.jux.com/ cloudfront.blogspot.com
    Mohammad TariqMohammad Tariq
    Jan 17, 2013 at 8:06 pm
    Jan 20, 2013 at 12:25 pm
  • Hi, I have recently installed hadoop-1.0.4 on a linux machine. Whilst working through the post-install instructions contained in the “Quick Start” guide, I incurred the following catastrophic Java ...
    Sean HudsonSean Hudson
    Jan 17, 2013 at 2:51 pm
    Jan 25, 2013 at 1:08 pm
  • Hello, I was estimating how much disk space do I need for my cluster. I have 24 million JSON documents approx. 5kb each the Json is to be stored into HBASE with some identifying data in coloumns and ...
    Panshul WhisperPanshul Whisper
    Jan 18, 2013 at 12:11 pm
    Jan 19, 2013 at 3:40 am
  • Hello, Is there a standard way to prevent the failure of Namenode crash in a Hadoop cluster? or what is the standard or best practice for overcoming the Single point failure problem of Hadoop. I am ...
    Panshul WhisperPanshul Whisper
    Jan 15, 2013 at 1:25 am
    Jan 17, 2013 at 1:39 pm
  • Hello all, I'd like to pick the community brain on average throughput speeds for a moderately specced 4-node Hadoop cluster with 1GigE networking. Is it reasonable to expect constant average speeds ...
    Artem ErvitsArtem Ervits
    Jan 3, 2013 at 10:01 pm
    Jan 4, 2013 at 4:15 pm
  • Hello! I've found that jobcache directory became very large on our cluster, e.g.: # du -sh /data?/mapred/local/taskTracker/user/jobcache 465G /data1/mapred/local/taskTracker/user/jobcache 464G ...
    Ivan TretyakovIvan Tretyakov
    Jan 9, 2013 at 2:42 pm
    Jan 17, 2013 at 9:53 am
  • I am using CDH4 (2.0.0-mr1-cdh4.1.2) vm running on my mbp. I was trying to invoke a remote method in the ClientProtocol via RPC, however I am getting this exception. 2013-01-29 11:20:45,810 ERROR ...
    Kim ChewKim Chew
    Jan 29, 2013 at 8:34 pm
    Jan 30, 2013 at 5:11 pm
  • Moving from general@ to user@. The general list is not for technical questions, it's to discuss project-wide issues. On Tue, Jan 22, 2013 at 1:03 PM, Mohit Vadhera wrote: If your Namenode filesystem ...
    Andy IsaacsonAndy Isaacson
    Jan 22, 2013 at 9:08 pm
    Jan 23, 2013 at 5:54 pm
  • Hi all in this list! My name is Cristián Carranza, a statistician and quality consultant that for the second time, intends to learn Hadoop and Big Data related issues. I’am requesting advice in order ...
    Cristian CarranzaCristian Carranza
    Jan 4, 2013 at 4:31 pm
    Jan 4, 2013 at 9:46 pm
  • Melody FleishauerMelody Fleishauer
    Jan 8, 2013 at 3:54 pm
    Jan 17, 2013 at 4:38 pm
  • Hi, Is it possible to keep 1 Petabyte in a single data node? If not, How much is the maximum storage for a particular data node? Regards, M. Jeba
    Jeba earnestJeba earnest
    Jan 30, 2013 at 9:00 am
    Jan 30, 2013 at 1:38 pm
  • how to remove non dfs space from hadoop cluster -- * * * Thanx and Regards* * Vikas Jadhav*
    Vikas JadhavVikas Jadhav
    Jan 16, 2013 at 7:47 am
    Jan 17, 2013 at 2:57 pm
  • I have 1 hadoop master which name node locates and 2 slave which datanode locate. If i choose a small data like 200M, it can be done. But if i run 30G data, Map is done. But the reduce report error ...
    YaotianYaotian
    Jan 11, 2013 at 3:24 am
    Jan 16, 2013 at 3:24 am
  • Hi, hadoop can skip bad records http://devblog.factual.com/practical-hadoop-streaming-dealing-with-brittle-code. But it is also possible to skip entire tasks? -Håvard -- Håvard Wahl Kongsgård Faculty ...
    Håvard Wahl KongsgårdHåvard Wahl Kongsgård
    Jan 5, 2013 at 9:54 am
    Jan 8, 2013 at 4:41 am
  • Hello, I am using hadoop with TextInputFormat, a mapper and no reducers. I am running my jobs at Amazon EMR. When I run my job, I set both following options ...
    Marcelo Elias Del ValleMarcelo Elias Del Valle
    Jan 28, 2013 at 3:54 pm
    Jan 29, 2013 at 8:08 pm
  • Hi, Lets say I have the standard helloworld program http://hadoop.apache.org/docs/r0.17.0/mapred_tutorial.html#Example%3A+WordCount+v2.0 Now, lets say, I want to start the counting not from zero but ...
    Jamal sashaJamal sasha
    Jan 21, 2013 at 11:52 pm
    Jan 22, 2013 at 7:59 am
  • Hello list, I have a pseudo distributed setup on my laptop. Everything was working fine untill now. But lately HDFS has started taking a lot of time to leave the safemode. Infact, I have to it ...
    Mohammad TariqMohammad Tariq
    Jan 19, 2013 at 10:36 pm
    Jan 20, 2013 at 1:05 pm
  • Hi guys: I've finally extracted my problem of loading a special filesystem into a unit test. Below, clearly, Im creating a raw configuration and adding a single resource to it (core-site.xml) ...
    Jay VyasJay Vyas
    Jan 17, 2013 at 12:33 am
    Jan 29, 2013 at 1:59 am
  • Hi, I am not sure what I am doing wrong. I copy my input files from local to hdfs at local /user/hduser/data/input1.txt /user/hduser/data/input2.txt In my driver code: I have ...
    Jamal sashaJamal sasha
    Jan 17, 2013 at 11:57 pm
    Jan 18, 2013 at 4:12 am
  • Hello, I have a hadoop cluster setup of 10 nodes and I an in need of implementing queues in the cluster for receiving high volumes of data. Please suggest what will be more efficient to use in the ...
    Panshul WhisperPanshul Whisper
    Jan 11, 2013 at 3:12 am
    Jan 11, 2013 at 3:06 pm
  • Hi, I have data in json format like: {key:[values.....]} key, values are longints. Now, I want to do a fast lookup of a key. How would I implement a binary search in map reduce abstraction. Or am i ...
    Jamal sashaJamal sasha
    Jan 7, 2013 at 11:21 pm
    Jan 8, 2013 at 9:15 am
  • Hello, I'd like to use more than one reduce task with Hadoop Streaming and I'd like to have only one result. Is it possible? Or should I run one more job to merge the result? And is it the same with ...
    Pavel HančarPavel Hančar
    Jan 3, 2013 at 9:11 pm
    Jan 5, 2013 at 2:32 pm
  • Hi, My log files are generated and saved in a windows machine. Now I have to move those remote files to the Hadoop cluster (HDFS) either in synchronous or asynchronous way. I have gone through flume ...
    Mahesh BalijaMahesh Balija
    Jan 17, 2013 at 10:03 am
    Jan 21, 2013 at 5:17 am
  • Hi, I'm following the Hadoop 1.1.1 tutorial (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and I have these questions: 1.) The Standalone Operation ...
    Glen MazzaGlen Mazza
    Jan 17, 2013 at 1:55 pm
    Jan 19, 2013 at 5:49 pm
  • Looks like weight for both pools is equal and all map slots are used. Therefore I don't believe anyone has priority for the next slots. Try setting research weight to 2. This should allow research to ...
    Justin WorkmanJustin Workman
    Jan 16, 2013 at 4:09 pm
    Jan 17, 2013 at 6:36 pm
  • hi, After I compiled libhdfs, the version of hadoop become 1.0.4-SNAPSHOT, is this a stable version or a dev version? Can I use this version on production? If this version is not for production, how ...
    Xibin LiuXibin Liu
    Jan 28, 2013 at 6:36 am
    Jan 29, 2013 at 2:19 am
  • Hi all. I'm writing to you to ask for advice or a hint to the right direction. In our department, more and more researchers ask us (IT administrators) to assemble (or to buy) GPGPU powered ...
    Roberto NunnariRoberto Nunnari
    Jan 17, 2013 at 8:24 am
    Jan 28, 2013 at 10:34 pm
  • Hi, I would like to report a bug . I get negative value of function unix_timestamp. I want to get a duration of format HH:mm:ss unix_timestamp(duration,'HH:mm:ss') When I test it backwards ,it works ...
    Carolina Vizuete MartinezCarolina Vizuete Martinez
    Jan 15, 2013 at 9:38 am
    Jan 28, 2013 at 3:17 am
  • Hello, I was wondering if anyone is using spring for hadoop to execute map reduce jobs or to perform hbase operations on a hadoop cluster using spring data for hadoop. Please suggest me a working ...
    Panshul WhisperPanshul Whisper
    Jan 18, 2013 at 10:34 pm
    Jan 25, 2013 at 2:40 am
  • I want to load a SegY <http://en.wikipedia.org/wiki/SEG_Y file onto HDFS of a 3-node Apache Hadoop cluster. To summarize, the SegY file consists of : 1. 3200 bytes *textual header* 2. 400 bytes ...
    Kaliyug AntagonistKaliyug Antagonist
    Jan 16, 2013 at 3:32 pm
    Jan 22, 2013 at 6:04 pm
  • Since this is a Hadoop question, it should be sent <span class="m_body_email_addr" title="858a0c8e479a78c1038b7355244ec07c" user@hadoop.apache.org</span (which I'm now sending this to and I put ...
    Jean-Daniel CryansJean-Daniel Cryans
    Jan 17, 2013 at 6:04 pm
    Jan 17, 2013 at 9:52 pm
  • Hello, Is it possible or how is it possible to query mongodb directly from hadoop. Or is it possible to query hbase or json files stored in hdfs in a similar way as we can query the json documents in ...
    Panshul WhisperPanshul Whisper
    Jan 16, 2013 at 10:20 am
    Jan 16, 2013 at 3:33 pm
  • Hi ESGLinux, In production, you need to run QJM on at least 3 nodes. You also need to run ZKFC on at least 3 nodes. You can run them on the same nodes if you like, though. Of course, none of this is ...
    Colin McCabeColin McCabe
    Jan 14, 2013 at 7:50 pm
    Jan 15, 2013 at 10:18 am
  • We are somewhat new to Hadoop and are looking to run some experiments with HDFS, Pig, and HBase. With that in mind, I have a few questions: What is the easiest (preferably free) Hadoop distro to get ...
    John LilleyJohn Lilley
    Jan 11, 2013 at 10:29 am
    Jan 11, 2013 at 11:31 am
  • For example,look at the documents about HDFS shell guide: In 0.17, the prefix of HDFS shell is hadoop dfs: http://hadoop.apache.org/docs/r0.17.2/hdfs_shell.html In 0.19, the prefix of HDFS shell is ...
    javaLeejavaLee
    Jan 8, 2013 at 11:35 am
    Jan 11, 2013 at 4:31 am
  • Hi all, Could you tell me how Hadoop decide tha capacity of each datanodes? I've installed CDH3 on 2 VM machine, each VM has 100G space, And I found that Hadoop occupied 50G/100G, why? Thanks. Best ...
    DoraDora
    Jan 9, 2013 at 1:43 pm
    Jan 9, 2013 at 4:40 pm
  • Actually, those instructions are for Hadoop 0.24, not 2.0.2-alpha. Glen -- Glen Mazza Talend Community Coders - coders.talend.com blog: www.jroller.com/gmazza
    Glen MazzaGlen Mazza
    Jan 4, 2013 at 11:46 am
    Jan 4, 2013 at 11:44 pm
  • Hi all, I want to run a MapReduce job using the Hadoop Java api from my analytics server. It is not the master or even a data node but it has the same Hadoop installation as all the nodes in the ...
    Amit SelaAmit Sela
    Jan 24, 2013 at 12:14 pm
    Jan 27, 2013 at 12:09 pm
  • Folks, Its been an year and my HDFS / Solar /Hive setup is working flawless. The data logs which were meaningless to my business all of a sudden became precious to the extent that our management ...
    Steve EdisonSteve Edison
    Jan 24, 2013 at 11:29 pm
    Jan 25, 2013 at 7:45 am
  • hi all, I found hdfs du periodicity(one hour), and because my disk is big, the smallest one is 15T, so when hdfs exec du, datanode will not respond for about 3 minuts because of io loading, this ...
    Xibin LiuXibin Liu
    Jan 24, 2013 at 7:21 am
    Jan 25, 2013 at 2:35 am
  • Dear all, I know it is best to use small amount of mem in mapper and reduce. However, sometimes it is hard to do so. For example, in machine learning algorithms, it is common to load the model into ...
    Shaojun ZhaoShaojun Zhao
    Jan 18, 2013 at 8:06 pm
    Jan 18, 2013 at 10:55 pm
  • Hi, looking at the derivation of the 0.23.x & 2.0.x branches on one hand, and the 1.x branches on the other, as described here ...
    Glen MazzaGlen Mazza
    Jan 18, 2013 at 2:49 pm
    Jan 18, 2013 at 8:38 pm
  • I have a mapper public class BuildGraph{ public void config(JobConf job){ *<==this block doesnt seems to be exexcuting at all :(* super.configure(job); this.currentId = job.getInt("currentId",0); if ...
    Jamal sashaJamal sasha
    Jan 16, 2013 at 1:25 am
    Jan 16, 2013 at 12:46 pm
  • Hi I'm using Hadoop 1.0.4 and using the hadoop.mapreduce API having problems compiling a simple class to implement a reduce-side data join of 2 files. I'm trying to do this using contrib.utils.join ...
    Michael ForageMichael Forage
    Jan 11, 2013 at 3:32 pm
    Jan 15, 2013 at 5:29 pm
  • Hello, I have a hadoop cluster of 5 nodes with a total of available HDFS space 130 GB with replication set to 5. I have a file of 115 GB, which needs to be copied to the HDFS and processed. Do I need ...
    Panshul WhisperPanshul Whisper
    Jan 11, 2013 at 3:03 am
    Jan 11, 2013 at 6:13 am
  • Hi al, I have a java application jar that converts some files and writes directly into hdfs. If I want to run the jar I need to run it using "hadoop jar <application jar ", so that it can access HDFS ...
    Krishna RaoKrishna Rao
    Jan 4, 2013 at 2:41 pm
    Jan 7, 2013 at 11:21 am
  • Oozie question I'm trying to run an Oozie workflow (sqoop action) from the Hue console and it fails every time. No exception in the oozie log but I see this in the Job Tracker log file. Two primary ...
    Corbett MartinCorbett Martin
    Jan 29, 2013 at 11:30 pm
    Jan 30, 2013 at 9:07 pm
  • Hi All, I am trying to override the value of mapred.tasktracker.map.tasks.maximum in mapred-site.xml through my job configuration object will it reflect in the corresponding task trackers? or do i ...
    Manoj BabuManoj Babu
    Jan 28, 2013 at 5:07 am
    Jan 28, 2013 at 4:24 pm
  • Hi guys, I have a quick question regarding to fire scheduler of Hadoop, I am reading this article = http://blog.cloudera.com/blog/2008/11/job-scheduling-in-hadoop/, my question is from the following ...
    Lin MaLin Ma
    Jan 20, 2013 at 2:26 pm
    Jan 26, 2013 at 3:38 pm
  • Hi all, I've been asked to check whether is possible to use certificates to secure the connection between Hadoop and Oozie and the "external world" or not. Case is this: We have to develop a ...
    Fabio PitzoluFabio Pitzolu
    Jan 22, 2013 at 3:20 pm
    Jan 22, 2013 at 5:54 pm
Jan 2013
12345
Group Navigation
period‹ prev | Jan 2013 | next ›
Group Overview
grouphdfs-user @
categorieshadoop
discussions207
posts838
users238
websitehadoop.apache.org...
irc#hadoop

238 users for January 2013

Harsh J: 86 posts Mohammad Tariq: 45 posts Hemanth Yamijala: 26 posts Jamal sasha: 22 posts Jean-Marc Spaggiari: 21 posts Panshul Whisper: 21 posts John Lilley: 17 posts Nitin Pawar: 16 posts Vikas Jadhav: 14 posts Mahesh Balija: 13 posts Vinod Kumar Vavilapalli: 13 posts Bejoy Hadoop: 11 posts Glen Mazza: 11 posts Ivan Tretyakov: 11 posts Jay Vyas: 11 posts Yaotian: 9 posts Artem Ervits: 8 posts Michel Segel: 8 posts Samir das mohapatra: 8 posts Shashwat Shriparv: 8 posts
show more
Archives