Search Discussions

205 discussions - 1,082 posts

  • Hi, I recently used a backup tool to back up all my HDFS data to S3. The data is on S3 in multiparts. I need to test the restore now. Could you please give me some pointers on how to test this. 1) Do ...
    Prem yadavPrem yadav
    Aug 8, 2012 at 1:21 pm
    Aug 18, 2012 at 5:36 am
  • hi all, how can i unsubscribe form this mailing list? thanks, Eyal Golan <span class="m_body_email_addr" title="b04a676714fc56e594d86d1078f8d8ca" egolan74@gmail.com</span Visit ...
    Eyal GolanEyal Golan
    Aug 8, 2012 at 10:56 am
    Aug 28, 2012 at 7:15 pm
  • Hi, i am just learning the Hadoop and i am setting the development environment with CDH3 pseudo distributed mode without any ssh cofiguration in CentOS 6.2 . i can run the sample programs as usual ...
    Anand sharmaAnand sharma
    Aug 9, 2012 at 10:16 am
    Aug 11, 2012 at 3:55 pm
  • Hi, i am currently trying to run my hadoop program on a cluster. Sadly though my datanodes and tasktrackers seem to have difficulties with their communication as their logs say: * Some datanodes and ...
    Björn-Elmar MacekBjörn-Elmar Macek
    Aug 13, 2012 at 12:39 pm
    Aug 20, 2012 at 10:16 am
  • Hi folks! I have just read about the HDFS RAID feature that was added to Hadoop 0.21 or 0.22. and I am quite curious to know if people use it, what kind of use they have and what they think about ...
    Sourygna LuangsaySourygna Luangsay
    Aug 8, 2012 at 4:46 pm
    Aug 9, 2012 at 10:56 am
  • I'm facing the exact same issue on 0.20.2-cdh3u0. Does anybody have an idea? Tnx. Best, Christoph -------------------------- Subject Re: pending clean up step Date Tue, 14 Feb 2012 13:17:04 GMT ...
    Aug 28, 2012 at 11:59 am
    Aug 29, 2012 at 10:51 pm
  • Hi, Moving this to the <span class="m_body_email_addr" title="858a0c8e479a78c1038b7355244ec07c" user@hadoop.apache.org</span lists. The general@ lists is only for project level discussions, not ...
    Harsh JHarsh J
    Aug 14, 2012 at 3:11 am
    Aug 16, 2012 at 2:10 pm
  • Hi, We have a hadoop cluster of version 0.20.2 in production. Now we have another new Hadoop cluster using cloudera's CDH3U4. We like to run distcp to copy files between the two clusters. Since the ...
    Jian FangJian Fang
    Aug 9, 2012 at 8:17 pm
    Aug 15, 2012 at 5:25 pm
  • I am very new to Hadoop. I am considering setting up a Hadoop cluster consisting of 5 nodes where each node has 3 internal hard drives. I understand HDFS has a configurable redundancy feature but ...
    Aji JanisAji Janis
    Aug 10, 2012 at 6:38 pm
    Aug 14, 2012 at 12:16 am
  • Hi, i have a question concerning the execution of reducers. To use effectively the data locality of blocks in my use case i want to control on which node a reducer will be executed. In my scenario i ...
    Eduard SkaleyEduard Skaley
    Aug 27, 2012 at 5:12 pm
    Aug 30, 2012 at 5:27 pm
  • Hi I am trying to use the Hadoop filesystem abstraction with S3 but in my tinkering I am not having a great deal of success. I am particularly interested in the ability to mimic a directory structure ...
    Chris CollinsChris Collins
    Aug 28, 2012 at 5:06 pm
    Aug 30, 2012 at 11:49 am
  • hi, I have doc files in msword doc and docx format. These have entries which are seperated by an empty line. Is it possible for me to read these lines separated from empty lines at a time. Also which ...
    Siddharth TiwariSiddharth Tiwari
    Aug 24, 2012 at 5:52 am
    Aug 25, 2012 at 6:18 pm
  • Hello list, I have a flat file in which data is stored as lines of 107 bytes each. I need to skip the first 8 lines(as they don't contain any valuable info). Thereafter, I have to read each line and ...
    Mohammad TariqMohammad Tariq
    Aug 1, 2012 at 8:25 pm
    Aug 3, 2012 at 3:24 pm
  • Hello, I had a running hadoop cluster. I restarted it and after that namenode is unable to start. I am getting error saying that it's not formatted. :( Is it possible to recover the data on HDFS? ...
    Abhay RatnaparkhiAbhay Ratnaparkhi
    Aug 24, 2012 at 7:29 am
    Aug 27, 2012 at 5:35 pm
  • Hello, I'm thinking about building a hadoop cluster to analyze all the unsubscribe mails that people mistakenly send to this address. How many PB of storage will I need? - Ryan
    Hennig, RyanHennig, Ryan
    Aug 8, 2012 at 11:12 pm
    Aug 9, 2012 at 3:25 am
  • Hi, I'm doing some research that involves pulling data stored in a mysql cluster directly for a map reduce job, without storing the data in HDFS. I'd like to run hadoop task tracker nodes directly on ...
    Tharindu MathewTharindu Mathew
    Aug 21, 2012 at 9:07 am
    Aug 22, 2012 at 6:31 am
  • I am trying to setup a small cluster using hadoop 2.0.0 and using PI example to validate the setup. When I have 1 master and 1 slave the example works fine. I am getting exceptions with the PI ...
    Arjun ReddyArjun Reddy
    Aug 8, 2012 at 9:40 pm
    Aug 11, 2012 at 4:51 am
  • A AshwinA Ashwin
    Aug 8, 2012 at 3:10 pm
    Aug 11, 2012 at 4:12 am
  • Hi, I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free Edition). In CM, there are a number of options for ...
    Aug 16, 2012 at 4:29 pm
    Aug 19, 2012 at 12:34 pm
  • Hi Users, How to open HDFS zip file(.gz) file in hadoop.? example: bin/hadoop fs -ls /user/hive/warehouse/sample -rw-r--r-- 4 root supergroup 465141227 2012-08-14 17:02 ...
    prabhu Kprabhu K
    Aug 17, 2012 at 6:48 am
    Aug 18, 2012 at 3:18 pm
  • Hi Users, Is it possible from HDFS file to local unix system, is there any command? as anyone knows please reply. Thanks, Prabhu.
    prabhu Kprabhu K
    Aug 10, 2012 at 2:21 pm
    Aug 11, 2012 at 4:09 am
  • Dear list, Lets say i have a file, like this: id \t at,tlng <-- structure 1\t40.123,-50.432 2\t41.431,-43.32 ... ... lets call it: 'points.txt' I'm trying to build a map-reduce job that runs over ...
    Dexter morganDexter morgan
    Aug 27, 2012 at 8:46 pm
    Aug 30, 2012 at 8:06 pm
  • Hi, I have a WAR which is deployed on tomcat server the WAR contains some java classes which uploads files, will i be able to upload directly in to hadoop iam using the below code in one of my java ...
    Visioner SadakVisioner Sadak
    Aug 30, 2012 at 8:32 am
    Aug 30, 2012 at 7:40 pm
  • Hello folks, I am new to hadoop, I just want to get information that how hadoop framework is usefull for real time service.?can any one explain me..? Thanks.
    Mahout userMahout user
    Aug 19, 2012 at 3:44 pm
    Aug 22, 2012 at 6:22 pm
  • All We are getting the following show in when we talk to hadoop 1.0.3 Seems it relates to these lines in Configuration.java public Configuration(boolean loadDefaults) { 225 this.loadDefaults = ...
    Ben CuthbertBen Cuthbert
    Aug 17, 2012 at 3:59 pm
    Aug 22, 2012 at 3:05 pm
  • Hi, I was going through the Apache Hadoop's distribution dependencies (jars in lib folder) and I could not find avro-1.x.x.jar. I though hadoop internally uses avro as its serialization mechanism for ...
    Rahul BhattacharjeeRahul Bhattacharjee
    Aug 22, 2012 at 5:40 am
    Aug 22, 2012 at 7:47 am
  • I am a bit confused about the different options for namenode high availability (or something along those lines) in CDH4 (hadoop-2.0.0). I understand that the secondary namenode is deprecated, and ...
    Jan Van BesienJan Van Besien
    Aug 16, 2012 at 8:12 am
    Aug 16, 2012 at 9:08 pm
  • Hi folks, Replying to this thread is not going to get you unsubscribed and will just annoy everyone else who's subscribed. To unsubscribe please send an email to <span class="m_body_email_addr" ...
    Andy IsaacsonAndy Isaacson
    Aug 29, 2012 at 9:55 pm
    Aug 30, 2012 at 5:35 am
  • Epic Ryan!!! Sent from my Windows Phone ------------------------------ Da: Hennig, Ryan Inviato: 28/08/2012 21:14 A: <span class="m_body_email_addr" title="858a0c8e479a78c1038b7355244ec07c" ...
    Fabio PitzoluFabio Pitzolu
    Aug 28, 2012 at 9:50 pm
    Aug 29, 2012 at 6:00 pm
  • Hi Users. We have flat files on mainframes with around a billion records. We need to sort them and then use them with different jobs on mainframe for report generation. I was wondering was there any ...
    Siddharth TiwariSiddharth Tiwari
    Aug 28, 2012 at 4:24 pm
    Aug 29, 2012 at 10:38 am
  • Hi, I am trying to install sqoop. i am different commands to install in my Ubuntu but nothing is working. Can someone help me on the same these the commands i have tried sudo yum -y install ...
    Rahul pRahul p
    Aug 21, 2012 at 10:44 am
    Aug 28, 2012 at 7:15 pm
  • We have smaller nodes (4 to 6 disks), and we used to write logs to the same disk as where the OS is. So if that disks goes then i don't really care about tasktrackers failing. Also, the fact that ...
    Koert KuipersKoert Kuipers
    Aug 26, 2012 at 5:32 pm
    Aug 26, 2012 at 7:04 pm
  • Hi, I want to broadcast some data to all nodes under Hadoop 0.20.2. I tested DistributedCache module. Unfortunately, it was time-consuming and runtime is important for my work. I want to write a MR ...
    Hamid OliaeiHamid Oliaei
    Aug 23, 2012 at 8:42 am
    Aug 23, 2012 at 1:37 pm
  • I configure a job in hadoop ,set the number of map tasks in the code to 8. Then I run the job and it gets 152 map tasks. Can't get why its being overriden and whhere it get 152 from. The ...
    Nutch buddyNutch buddy
    Aug 21, 2012 at 12:20 pm
    Aug 23, 2012 at 11:31 am
  • Hi I have two mappers MAP1 and MAP2, which collect data from two different files, In reducer I want to traverse all keys and values of MAP2 for each key and value of MAP1. How can I achieve it in one ...
    Siddharth TiwariSiddharth Tiwari
    Aug 20, 2012 at 7:54 pm
    Aug 21, 2012 at 8:03 am
  • We have an application or a series of applications that listen to incoming feeds they then distribute this data in XML form to a number of queues. Another set of processes listen to these queues and ...
    Robert NicholsonRobert Nicholson
    Aug 19, 2012 at 4:47 pm
    Aug 20, 2012 at 2:14 am
  • Are there any utilities available to help parse jobtracker log files? Hank Cohen <span class="m_body_email_addr" title="a2c96842b14ad23483355b001088ac32" hank.cohen@altior.com</span ...
    Hank CohenHank Cohen
    Aug 17, 2012 at 4:30 am
    Aug 18, 2012 at 12:28 am
  • Hi users, I am working on a CDH3 cluster of 12 nodes (Task Trackers running on all the 12 nodes and 1 node running the Job Tracker). In order to perform a WordCount benchmark test, I did the ...
    Gaurav DasguptaGaurav Dasgupta
    Aug 16, 2012 at 2:13 pm
    Aug 17, 2012 at 12:53 pm
  • Hello all, I'm using CDH3u3. If I want to process one File, set to non splitable hadoop starts one Mapper and no Reducer (thats ok for this test scenario). The Mapper goes through a configuration ...
    Matthias KrickeMatthias Kricke
    Aug 13, 2012 at 1:17 pm
    Aug 13, 2012 at 4:08 pm
  • Hello I use "Hadoop Crypto Compressor" from this site" https://github.com/geisbruch/HadoopCryptoCompressor" for encryption hdfs files. I've downloaded the complete code & create the jar file,Change ...
    Farrokh ShahriariFarrokh Shahriari
    Aug 7, 2012 at 7:41 am
    Aug 10, 2012 at 10:11 am
  • I'm having some trouble with permissions on HDFS. I'm trying to create a file in a directory where the user belongs to a group that has write permissions, but it doesn't seem to be working. First, ...
    John ArmstrongJohn Armstrong
    Aug 9, 2012 at 11:56 am
    Aug 9, 2012 at 6:08 pm
  • Hi, I am trying to add a file to HDFS programmatically. In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration object as follows Configuration configuration = null ...
    Chandra Mohan, Ananda Vel MuruganChandra Mohan, Ananda Vel Murugan
    Aug 9, 2012 at 7:08 am
    Aug 9, 2012 at 8:43 am
  • nothing has confused me as much in hadoop as FileSystem.close(). any decent java programmer that sees that an object implements Closable writes code like this: Final FileSystem fs = ...
    Koert KuipersKoert Kuipers
    Aug 6, 2012 at 5:33 pm
    Aug 7, 2012 at 3:54 pm
  • Hi All, I executed the "MRBench" program from "hadoop-test.jar" in my 12 node CDH3 cluster. After executing, I had some strange observations regarding the number of Maps it ran. First I ran the ...
    Gaurav DasguptaGaurav Dasgupta
    Aug 28, 2012 at 10:32 am
    Aug 29, 2012 at 5:12 pm
  • Hello, I am getting the following error when trying to execute a hadoop job on a 5-node cluster: Caused by: java.io.IOException: Call to *** failed on local exception: java.io.EOFException at ...
    Caetano SauerCaetano Sauer
    Aug 28, 2012 at 1:46 pm
    Aug 29, 2012 at 9:24 am
  • Hi All, I was running a cluster of one master and 4 slaves. I copied the hadoop_install folder from the master to all 4 slaves, and configured them well. How ever when i sh start-all.sh from the ...
    Charles AICharles AI
    Aug 27, 2012 at 7:04 am
    Aug 28, 2012 at 9:14 am
  • When implementing Nutch 1.0 awhile back, we had to point our scripts to /opt/freeware/bin to allow Nutch's Hadoop code to utilize the more Linux-like versions of various system commands (like df) in ...
    James F WaltonJames F Walton
    Aug 17, 2012 at 6:46 pm
    Aug 27, 2012 at 1:50 am
  • Hi I had a 2 node cluster earlier.I added 2 nodes on the fly in hadoop cluster. Ran these commands hadoop dfsadmin -refreshNodes hadoop mradmin -refreshNodes But still when i check through fsck or ...
    Iwannaplay gamesIwannaplay games
    Aug 23, 2012 at 1:48 pm
    Aug 24, 2012 at 11:50 am
  • Hi, I am new to Hadoop. What would be the best way to learn hadoop and eco system around it? Thanks, Pravin
    Pravin SinhaPravin Sinha
    Aug 23, 2012 at 4:20 pm
    Aug 23, 2012 at 6:26 pm
  • Hi, Can I change the default delimiter of tab in the output of a job to something else ? How to achieve it ?
    Siddharth TiwariSiddharth Tiwari
    Aug 21, 2012 at 7:42 am
    Aug 21, 2012 at 5:27 pm
Group Navigation
period‹ prev | Aug 2012 | next ›
Group Overview
groupmapreduce-user @

353 users for August 2012

Harsh J: 99 posts Mohammad Tariq: 46 posts Bejoy KS: 32 posts Rahul p: 26 posts Bertrand Dechoux: 25 posts Siddharth Tiwari: 21 posts Michel Segel: 20 posts Sathyavageeswaran: 20 posts Steve Loughran: 17 posts Gaurav Dasgupta: 15 posts Anil Gupta: 13 posts Håvard Wahl Kongsgård: 12 posts Björn-Elmar Macek: 11 posts Chandra Mohan, Ananda Vel Murugan: 11 posts Koert Kuipers: 11 posts Anand sharma: 10 posts Arun C Murthy: 10 posts Ted Dunning: 10 posts Eduard Skaley: 9 posts Mohit Anchlia: 9 posts
show more