Search Discussions
-
Hi, I am migrating from Apache hadoop 0.20.205 to CDH3u3. I don't want to lose the data that is in the HDFS of Apache hadoop 0.20.205. How do I migrate to CDH3u3 but keep the data that I have on ...
Austin Chungath
May 3, 2012 at 6:11 am
May 9, 2012 at 11:26 am -
Hi, I wonder if someone could give some pointers with a problem I'm having? I have a 7 machine cluster setup for testing and we have been pouring data into it for a week without issue, have learnt ...
Darrell Taylor
May 9, 2012 at 4:53 pm
May 11, 2012 at 10:37 am -
Hi, I wonder that if Hadoop can solve effectively the question as following: ========================================== input file: a.txt, b.txt result: c.txt a.txt: id1,name1,age1,.. ...
Liuzhg
May 29, 2012 at 10:13 am
May 30, 2012 at 1:56 pm -
Hi, We have a 5node cdh3u4 cluster running. When i try to do teragen/terasort some of the map tasks are Failed/Killed and the logs show similar error on all machines. 2012-05-22 09:43:50,831 INFO ...
Sandeep Reddy P
May 22, 2012 at 2:02 pm
May 22, 2012 at 6:14 pm -
Hello, We have about 50 VMs and we want to distribute processing across them. However these VMs share a huge data storage system and thus their "virtual" HDD are all located in the same computer ...
Pierre Antoine Du Bois De Naurois
May 17, 2012 at 8:39 pm
May 18, 2012 at 10:28 am -
Hi I am trying to configure Hadoop 1.0. in pseudodistributed mode. But when I run the pi example given in the hadoop distribution, I get the error mentioned in title. Can someone please help me and ...
Waqas latif
May 16, 2012 at 1:21 pm
May 16, 2012 at 6:02 pm -
Hi, i recently downloaded and successfully installed hadoop-1.0.1 in my ubuntu 10.04 LTS. I have hadoop-1.0.1.tar.gz downloaded and now i want to design map-reduce application. As suggested by some ...
Ravi Joshi
May 17, 2012 at 1:18 pm
Jul 1, 2012 at 4:42 pm -
Hi Experts, I am fairly new to hadoop MapR and I was trying to run a matrix multiplication example presented by Mr. Norstadt under following link http://www.norstad.org/matrix-multiply/index.html. I ...
Waqas latif
May 25, 2012 at 9:43 am
May 30, 2012 at 5:31 pm -
Hi, all, I got the exception below in the mapper. I already have my global Hadoop heap at 5 GB, but is there a specific other setting? Or maybe I should troubleshoot for memory? But the same ...
Mark Kerzner
May 23, 2012 at 7:17 pm
May 24, 2012 at 3:45 am -
If I understand it right HOD is mentioned mainly for merging existing HPC clusters with hadoop and for testing purposes.. I cannot find what is the role of Torque here (just initial nodes ...
Merto Mertek
May 17, 2012 at 7:12 pm
May 21, 2012 at 3:00 pm -
Hi, I am novice in Hadoop. Kindly suggest how do we load log files into hdfs. Please suggest the command and steps. Thanks in advance!! -- View this message in context ...
AnExplorer
May 13, 2012 at 4:24 am
May 15, 2012 at 2:04 am -
I'm new to Hadoop and am trying to get it setup in Eclipse. I'm following the "Working with Hadoop under Eclipse" wiki to do this. First let me make sure this will do what I am hoping it will do. I'm ...
Wilson Wayne - wwilso
May 7, 2012 at 10:10 pm
May 8, 2012 at 6:14 pm -
Hi , Has anyone used Hadoop and splunk, or any other real-time processing tool over Hadoop? Regards, Shreya This e-mail and any files transmitted with it are for the sole use of the intended ...
Shreya Pal
May 18, 2012 at 7:11 pm
May 28, 2012 at 1:15 pm -
I have a rather large map reduce job which takes few days. I was wondering if its possible for me to freeze the job or make the job less intensive. Is it possible to reduce the number of slots per ...
Rita
May 11, 2012 at 10:44 am
May 11, 2012 at 3:59 pm -
Hi, We are getting 100TB of data with replication factor of 3 this goes to 300TB of data. We are planning to use hadoop with 65nodes. We want to know which option will be better in terms of hardware ...
Sandeep Reddy P
May 31, 2012 at 6:42 pm
Jun 1, 2012 at 8:23 am -
Hi, We are about to build a 10 machine cluster with 40Tb of storage, obviously as this gets full actually trying to create an offsite backup becomes a problem unless we build another 10 machine ...
Darrell Taylor
May 29, 2012 at 4:20 pm
May 31, 2012 at 7:51 pm -
Hi guys, this is a very simple program, trying to use TextInputFormat and SequenceFileoutputFormat. Should be easy but I get the same error. Here is my configurations ...
Mark question
May 29, 2012 at 7:57 pm
May 30, 2012 at 6:32 pm -
it seems that if I put too many records into the same mapper output key, all these records are grouped into one key one one reducer, then the reducer became out of memory. but the reducer interface ...
Yang
May 9, 2012 at 6:51 pm
May 11, 2012 at 5:58 am -
Hi folks, (Resending to this group, sent to common-dev before, pretty sure that's for Hadoop internal development - sorry for that..) I'm pretty stuck here. I've been researching for hours and I ...
Todd McFarland
May 19, 2012 at 3:30 pm
Sep 13, 2012 at 6:45 pm -
All, We are trying to implement sqoop in our environment which has 30 mysql sharded databases and all the databases have around 30 databases with 150 tables in each of the database which are all ...
Srinivas Surasani
May 31, 2012 at 10:03 pm
Jun 2, 2012 at 12:09 am -
We get click data through API calls. I now need to send this data to our hadoop environment. I am wondering if I could open one sequence file and write to it until it's of certain size. Once it's ...
Mohit Anchlia
May 25, 2012 at 2:55 pm
May 31, 2012 at 2:38 am -
Hello Hadoop community, I have been trying to set up a double node Hadoop cluster (following the instructions in - ...
Rohit Pandey
May 29, 2012 at 5:27 pm
May 30, 2012 at 2:17 pm -
Hi, I am trying to run few benchmarks on a small hadoop-cluster of 4 VMs (2 on 2 phyiscal hosts, each VM having 1 cpu core, 2GB ram, individual disk and Gbps bridged connectivity). I am using ...
Akshay Singh
May 23, 2012 at 8:38 pm
May 29, 2012 at 3:15 pm -
I am using Cloudera distribution cdh3u1. When trying to check native codecs for better decompression performance such as Snappy or LZO, I ran into issues with random access using ...
Jason B
May 21, 2012 at 10:57 pm
May 22, 2012 at 4:54 pm -
Hi, Is FileSystem.append supported on hadoop 1.0.x? (1.0.3 in particular). Reading this list I thought it was back in for 1.0, but it's disabled by default so I'm not 100% sure. It would be great to ...
Rodney O'Donnell
May 18, 2012 at 6:43 am
May 21, 2012 at 1:24 pm -
Dear all, I have one single input file, which contains, on every line, some hydrological calibration models (data). Each line of the file should be processed and then the output from every line ...
Biro lehel
May 20, 2012 at 9:19 am
May 20, 2012 at 11:14 am -
Dear experts, Today is my tenth day working with Hadoop on installing on my windows machine. I am trying again and again because , some where someone has written that it works on Windows with ...
Ravishankar Nair
May 18, 2012 at 1:50 am
May 18, 2012 at 8:58 pm -
Hi, I have a question to the hadoop experts: I have two HDFS, in different subnet. HDFS1 : 192.168.*.* HDFS2: 10.10.*.* the namenode of HDFS2 has two NIC. One connected to 192.168.*.* and another to ...
Arindam Choudhury
May 11, 2012 at 1:45 pm
May 11, 2012 at 2:38 pm -
Hi All, Which is the best monitoring tool for Hadoop cluster monitoring? Ganglia or Nagios? Thanks, Manu S
Manu S
May 11, 2012 at 5:32 am
May 11, 2012 at 9:20 am -
We are looking at doing some initial analysis on SQL text info within the query runs to come up with some kind of path output to depict how various tables are linked to each other. For example. A ...
Karanveer Singh
May 10, 2012 at 7:47 am
May 10, 2012 at 3:13 pm -
So, I installed Hadoop on my imac via port install hadoop and after working through a few configuration issues tried to test the setup with calculation of PI. Unfortunately, I got this answer ...
Alex Paransky
May 9, 2012 at 12:36 am
May 9, 2012 at 4:53 am -
I am running a task which gets to 66% of the Reduce step and then hangs indefinitely. Here is the log file (I apologize if I am putting too much here but I am not exactly sure what is relevant) ...
Keith Thompson
May 2, 2012 at 9:50 pm
May 4, 2012 at 4:52 pm -
Hi All, Can we find out the complete block names from the fsimage we have? Scenario: Accidentally we had lost the hdfs data. We have the previous fsimage before the data loss. We have restored some ...
Manu S
May 3, 2012 at 8:15 am
May 3, 2012 at 10:02 am -
Hi All, How to compare to input file In M/R Job. let A Log file around 30GB and B Log file size is around 60 GB I wanted to know how i will define <K,V inside the mapper. Thanks samir.
Samir das mohapatra
May 23, 2012 at 7:47 pm
May 24, 2012 at 6:19 pm -
I have always wondered about this and and not sure as to phenomenon. When I fire a map reduce job to copy data over in a distributed fashion I would expect to see mappers executing the copy. What ...
Ranjith
May 22, 2012 at 1:19 am
May 22, 2012 at 7:37 pm -
I have large volume of stream log data. Each data record contains a time stamp, which is very important to the analysis. For example, I have data format like this: (1) 20:30:21 01/April/2012 ...
Zhiwei Lin
May 21, 2012 at 8:02 pm
May 22, 2012 at 1:59 pm -
Hi, I am trying to install pipes on os x following the instructions here I can compile utils, but not pipes itself. System: gcc version 4.2.1, java version "1.6.0_31" , OS X 10.7.3 hadoop: 1.0.3 When ...
Peter Cogan
May 21, 2012 at 11:08 am
May 21, 2012 at 6:00 pm -
Hi, I am a newbie on Hadoop and have a quick question on optimal compute vs. storage resources for MapReduce. If I have a multiprocessor node with 4 processors, will Hadoop schedule higher number of ...
Satheesh Kumar
May 11, 2012 at 4:51 pm
May 16, 2012 at 4:50 pm -
Hi, I am seeing an issue where Namenode does not start due an EOFException. The disk was full and I cleared space up but I am unable to get past this exception. Any ideas on how this can be resolved? ...
Prashant Kommireddi
May 14, 2012 at 5:20 pm
May 15, 2012 at 11:51 pm -
Hi , I am new user to hadoop . I have installed hadoop0.19.1 on single windows machine. Its http://localhost:50030/jobtracker.jsp and http://localhost:50070/dfshealth.jsp pages are working fine but ...
Mohit Kundra
May 11, 2012 at 7:00 am
May 11, 2012 at 12:55 pm -
Hi I am running Hadoop-1.0.1 with Sun jdk1.6.0_23. My system is a head node with 14 compute blades When trying to start hadoop, I get the following message in the logs for each data node: 2012-05-09 ...
Fourie Joubert
May 9, 2012 at 3:15 pm
May 10, 2012 at 9:04 am -
Hello All, As Apache Hadoop community is ready to release the next 2.0 alpha version of Hadoop , i would like to bring attention towards need to make better documentation of the tutorials and ...
Jagat
May 4, 2012 at 7:21 pm
May 9, 2012 at 2:17 pm -
Hi, I try to run a Hadoop reduce-side join, then I get the following: java.lang.NoClassDefFoundError: org/apache/hadoop/contrib/utils/join/DataJoinMapperBase at ...
唐方爽
May 4, 2012 at 7:27 am
May 4, 2012 at 10:57 am -
It sounds like an exciting feature. Does anyone have tried this in practice? How does the hot standby namenode perform and how reliable is the HDFS recovery? Is it now a good chance to migrate to ...
Shi Yu
May 3, 2012 at 2:29 pm
May 3, 2012 at 5:59 pm -
Dear All, A very stupid question here. I installed Hadoop 1.0.1 in Ubuntu by deb provided. but how can I specify the hadoop installation directory in eclipse plugin. Thanks for every answer. robot ...
Zillou
May 2, 2012 at 10:34 am
May 3, 2012 at 2:59 am -
Is there a way to compress map only jobs to compress map output that gets stored on hdfs as part-m-* files? In pig I used : Would these work form plain map reduce jobs as well? set ...
Mohit Anchlia
May 1, 2012 at 12:25 am
May 1, 2012 at 4:08 am -
Hi All, Did any one work on hadoop with LDAP integration. Please help me for same. Thanks samir
Samir das mohapatra
May 26, 2012 at 12:41 pm
May 29, 2012 at 2:14 pm -
Hi All, How to configure the external jar , which is use by application internally. For eample: JDBC ,Hive Driver etc. Note:- I dont have permission to start and stop the hadoop machine. So I need to ...
Samir das mohapatra
May 26, 2012 at 12:49 pm
May 27, 2012 at 4:59 am -
Hi, I encounter a problem about when I install the LZO, after i install it, I found that it can run on Pig scripts and streaming scripts and when I check these jobs though jobtracker , it shows that ...
Yingnan.ma
May 23, 2012 at 9:55 am
May 23, 2012 at 12:10 pm -
Hi, I am using a Hadoop cluster of my own construction on EC2, and I am running out of hard drive space with maps. If I knew which directories are used by Hadoop for map spill, I could use the large ...
Mark Kerzner
May 22, 2012 at 1:29 pm
May 22, 2012 at 2:02 pm
Group Overview
group | common-user |
categories | hadoop |
discussions | 131 |
posts | 560 |
users | 175 |
website | hadoop.apache.org... |
irc | #hadoop |
175 users for May 2012
Archives
- February 2013 (260)
- January 2013 (843)
- December 2012 (621)
- November 2012 (729)
- October 2012 (955)
- September 2012 (794)
- August 2012 (1,201)
- July 2012 (362)
- June 2012 (408)
- May 2012 (560)
- April 2012 (462)
- March 2012 (579)
- February 2012 (634)
- January 2012 (468)
- December 2011 (492)
- November 2011 (406)
- October 2011 (567)
- September 2011 (590)
- August 2011 (474)
- July 2011 (433)
- June 2011 (542)
- May 2011 (595)
- April 2011 (558)
- March 2011 (681)
- February 2011 (577)
- January 2011 (514)
- December 2010 (443)
- November 2010 (389)
- October 2010 (499)
- September 2010 (444)
- August 2010 (495)
- July 2010 (487)
- June 2010 (474)
- May 2010 (420)
- April 2010 (430)
- March 2010 (630)
- February 2010 (492)
- January 2010 (471)
- December 2009 (543)
- November 2009 (691)
- October 2009 (558)
- September 2009 (614)
- August 2009 (677)
- July 2009 (885)
- June 2009 (847)
- May 2009 (812)
- April 2009 (881)
- March 2009 (728)
- February 2009 (726)
- January 2009 (517)
- December 2008 (457)
- November 2008 (566)
- October 2008 (658)
- September 2008 (708)
- August 2008 (539)
- July 2008 (706)
- June 2008 (542)
- May 2008 (560)
- April 2008 (718)
- March 2008 (657)
- February 2008 (611)
- January 2008 (546)
- December 2007 (446)
- November 2007 (445)
- October 2007 (448)
- September 2007 (406)
- August 2007 (306)
- July 2007 (241)
- June 2007 (216)
- May 2007 (150)
- April 2007 (100)
- March 2007 (138)
- February 2007 (70)
- January 2007 (105)
- December 2006 (62)
- November 2006 (115)
- October 2006 (87)
- September 2006 (56)
- August 2006 (73)
- July 2006 (57)
- June 2006 (35)
- May 2006 (42)
- April 2006 (43)
- March 2006 (74)
- February 2006 (17)