FAQ
Hi, I am stuck in a basic problem but can't figure out. My previous
verbose logging problem is the same as the one mentioned in the old post.

http://mail-archives.apache.org/mod_mbox/nutch-user/200901.mbox/%3C0ADBD67BD6811A4BB2144D805124714D03F754A447@KAEX1.Dom.Rastatt.de%3E

First quesiton, if I see a lot of logs on the screen like (as mentioned in Tom White's ``Hadoop: The definitive Guide'' book, page 23):


09/04/07 12:34:35 INFO mapred.MapTask: numReduceTasks: 1
09/04/07 12:34:35 INFO mapred.MapTask: io.sort.mb = 100
09/04/07 12:34:35 INFO mapred.MapTask: data buffer = 79691776/99614720
09/04/07 12:34:35 INFO mapred.MapTask: record buffer = 262144/327680
09/04/07 12:34:35 INFO mapred.MapTask: Starting flush of map output
09/04/07 12:34:36 INFO mapred.MapTask: Finished spill 0

does it mean I am running in the standalone mode? I think in a real cluster mode I should not see these. When I was running my code in real cluster model, I only see output like

Map 10% Reduce 0%

and all the logs are written to logs/userlogs folder.


So I guess I entered a LocalJob mode (standalone) mistakenly, but not in the real cluster mode. However, I did setup the three xml files correctly I think, and I started up the MapReduce daemons (start-dfs.sh, start-mapred.sh). So why the code is still running in standalone mode? Anything else I should pay attention to? Thanks!


Shi

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedJun 6, '11 at 5:21p
activeJun 6, '11 at 5:21p
posts1
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Shi Yu: 1 post

People

Translate

site design / logo © 2022 Grokbase