Hi, I am stuck in a basic problem but can't figure out. My previous
verbose logging problem is the same as the one mentioned in the old post.


First quesiton, if I see a lot of logs on the screen like (as mentioned in Tom White's ``Hadoop: The definitive Guide'' book, page 23):

09/04/07 12:34:35 INFO mapred.MapTask: numReduceTasks: 1
09/04/07 12:34:35 INFO mapred.MapTask: io.sort.mb = 100
09/04/07 12:34:35 INFO mapred.MapTask: data buffer = 79691776/99614720
09/04/07 12:34:35 INFO mapred.MapTask: record buffer = 262144/327680
09/04/07 12:34:35 INFO mapred.MapTask: Starting flush of map output
09/04/07 12:34:36 INFO mapred.MapTask: Finished spill 0

does it mean I am running in the standalone mode? I think in a real cluster mode I should not see these. When I was running my code in real cluster model, I only see output like

Map 10% Reduce 0%

and all the logs are written to logs/userlogs folder.

So I guess I entered a LocalJob mode (standalone) mistakenly, but not in the real cluster mode. However, I did setup the three xml files correctly I think, and I started up the MapReduce daemons (start-dfs.sh, start-mapred.sh). So why the code is still running in standalone mode? Anything else I should pay attention to? Thanks!


Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
postedJun 6, '11 at 5:21p
activeJun 6, '11 at 5:21p

1 user in discussion

Shi Yu: 1 post



site design / logo © 2022 Grokbase