FAQ
Hi,

I am upgrading to 0.20 from 0.18, and right now the setup() gets called, but
the map() does not.

The log indicates that an input record was found - but it is not processed.

10/09/19 23:56:21 INFO mapred.TaskRunner: Task:attempt_local_0001_r_000000_0
is done. And is in the process of commiting
10/09/19 23:56:21 INFO mapred.LocalJobRunner:
10/09/19 23:56:21 INFO mapred.TaskRunner: Task attempt_local_0001_r_000000_0
is allowed to commit now
10/09/19 23:56:21 INFO mapred.JobClient: map 100% reduce 0%
10/09/19 23:56:21 INFO output.FileOutputCommitter: Saved output of task
'attempt_local_0001_r_000000_0' to hdfs://localhost:9000/scala/p1/output
10/09/19 23:56:21 INFO mapred.LocalJobRunner: reduce > reduce
10/09/19 23:56:21 INFO mapred.TaskRunner: Task
'attempt_local_0001_r_000000_0' done.
10/09/19 23:56:22 INFO mapred.JobClient: map 100% reduce 100%
10/09/19 23:56:22 INFO mapred.JobClient: Job complete: job_local_0001
10/09/19 23:56:22 INFO mapred.JobClient: Counters: 14
10/09/19 23:56:22 INFO mapred.JobClient: FileSystemCounters
10/09/19 23:56:22 INFO mapred.JobClient: FILE_BYTES_READ=35600
10/09/19 23:56:22 INFO mapred.JobClient: HDFS_BYTES_READ=10820
10/09/19 23:56:22 INFO mapred.JobClient: FILE_BYTES_WRITTEN=71798
10/09/19 23:56:22 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=421
10/09/19 23:56:22 INFO mapred.JobClient: Map-Reduce Framework
10/09/19 23:56:22 INFO mapred.JobClient: Reduce input groups=1
10/09/19 23:56:22 INFO mapred.JobClient: Combine output records=0
10/09/19 23:56:22 INFO mapred.JobClient: Map input records=1
10/09/19 23:56:22 INFO mapred.JobClient: Reduce shuffle bytes=0
10/09/19 23:56:22 INFO mapred.JobClient: Reduce output records=1
10/09/19 23:56:22 INFO mapred.JobClient: Spilled Records=2
10/09/19 23:56:22 INFO mapred.JobClient: Map output bytes=14
10/09/19 23:56:22 INFO mapred.JobClient: Combine input records=0
10/09/19 23:56:22 INFO mapred.JobClient: Map output records=1
10/09/19 23:56:22 INFO mapred.JobClient: Reduce input records=1

Thank you for advice.

Sincerely,
Mark

Search Discussions

  • Newpant at Sep 25, 2010 at 7:54 am
    hi, could you post your code

    2010/9/20 Mark Kerzner <markkerzner@gmail.com>
    Hi,

    I am upgrading to 0.20 from 0.18, and right now the setup() gets called,
    but
    the map() does not.

    The log indicates that an input record was found - but it is not processed.

    10/09/19 23:56:21 INFO mapred.TaskRunner:
    Task:attempt_local_0001_r_000000_0
    is done. And is in the process of commiting
    10/09/19 23:56:21 INFO mapred.LocalJobRunner:
    10/09/19 23:56:21 INFO mapred.TaskRunner: Task
    attempt_local_0001_r_000000_0
    is allowed to commit now
    10/09/19 23:56:21 INFO mapred.JobClient: map 100% reduce 0%
    10/09/19 23:56:21 INFO output.FileOutputCommitter: Saved output of task
    'attempt_local_0001_r_000000_0' to hdfs://localhost:9000/scala/p1/output
    10/09/19 23:56:21 INFO mapred.LocalJobRunner: reduce > reduce
    10/09/19 23:56:21 INFO mapred.TaskRunner: Task
    'attempt_local_0001_r_000000_0' done.
    10/09/19 23:56:22 INFO mapred.JobClient: map 100% reduce 100%
    10/09/19 23:56:22 INFO mapred.JobClient: Job complete: job_local_0001
    10/09/19 23:56:22 INFO mapred.JobClient: Counters: 14
    10/09/19 23:56:22 INFO mapred.JobClient: FileSystemCounters
    10/09/19 23:56:22 INFO mapred.JobClient: FILE_BYTES_READ=35600
    10/09/19 23:56:22 INFO mapred.JobClient: HDFS_BYTES_READ=10820
    10/09/19 23:56:22 INFO mapred.JobClient: FILE_BYTES_WRITTEN=71798
    10/09/19 23:56:22 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=421
    10/09/19 23:56:22 INFO mapred.JobClient: Map-Reduce Framework
    10/09/19 23:56:22 INFO mapred.JobClient: Reduce input groups=1
    10/09/19 23:56:22 INFO mapred.JobClient: Combine output records=0
    10/09/19 23:56:22 INFO mapred.JobClient: Map input records=1
    10/09/19 23:56:22 INFO mapred.JobClient: Reduce shuffle bytes=0
    10/09/19 23:56:22 INFO mapred.JobClient: Reduce output records=1
    10/09/19 23:56:22 INFO mapred.JobClient: Spilled Records=2
    10/09/19 23:56:22 INFO mapred.JobClient: Map output bytes=14
    10/09/19 23:56:22 INFO mapred.JobClient: Combine input records=0
    10/09/19 23:56:22 INFO mapred.JobClient: Map output records=1
    10/09/19 23:56:22 INFO mapred.JobClient: Reduce input records=1

    Thank you for advice.

    Sincerely,
    Mark
  • Biju .B at Sep 25, 2010 at 8:24 am
    Hi
    Think this will be solved once you restart the system
    this solved this kind of problem for me..
    On Sat, Sep 25, 2010 at 1:23 PM, newpant wrote:

    hi, could you post your code

    2010/9/20 Mark Kerzner <markkerzner@gmail.com>
    Hi,

    I am upgrading to 0.20 from 0.18, and right now the setup() gets called,
    but
    the map() does not.

    The log indicates that an input record was found - but it is not
    processed.
    10/09/19 23:56:21 INFO mapred.TaskRunner:
    Task:attempt_local_0001_r_000000_0
    is done. And is in the process of commiting
    10/09/19 23:56:21 INFO mapred.LocalJobRunner:
    10/09/19 23:56:21 INFO mapred.TaskRunner: Task
    attempt_local_0001_r_000000_0
    is allowed to commit now
    10/09/19 23:56:21 INFO mapred.JobClient: map 100% reduce 0%
    10/09/19 23:56:21 INFO output.FileOutputCommitter: Saved output of task
    'attempt_local_0001_r_000000_0' to hdfs://localhost:9000/scala/p1/output
    10/09/19 23:56:21 INFO mapred.LocalJobRunner: reduce > reduce
    10/09/19 23:56:21 INFO mapred.TaskRunner: Task
    'attempt_local_0001_r_000000_0' done.
    10/09/19 23:56:22 INFO mapred.JobClient: map 100% reduce 100%
    10/09/19 23:56:22 INFO mapred.JobClient: Job complete: job_local_0001
    10/09/19 23:56:22 INFO mapred.JobClient: Counters: 14
    10/09/19 23:56:22 INFO mapred.JobClient: FileSystemCounters
    10/09/19 23:56:22 INFO mapred.JobClient: FILE_BYTES_READ=35600
    10/09/19 23:56:22 INFO mapred.JobClient: HDFS_BYTES_READ=10820
    10/09/19 23:56:22 INFO mapred.JobClient: FILE_BYTES_WRITTEN=71798
    10/09/19 23:56:22 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=421
    10/09/19 23:56:22 INFO mapred.JobClient: Map-Reduce Framework
    10/09/19 23:56:22 INFO mapred.JobClient: Reduce input groups=1
    10/09/19 23:56:22 INFO mapred.JobClient: Combine output records=0
    10/09/19 23:56:22 INFO mapred.JobClient: Map input records=1
    10/09/19 23:56:22 INFO mapred.JobClient: Reduce shuffle bytes=0
    10/09/19 23:56:22 INFO mapred.JobClient: Reduce output records=1
    10/09/19 23:56:22 INFO mapred.JobClient: Spilled Records=2
    10/09/19 23:56:22 INFO mapred.JobClient: Map output bytes=14
    10/09/19 23:56:22 INFO mapred.JobClient: Combine input records=0
    10/09/19 23:56:22 INFO mapred.JobClient: Map output records=1
    10/09/19 23:56:22 INFO mapred.JobClient: Reduce input records=1

    Thank you for advice.

    Sincerely,
    Mark

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedSep 20, '10 at 4:59a
activeSep 25, '10 at 8:24a
posts3
users3
websitehadoop.apache.org...
irc#hadoop

3 users in discussion

Biju .B: 1 post Newpant: 1 post Mark Kerzner: 1 post

People

Translate

site design / logo © 2022 Grokbase