FAQ
What's the benefits from such design compared to multi-thread?

--

朱盛凯

Jash Zhu

复旦大学软件学院

Software School, Fudan University

Search Discussions

  • Jason Venner at Jul 14, 2008 at 1:38 pm
    One benefit is that if your map or reduce behaves badly it can't take
    down the task tracker.

    In our case we have some poorly behaved external native libraries we
    use, and we have to forcibly ensure that the child vms are killed when
    the child main finishes (often by kill -9), so the fact the child (task)
    is a separate jvm process is very helpful.

    The downside is the jvm start time. Has anyone experimented with the jar
    freezing for more than the standard boot class path jars to speed up
    startup?


    Shengkai Zhu wrote:
    What's the benefits from such design compared to multi-thread?
    --
    Jason Venner
    Attributor - Program the Web <http://www.attributor.com/>
    Attributor is hiring Hadoop Wranglers and coding wizards, contact if
    interested
  • Shengkai Zhu at Jul 14, 2008 at 2:14 pm
    Well, I got it.

    On 7/14/08, Jason Venner wrote:

    One benefit is that if your map or reduce behaves badly it can't take down
    the task tracker.

    In our case we have some poorly behaved external native libraries we use,
    and we have to forcibly ensure that the child vms are killed when the child
    main finishes (often by kill -9), so the fact the child (task) is a separate
    jvm process is very helpful.

    The downside is the jvm start time. Has anyone experimented with the jar
    freezing for more than the standard boot class path jars to speed up
    startup?


    Shengkai Zhu wrote:
    What's the benefits from such design compared to multi-thread?

    --
    Jason Venner
    Attributor - Program the Web <http://www.attributor.com/>
    Attributor is hiring Hadoop Wranglers and coding wizards, contact if
    interested


    --

    朱盛凯

    Jash Zhu

    复旦大学软件学院

    Software School, Fudan University
  • Torsten Curdt at Jul 14, 2008 at 2:25 pm

    On 7/14/08, Jason Venner wrote:
    One benefit is that if your map or reduce behaves badly it can't
    take down
    the task tracker.

    As the tracker jvm could also be monitored (and restarted) from
    outside, the internal execution might still be worth looking into. At
    least to have the option. We had a patch ...but it's terrible out of
    date.

    On the other hand starting up the second jvm is supposed to be much
    faster on 1.6 ....no idea if that's true though.

    cheers
    --
    Torsten

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedJul 14, '08 at 1:19p
activeJul 14, '08 at 2:25p
posts4
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase