FAQ
Hello,

I'm running nutch on a hadoop cluster but unfortunately I don't find under
hadoop_home/logs datanote logs but only a jobtracker log. I've not modified
nutch log4j.properties nor hadoops.
To the console I get printed mapred.JobClient stuff and also nutch stuff the
nutch class logs directly before running as a job.


--
Regards,
K. Gabriele

--- unchanged since 20/9/10 ---
P.S. If the subject contains "[LON]" or the addressee acknowledges the
receipt within 48 hours then I don't resend the email.
subject(this) ∈ L(LON*) ∨ ∃x. (x ∈ MyInbox ∧ Acknowledges(x, this) ∧ time(x)
< Now + 48h) ⇒ ¬resend(I, this).

If an email is sent by a sender that is not a trusted contact or the email
does not contain a valid code then the email is not received. A valid code
starts with a hyphen and ends with "X".
∀x. x ∈ MyInbox ⇒ from(x) ∈ MySafeSenderList ∨ (∃y. y ∈ subject(x) ∧ y ∈
L(-[a-z]+[0-9]X)).

Search Discussions

  • Gabriele Kahlout at Jun 1, 2011 at 12:58 pm
    Apparently the issue is more complicated that I first thought, and it is not
    a Nutch issue.

    Submitting a MapReduce Job to the JobTracker (through JobClient interface)
    the task is executed on another node, with different stdout and stderr from
    the ones the job is submitted. Hence, no matter what log settings in the
    job, those won't surface to my console.
    "The standard output (stdout) and error (stderr) streams of the task are
    read by the TaskTracker and logged to ${HADOOP_LOG_DIR}/userlogs" instead,
    possibly through:
    org.apache.hadoop.mapred.TaskLog.captureOutAndError(..).


    Postmortem one could use debugging scripts to read those
    logs.<http://hadoop.apache.org/common/docs/current/mapred_tutorial.html#Debugging>

    In my imagination it would be possible to aggregate the stdout outputs of
    each task (as they are logged to ${HADOOP_LOG_DIR}/userlogs) and output them
    to the console of the JobTracker, but it seemly horribly complicated.

    Perhaps the easiest thing would be for me to have a script that watches
    every addittion to ${HADOOP_LOG_DIR}/userlogs and outputs the content of
    each new syslog.

    The reason I'm posting is to share my findings, and possibly receive expert
    feedback /validate my thesis and workaround proposal.
    On Thu, May 26, 2011 at 11:58 PM, Gabriele Kahlout wrote:

    Hello,

    I'm running nutch on a hadoop cluster but unfortunately I don't find under
    hadoop_home/logs datanote logs but only a jobtracker log. I've not modified
    nutch log4j.properties nor hadoops.
    To the console I get printed mapred.JobClient stuff and also nutch stuff
    the nutch class logs directly before running as a job.


    --
    Regards,
    K. Gabriele

    --- unchanged since 20/9/10 ---
    P.S. If the subject contains "[LON]" or the addressee acknowledges the
    receipt within 48 hours then I don't resend the email.
    subject(this) ∈ L(LON*) ∨ ∃x. (x ∈ MyInbox ∧ Acknowledges(x, this) ∧
    time(x) < Now + 48h) ⇒ ¬resend(I, this).

    If an email is sent by a sender that is not a trusted contact or the email
    does not contain a valid code then the email is not received. A valid code
    starts with a hyphen and ends with "X".
    ∀x. x ∈ MyInbox ⇒ from(x) ∈ MySafeSenderList ∨ (∃y. y ∈ subject(x) ∧ y ∈
    L(-[a-z]+[0-9]X)).

    --
    Regards,
    K. Gabriele

    --- unchanged since 20/9/10 ---
    P.S. If the subject contains "[LON]" or the addressee acknowledges the
    receipt within 48 hours then I don't resend the email.
    subject(this) ∈ L(LON*) ∨ ∃x. (x ∈ MyInbox ∧ Acknowledges(x, this) ∧ time(x)
    < Now + 48h) ⇒ ¬resend(I, this).

    If an email is sent by a sender that is not a trusted contact or the email
    does not contain a valid code then the email is not received. A valid code
    starts with a hyphen and ends with "X".
    ∀x. x ∈ MyInbox ⇒ from(x) ∈ MySafeSenderList ∨ (∃y. y ∈ subject(x) ∧ y ∈
    L(-[a-z]+[0-9]X)).

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedMay 26, '11 at 9:59p
activeJun 1, '11 at 12:58p
posts2
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Gabriele Kahlout: 2 posts

People

Translate

site design / logo © 2022 Grokbase