FAQ
JobTracker TaskInitialization failure on cygwin
-----------------------------------------------

Key: HADOOP-4743
URL: https://issues.apache.org/jira/browse/HADOOP-4743
Project: Hadoop Core
Issue Type: Bug
Environment: Hadoop 0.19.0 on Cygwin (Windows Vista Ultimate) and Ubuntu Linux 8.10 cluster
Reporter: Masahiko Higashiyama


I wanted to test new release and have been trying to queue new job.
But it didn't work.

This error is found in JobTracker's log.

========================================
2008-11-29 21:39:00,276 ERROR org.apache.hadoop.mapred.EagerTaskInitializationListener: Job initialization failed:
java.util.regex.PatternSyntaxException: Illegal/unsupported escape sequence near index 47
Leibnitz_[0-9]+_job_200811292137_0001_leibnitz\hadoop_\Qwordcount\E+
^
at java.util.regex.Pattern.error(Unknown Source)
at java.util.regex.Pattern.escape(Unknown Source)
at java.util.regex.Pattern.atom(Unknown Source)
at java.util.regex.Pattern.sequence(Unknown Source)
at java.util.regex.Pattern.expr(Unknown Source)
at java.util.regex.Pattern.compile(Unknown Source)
at java.util.regex.Pattern.<init>(Unknown Source)
at java.util.regex.Pattern.compile(Unknown Source)
at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:638)
at org.apache.hadoop.mapred.JobHistory$JobInfo.logSubmitted(JobHistory.java:803)
at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:353)
at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:55)
at java.lang.Thread.run(Unknown Source)
========================================

leibnitz is Jobtracker's hostname and hadoop is username.

Then I wanted to fix it and created this patch.
========================================
--- src/mapred/org/apache/hadoop/mapred/JobHistory.java (revision 721683)
+++ src/mapred/org/apache/hadoop/mapred/JobHistory.java (working copy)
@@ -95,6 +95,10 @@
private static final String SECONDARY_FILE_SUFFIX = ".recover";
private static long jobHistoryBlockSize = 0;
private static String jobtrackerHostname;
+ /** Set to true on Windows platforms */
+ public static final boolean WINDOWS /* borrowed from Path.WINDOWS */
+ = System.getProperty("os.name").startsWith("Windows");
+
/**
* Record types are identifiers for each line of log in history files.
* A record type appears as the first token in a single line of log.
@@ -634,6 +638,9 @@

jobName = escapeRegexChars( jobName );

+ if (WINDOWS) {
+ user = escapeRegexChars( user );
+ }
// Make the pattern matching the job's history file
final Pattern historyFilePattern =
Pattern.compile(jobtrackerHostname + "_" + "[0-9]+" + "_"
========================================

Hadoop Core patched this seems to work well.

Maybe my environment is wrong. But if this is bug, please fix.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Masahiko Higashiyama (JIRA) at Nov 29, 2008 at 6:33 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Masahiko Higashiyama updated HADOOP-4743:
    -----------------------------------------

    Component/s: mapred
    Priority: Trivial (was: Major)
    Affects Version/s: 0.19.0
    JobTracker TaskInitialization failure on cygwin
    -----------------------------------------------

    Key: HADOOP-4743
    URL: https://issues.apache.org/jira/browse/HADOOP-4743
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Affects Versions: 0.19.0
    Environment: Hadoop 0.19.0 on Cygwin (Windows Vista Ultimate) and Ubuntu Linux 8.10 cluster
    Reporter: Masahiko Higashiyama
    Priority: Trivial

    I wanted to test new release and have been trying to queue new job.
    But it didn't work.
    This error is found in JobTracker's log.
    ========================================
    2008-11-29 21:39:00,276 ERROR org.apache.hadoop.mapred.EagerTaskInitializationListener: Job initialization failed:
    java.util.regex.PatternSyntaxException: Illegal/unsupported escape sequence near index 47
    Leibnitz_[0-9]+_job_200811292137_0001_leibnitz\hadoop_\Qwordcount\E+
    ^
    at java.util.regex.Pattern.error(Unknown Source)
    at java.util.regex.Pattern.escape(Unknown Source)
    at java.util.regex.Pattern.atom(Unknown Source)
    at java.util.regex.Pattern.sequence(Unknown Source)
    at java.util.regex.Pattern.expr(Unknown Source)
    at java.util.regex.Pattern.compile(Unknown Source)
    at java.util.regex.Pattern.<init>(Unknown Source)
    at java.util.regex.Pattern.compile(Unknown Source)
    at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:638)
    at org.apache.hadoop.mapred.JobHistory$JobInfo.logSubmitted(JobHistory.java:803)
    at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:353)
    at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:55)
    at java.lang.Thread.run(Unknown Source)
    ========================================
    leibnitz is Jobtracker's hostname and hadoop is username.
    Then I wanted to fix it and created this patch.
    ========================================
    --- src/mapred/org/apache/hadoop/mapred/JobHistory.java (revision 721683)
    +++ src/mapred/org/apache/hadoop/mapred/JobHistory.java (working copy)
    @@ -95,6 +95,10 @@
    private static final String SECONDARY_FILE_SUFFIX = ".recover";
    private static long jobHistoryBlockSize = 0;
    private static String jobtrackerHostname;
    + /** Set to true on Windows platforms */
    + public static final boolean WINDOWS /* borrowed from Path.WINDOWS */
    + = System.getProperty("os.name").startsWith("Windows");
    +
    /**
    * Record types are identifiers for each line of log in history files.
    * A record type appears as the first token in a single line of log.
    @@ -634,6 +638,9 @@

    jobName = escapeRegexChars( jobName );

    + if (WINDOWS) {
    + user = escapeRegexChars( user );
    + }
    // Make the pattern matching the job's history file
    final Pattern historyFilePattern =
    Pattern.compile(jobtrackerHostname + "_" + "[0-9]+" + "_"
    ========================================
    Hadoop Core patched this seems to work well.
    Maybe my environment is wrong. But if this is bug, please fix.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedNov 29, '08 at 6:31p
activeNov 29, '08 at 6:33p
posts2
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Masahiko Higashiyama (JIRA): 2 posts

People

Translate

site design / logo © 2022 Grokbase