FAQ
org.apache.hadoop.mapred.join.CompositeInputFormat does not initialize TextInput format files with the configuration resulting in an NullPointerException
----------------------------------------------------------------------------------------------------------------------------------------------------------

Key: HADOOP-3299
URL: https://issues.apache.org/jira/browse/HADOOP-3299
Project: Hadoop Core
Issue Type: Bug
Components: io, mapred
Affects Versions: 0.16.3
Reporter: Jason
Attachments: text.diff

The input formats are not initialized with the Configuration object before the isSplitable method is called.

bin/hadoop jar hadoop-0.16.3-examples.jar join -r 1 -inFormat org.apache.hadoop.mapred.KeyValueTextInputFormat -outFormat org.apache.hadoop.mapred.TextOutputFormat -joinOp outer datajoin/input datajoin/output -outKey org.apache.hadoop.io.Text -outValue org.apache.hadoop.mapred.join.TupleWritable
08/04/22 15:05:33 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Job started: Tue Apr 22 15:05:33 GMT-08:00 2008
08/04/22 15:05:33 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
08/04/22 15:05:33 INFO mapred.FileInputFormat: Total input paths to process : 2
java.lang.NullPointerException
at org.apache.hadoop.mapred.KeyValueTextInputFormat.isSplitable(KeyValueTextInputFormat.java:44)
at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:185)
at org.apache.hadoop.mapred.join.Parser$WNode.getSplits(Parser.java:304)
at org.apache.hadoop.mapred.join.Parser$CNode.getSplits(Parser.java:374)
at org.apache.hadoop.mapred.join.CompositeInputFormat.getSplits(CompositeInputFormat.java:129)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:542)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:803)
at org.apache.hadoop.examples.Join.run(Join.java:149)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.examples.Join.main(Join.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:155)


--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Jason (JIRA) at Apr 22, 2008 at 11:50 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jason updated HADOOP-3299:
    --------------------------

    Attachment: text.diff

    ensure that compressionCodes is initialized in isSplittable before dereferencing it.
    org.apache.hadoop.mapred.join.CompositeInputFormat does not initialize TextInput format files with the configuration resulting in an NullPointerException
    ----------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3299
    URL: https://issues.apache.org/jira/browse/HADOOP-3299
    Project: Hadoop Core
    Issue Type: Bug
    Components: io, mapred
    Affects Versions: 0.16.3
    Reporter: Jason
    Attachments: text.diff


    The input formats are not initialized with the Configuration object before the isSplitable method is called.
    bin/hadoop jar hadoop-0.16.3-examples.jar join -r 1 -inFormat org.apache.hadoop.mapred.KeyValueTextInputFormat -outFormat org.apache.hadoop.mapred.TextOutputFormat -joinOp outer datajoin/input datajoin/output -outKey org.apache.hadoop.io.Text -outValue org.apache.hadoop.mapred.join.TupleWritable
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
    Job started: Tue Apr 22 15:05:33 GMT-08:00 2008
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
    08/04/22 15:05:33 INFO mapred.FileInputFormat: Total input paths to process : 2
    java.lang.NullPointerException
    at org.apache.hadoop.mapred.KeyValueTextInputFormat.isSplitable(KeyValueTextInputFormat.java:44)
    at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:185)
    at org.apache.hadoop.mapred.join.Parser$WNode.getSplits(Parser.java:304)
    at org.apache.hadoop.mapred.join.Parser$CNode.getSplits(Parser.java:374)
    at org.apache.hadoop.mapred.join.CompositeInputFormat.getSplits(CompositeInputFormat.java:129)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:542)
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:803)
    at org.apache.hadoop.examples.Join.run(Join.java:149)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.examples.Join.main(Join.java:158)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:52)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jason (JIRA) at Apr 22, 2008 at 11:52 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jason updated HADOOP-3299:
    --------------------------

    Attachment: keyvaluetext.diff

    ensure that compressionCodes is initialized in isSplittable before dereferencing it.
    org.apache.hadoop.mapred.join.CompositeInputFormat does not initialize TextInput format files with the configuration resulting in an NullPointerException
    ----------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3299
    URL: https://issues.apache.org/jira/browse/HADOOP-3299
    Project: Hadoop Core
    Issue Type: Bug
    Components: io, mapred
    Affects Versions: 0.16.3
    Reporter: Jason
    Attachments: keyvaluetext.diff, text.diff


    The input formats are not initialized with the Configuration object before the isSplitable method is called.
    bin/hadoop jar hadoop-0.16.3-examples.jar join -r 1 -inFormat org.apache.hadoop.mapred.KeyValueTextInputFormat -outFormat org.apache.hadoop.mapred.TextOutputFormat -joinOp outer datajoin/input datajoin/output -outKey org.apache.hadoop.io.Text -outValue org.apache.hadoop.mapred.join.TupleWritable
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
    Job started: Tue Apr 22 15:05:33 GMT-08:00 2008
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
    08/04/22 15:05:33 INFO mapred.FileInputFormat: Total input paths to process : 2
    java.lang.NullPointerException
    at org.apache.hadoop.mapred.KeyValueTextInputFormat.isSplitable(KeyValueTextInputFormat.java:44)
    at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:185)
    at org.apache.hadoop.mapred.join.Parser$WNode.getSplits(Parser.java:304)
    at org.apache.hadoop.mapred.join.Parser$CNode.getSplits(Parser.java:374)
    at org.apache.hadoop.mapred.join.CompositeInputFormat.getSplits(CompositeInputFormat.java:129)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:542)
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:803)
    at org.apache.hadoop.examples.Join.run(Join.java:149)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.examples.Join.main(Join.java:158)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:52)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Apr 26, 2008 at 12:49 am
    [ https://issues.apache.org/jira/browse/HADOOP-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3299:
    ----------------------------------

    Attachment: 3299-0.patch

    The issue is with o.a.h.mapred.join.Parser. When it instantiates an InputFormat through ReflectionUtils::newInstance, it passes in a null JobConf so the InputFormat- if {{Configurable}} or {{JobConfigurable}}- remains uninitialized.

    The test case includes a separate class, {{ConfigurableInputFormat}} because the Parser doesn't support inner classes (i.e. '$' is unexpected in its grammar).
    org.apache.hadoop.mapred.join.CompositeInputFormat does not initialize TextInput format files with the configuration resulting in an NullPointerException
    ----------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3299
    URL: https://issues.apache.org/jira/browse/HADOOP-3299
    Project: Hadoop Core
    Issue Type: Bug
    Components: io, mapred
    Affects Versions: 0.16.3
    Reporter: Jason
    Fix For: 0.18.0

    Attachments: 3299-0.patch, keyvaluetext.diff, text.diff


    The input formats are not initialized with the Configuration object before the isSplitable method is called.
    bin/hadoop jar hadoop-0.16.3-examples.jar join -r 1 -inFormat org.apache.hadoop.mapred.KeyValueTextInputFormat -outFormat org.apache.hadoop.mapred.TextOutputFormat -joinOp outer datajoin/input datajoin/output -outKey org.apache.hadoop.io.Text -outValue org.apache.hadoop.mapred.join.TupleWritable
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
    Job started: Tue Apr 22 15:05:33 GMT-08:00 2008
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
    08/04/22 15:05:33 INFO mapred.FileInputFormat: Total input paths to process : 2
    java.lang.NullPointerException
    at org.apache.hadoop.mapred.KeyValueTextInputFormat.isSplitable(KeyValueTextInputFormat.java:44)
    at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:185)
    at org.apache.hadoop.mapred.join.Parser$WNode.getSplits(Parser.java:304)
    at org.apache.hadoop.mapred.join.Parser$CNode.getSplits(Parser.java:374)
    at org.apache.hadoop.mapred.join.CompositeInputFormat.getSplits(CompositeInputFormat.java:129)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:542)
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:803)
    at org.apache.hadoop.examples.Join.run(Join.java:149)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.examples.Join.main(Join.java:158)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:52)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Apr 26, 2008 at 12:49 am
    [ https://issues.apache.org/jira/browse/HADOOP-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3299:
    ----------------------------------

    Fix Version/s: 0.18.0
    Status: Patch Available (was: Open)
    org.apache.hadoop.mapred.join.CompositeInputFormat does not initialize TextInput format files with the configuration resulting in an NullPointerException
    ----------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3299
    URL: https://issues.apache.org/jira/browse/HADOOP-3299
    Project: Hadoop Core
    Issue Type: Bug
    Components: io, mapred
    Affects Versions: 0.16.3
    Reporter: Jason
    Fix For: 0.18.0

    Attachments: 3299-0.patch, keyvaluetext.diff, text.diff


    The input formats are not initialized with the Configuration object before the isSplitable method is called.
    bin/hadoop jar hadoop-0.16.3-examples.jar join -r 1 -inFormat org.apache.hadoop.mapred.KeyValueTextInputFormat -outFormat org.apache.hadoop.mapred.TextOutputFormat -joinOp outer datajoin/input datajoin/output -outKey org.apache.hadoop.io.Text -outValue org.apache.hadoop.mapred.join.TupleWritable
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
    Job started: Tue Apr 22 15:05:33 GMT-08:00 2008
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
    08/04/22 15:05:33 INFO mapred.FileInputFormat: Total input paths to process : 2
    java.lang.NullPointerException
    at org.apache.hadoop.mapred.KeyValueTextInputFormat.isSplitable(KeyValueTextInputFormat.java:44)
    at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:185)
    at org.apache.hadoop.mapred.join.Parser$WNode.getSplits(Parser.java:304)
    at org.apache.hadoop.mapred.join.Parser$CNode.getSplits(Parser.java:374)
    at org.apache.hadoop.mapred.join.CompositeInputFormat.getSplits(CompositeInputFormat.java:129)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:542)
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:803)
    at org.apache.hadoop.examples.Join.run(Join.java:149)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.examples.Join.main(Join.java:158)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:52)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Apr 26, 2008 at 12:51 am
    [ https://issues.apache.org/jira/browse/HADOOP-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas reassigned HADOOP-3299:
    -------------------------------------

    Assignee: Chris Douglas
    org.apache.hadoop.mapred.join.CompositeInputFormat does not initialize TextInput format files with the configuration resulting in an NullPointerException
    ----------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3299
    URL: https://issues.apache.org/jira/browse/HADOOP-3299
    Project: Hadoop Core
    Issue Type: Bug
    Components: io, mapred
    Affects Versions: 0.16.3
    Reporter: Jason
    Assignee: Chris Douglas
    Fix For: 0.18.0

    Attachments: 3299-0.patch, keyvaluetext.diff, text.diff


    The input formats are not initialized with the Configuration object before the isSplitable method is called.
    bin/hadoop jar hadoop-0.16.3-examples.jar join -r 1 -inFormat org.apache.hadoop.mapred.KeyValueTextInputFormat -outFormat org.apache.hadoop.mapred.TextOutputFormat -joinOp outer datajoin/input datajoin/output -outKey org.apache.hadoop.io.Text -outValue org.apache.hadoop.mapred.join.TupleWritable
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
    Job started: Tue Apr 22 15:05:33 GMT-08:00 2008
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
    08/04/22 15:05:33 INFO mapred.FileInputFormat: Total input paths to process : 2
    java.lang.NullPointerException
    at org.apache.hadoop.mapred.KeyValueTextInputFormat.isSplitable(KeyValueTextInputFormat.java:44)
    at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:185)
    at org.apache.hadoop.mapred.join.Parser$WNode.getSplits(Parser.java:304)
    at org.apache.hadoop.mapred.join.Parser$CNode.getSplits(Parser.java:374)
    at org.apache.hadoop.mapred.join.CompositeInputFormat.getSplits(CompositeInputFormat.java:129)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:542)
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:803)
    at org.apache.hadoop.examples.Join.run(Join.java:149)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.examples.Join.main(Join.java:158)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:52)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Apr 26, 2008 at 2:05 am
    [ https://issues.apache.org/jira/browse/HADOOP-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12592546#action_12592546 ]

    Hadoop QA commented on HADOOP-3299:
    -----------------------------------

    +1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12380974/3299-0.patch
    against trunk revision 645773.

    @author +1. The patch does not contain any @author tags.

    tests included +1. The patch appears to include 6 new or modified tests.

    javadoc +1. The javadoc tool did not generate any warning messages.

    javac +1. The applied patch does not generate any new javac compiler warnings.

    release audit +1. The applied patch does not generate any new release audit warnings.

    findbugs +1. The patch does not introduce any new Findbugs warnings.

    core tests +1. The patch passed core unit tests.

    contrib tests +1. The patch passed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2334/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2334/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2334/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2334/console

    This message is automatically generated.
    org.apache.hadoop.mapred.join.CompositeInputFormat does not initialize TextInput format files with the configuration resulting in an NullPointerException
    ----------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3299
    URL: https://issues.apache.org/jira/browse/HADOOP-3299
    Project: Hadoop Core
    Issue Type: Bug
    Components: io, mapred
    Affects Versions: 0.16.3
    Reporter: Jason
    Assignee: Chris Douglas
    Fix For: 0.18.0

    Attachments: 3299-0.patch, keyvaluetext.diff, text.diff


    The input formats are not initialized with the Configuration object before the isSplitable method is called.
    bin/hadoop jar hadoop-0.16.3-examples.jar join -r 1 -inFormat org.apache.hadoop.mapred.KeyValueTextInputFormat -outFormat org.apache.hadoop.mapred.TextOutputFormat -joinOp outer datajoin/input datajoin/output -outKey org.apache.hadoop.io.Text -outValue org.apache.hadoop.mapred.join.TupleWritable
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
    Job started: Tue Apr 22 15:05:33 GMT-08:00 2008
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
    08/04/22 15:05:33 INFO mapred.FileInputFormat: Total input paths to process : 2
    java.lang.NullPointerException
    at org.apache.hadoop.mapred.KeyValueTextInputFormat.isSplitable(KeyValueTextInputFormat.java:44)
    at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:185)
    at org.apache.hadoop.mapred.join.Parser$WNode.getSplits(Parser.java:304)
    at org.apache.hadoop.mapred.join.Parser$CNode.getSplits(Parser.java:374)
    at org.apache.hadoop.mapred.join.CompositeInputFormat.getSplits(CompositeInputFormat.java:129)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:542)
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:803)
    at org.apache.hadoop.examples.Join.run(Join.java:149)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.examples.Join.main(Join.java:158)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:52)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Owen O'Malley (JIRA) at May 7, 2008 at 10:38 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Owen O'Malley updated HADOOP-3299:
    ----------------------------------

    Resolution: Fixed
    Hadoop Flags: [Reviewed]
    Status: Resolved (was: Patch Available)

    I just committed this. Thanks, Chris!
    org.apache.hadoop.mapred.join.CompositeInputFormat does not initialize TextInput format files with the configuration resulting in an NullPointerException
    ----------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3299
    URL: https://issues.apache.org/jira/browse/HADOOP-3299
    Project: Hadoop Core
    Issue Type: Bug
    Components: io, mapred
    Affects Versions: 0.16.3
    Reporter: Jason
    Assignee: Chris Douglas
    Fix For: 0.18.0

    Attachments: 3299-0.patch, keyvaluetext.diff, text.diff


    The input formats are not initialized with the Configuration object before the isSplitable method is called.
    bin/hadoop jar hadoop-0.16.3-examples.jar join -r 1 -inFormat org.apache.hadoop.mapred.KeyValueTextInputFormat -outFormat org.apache.hadoop.mapred.TextOutputFormat -joinOp outer datajoin/input datajoin/output -outKey org.apache.hadoop.io.Text -outValue org.apache.hadoop.mapred.join.TupleWritable
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
    Job started: Tue Apr 22 15:05:33 GMT-08:00 2008
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
    08/04/22 15:05:33 INFO mapred.FileInputFormat: Total input paths to process : 2
    java.lang.NullPointerException
    at org.apache.hadoop.mapred.KeyValueTextInputFormat.isSplitable(KeyValueTextInputFormat.java:44)
    at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:185)
    at org.apache.hadoop.mapred.join.Parser$WNode.getSplits(Parser.java:304)
    at org.apache.hadoop.mapred.join.Parser$CNode.getSplits(Parser.java:374)
    at org.apache.hadoop.mapred.join.CompositeInputFormat.getSplits(CompositeInputFormat.java:129)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:542)
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:803)
    at org.apache.hadoop.examples.Join.run(Join.java:149)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.examples.Join.main(Join.java:158)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:52)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hudson (JIRA) at May 8, 2008 at 12:25 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12595206#action_12595206 ]

    Hudson commented on HADOOP-3299:
    --------------------------------

    Integrated in Hadoop-trunk #484 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/484/])
    org.apache.hadoop.mapred.join.CompositeInputFormat does not initialize TextInput format files with the configuration resulting in an NullPointerException
    ----------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3299
    URL: https://issues.apache.org/jira/browse/HADOOP-3299
    Project: Hadoop Core
    Issue Type: Bug
    Components: io, mapred
    Affects Versions: 0.16.3
    Reporter: Jason
    Assignee: Chris Douglas
    Fix For: 0.18.0

    Attachments: 3299-0.patch, keyvaluetext.diff, text.diff


    The input formats are not initialized with the Configuration object before the isSplitable method is called.
    bin/hadoop jar hadoop-0.16.3-examples.jar join -r 1 -inFormat org.apache.hadoop.mapred.KeyValueTextInputFormat -outFormat org.apache.hadoop.mapred.TextOutputFormat -joinOp outer datajoin/input datajoin/output -outKey org.apache.hadoop.io.Text -outValue org.apache.hadoop.mapred.join.TupleWritable
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
    Job started: Tue Apr 22 15:05:33 GMT-08:00 2008
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
    08/04/22 15:05:33 INFO mapred.FileInputFormat: Total input paths to process : 2
    java.lang.NullPointerException
    at org.apache.hadoop.mapred.KeyValueTextInputFormat.isSplitable(KeyValueTextInputFormat.java:44)
    at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:185)
    at org.apache.hadoop.mapred.join.Parser$WNode.getSplits(Parser.java:304)
    at org.apache.hadoop.mapred.join.Parser$CNode.getSplits(Parser.java:374)
    at org.apache.hadoop.mapred.join.CompositeInputFormat.getSplits(CompositeInputFormat.java:129)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:542)
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:803)
    at org.apache.hadoop.examples.Join.run(Join.java:149)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.examples.Join.main(Join.java:158)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:52)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Robert Chansler (JIRA) at Jun 30, 2008 at 9:51 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Robert Chansler updated HADOOP-3299:
    ------------------------------------

    Release Note: Changed the TextInputFormat and KeyValueTextInput classes to initialize the compressionCodecs member variable before dereferencing it. (was: Force the TextInputFormat and KeyValueTextInput classes to initialize the compressionCodecs member variable before dereferencing it.)
    org.apache.hadoop.mapred.join.CompositeInputFormat does not initialize TextInput format files with the configuration resulting in an NullPointerException
    ----------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3299
    URL: https://issues.apache.org/jira/browse/HADOOP-3299
    Project: Hadoop Core
    Issue Type: Bug
    Components: io, mapred
    Affects Versions: 0.16.3
    Reporter: Jason
    Assignee: Chris Douglas
    Fix For: 0.18.0

    Attachments: 3299-0.patch, keyvaluetext.diff, text.diff


    The input formats are not initialized with the Configuration object before the isSplitable method is called.
    bin/hadoop jar hadoop-0.16.3-examples.jar join -r 1 -inFormat org.apache.hadoop.mapred.KeyValueTextInputFormat -outFormat org.apache.hadoop.mapred.TextOutputFormat -joinOp outer datajoin/input datajoin/output -outKey org.apache.hadoop.io.Text -outValue org.apache.hadoop.mapred.join.TupleWritable
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
    Job started: Tue Apr 22 15:05:33 GMT-08:00 2008
    08/04/22 15:05:33 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
    08/04/22 15:05:33 INFO mapred.FileInputFormat: Total input paths to process : 2
    java.lang.NullPointerException
    at org.apache.hadoop.mapred.KeyValueTextInputFormat.isSplitable(KeyValueTextInputFormat.java:44)
    at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:185)
    at org.apache.hadoop.mapred.join.Parser$WNode.getSplits(Parser.java:304)
    at org.apache.hadoop.mapred.join.Parser$CNode.getSplits(Parser.java:374)
    at org.apache.hadoop.mapred.join.CompositeInputFormat.getSplits(CompositeInputFormat.java:129)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:542)
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:803)
    at org.apache.hadoop.examples.Join.run(Join.java:149)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.examples.Join.main(Join.java:158)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:52)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedApr 22, '08 at 11:50p
activeJun 30, '08 at 9:51p
posts10
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Robert Chansler (JIRA): 10 posts

People

Translate

site design / logo © 2022 Grokbase