FAQ
DFs copyFromLocal fails with NullPointerException for a single file
-------------------------------------------------------------------

Key: HADOOP-612
URL: http://issues.apache.org/jira/browse/HADOOP-612
Project: Hadoop
Issue Type: Bug
Components: dfs
Affects Versions: 0.7.1
Reporter: Sanjay Dahiya
Priority: Blocker


DFS copyFromLocal fails with NullPointerException when copying a single file to DFS. Copying a directory works.

public File getFile(String dirsProp, String path)
throws IOException {
String[] dirs = getStrings(dirsProp); <===== returns null for a single file
int hashCode = path.hashCode();
for (int i = 0; i < dirs.length; i++) { // try each local dir <==== Throws NullPointerException
int index = (hashCode+i & Integer.MAX_VALUE) % dirs.length;
File file = new File(dirs[index], path);
File dir = file.getParentFile();
if (dir.exists() || dir.mkdirs()) {
return file;
}
}
throw new IOException("No valid local directories in property: "+dirsProp);
}



Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.conf.Configuration.getFile(Configuration.java:397)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.newBackupFile(DFSClient.java:913)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.(DFSClient.java:276)
at org.apache.hadoop.dfs.DistributedFileSystem.createRaw(DistributedFileSystem.java:104)
at org.apache.hadoop.fs.FSDataOutputStream$Summer.(FSDataOutputStream.java:45)
at org.apache.hadoop.fs.FSDataOutputStream.(FileSystem.java:271)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:178)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:77)
at org.apache.hadoop.dfs.DistributedFileSystem.copyFromLocalFile(DistributedFileSystem.java:186)
at org.apache.hadoop.dfs.DFSShell.copyFromLocal(DFSShell.java:45)
at org.apache.hadoop.dfs.DFSShell.run(DFSShell.java:516)
at org.apache.hadoop.util.ToolBase.doMain(ToolBase.java:187)
at org.apache.hadoop.dfs.DFSShell.main(DFSShell.java:570)


--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

Search Discussions

  • dhruba borthakur (JIRA) at Oct 26, 2006 at 10:13 pm
    [ http://issues.apache.org/jira/browse/HADOOP-612?page=comments#action_12445022 ]

    dhruba borthakur commented on HADOOP-612:
    -----------------------------------------

    Hi sanjay,
    Is it likely that the property "dfs.client.buffer.dir" is not set in your configuration? if so, then this bug can be fixed by simply putting a try, catch block in the code segment you pointed out. Please let me know if you want me to fix it.
    DFs copyFromLocal fails with NullPointerException for a single file
    -------------------------------------------------------------------

    Key: HADOOP-612
    URL: http://issues.apache.org/jira/browse/HADOOP-612
    Project: Hadoop
    Issue Type: Bug
    Components: dfs
    Affects Versions: 0.7.1
    Reporter: Sanjay Dahiya
    Priority: Blocker

    DFS copyFromLocal fails with NullPointerException when copying a single file to DFS. Copying a directory works.
    public File getFile(String dirsProp, String path)
    throws IOException {
    String[] dirs = getStrings(dirsProp); <===== returns null for a single file
    int hashCode = path.hashCode();
    for (int i = 0; i < dirs.length; i++) { // try each local dir <==== Throws NullPointerException
    int index = (hashCode+i & Integer.MAX_VALUE) % dirs.length;
    File file = new File(dirs[index], path);
    File dir = file.getParentFile();
    if (dir.exists() || dir.mkdirs()) {
    return file;
    }
    }
    throw new IOException("No valid local directories in property: "+dirsProp);
    }
    Exception in thread "main" java.lang.NullPointerException
    at org.apache.hadoop.conf.Configuration.getFile(Configuration.java:397)
    at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.newBackupFile(DFSClient.java:913)
    at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.<init>(DFSClient.java:903)
    at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:276)
    at org.apache.hadoop.dfs.DistributedFileSystem.createRaw(DistributedFileSystem.java:104)
    at org.apache.hadoop.fs.FSDataOutputStream$Summer.<init>(FSDataOutputStream.java:56)
    at org.apache.hadoop.fs.FSDataOutputStream$Summer.<init>(FSDataOutputStream.java:45)
    at org.apache.hadoop.fs.FSDataOutputStream.<init>(FSDataOutputStream.java:146)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:271)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:178)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:77)
    at org.apache.hadoop.dfs.DistributedFileSystem.copyFromLocalFile(DistributedFileSystem.java:186)
    at org.apache.hadoop.dfs.DFSShell.copyFromLocal(DFSShell.java:45)
    at org.apache.hadoop.dfs.DFSShell.run(DFSShell.java:516)
    at org.apache.hadoop.util.ToolBase.doMain(ToolBase.java:187)
    at org.apache.hadoop.dfs.DFSShell.main(DFSShell.java:570)
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see: http://www.atlassian.com/software/jira
  • Sanjay Dahiya (JIRA) at Oct 30, 2006 at 8:47 am
    [ http://issues.apache.org/jira/browse/HADOOP-612?page=comments#action_12445519 ]

    Sanjay Dahiya commented on HADOOP-612:
    --------------------------------------

    sorry for late response, missed it somehow.
    I will cross check and get back soon.
    DFs copyFromLocal fails with NullPointerException for a single file
    -------------------------------------------------------------------

    Key: HADOOP-612
    URL: http://issues.apache.org/jira/browse/HADOOP-612
    Project: Hadoop
    Issue Type: Bug
    Components: dfs
    Affects Versions: 0.7.1
    Reporter: Sanjay Dahiya
    Priority: Blocker

    DFS copyFromLocal fails with NullPointerException when copying a single file to DFS. Copying a directory works.
    public File getFile(String dirsProp, String path)
    throws IOException {
    String[] dirs = getStrings(dirsProp); <===== returns null for a single file
    int hashCode = path.hashCode();
    for (int i = 0; i < dirs.length; i++) { // try each local dir <==== Throws NullPointerException
    int index = (hashCode+i & Integer.MAX_VALUE) % dirs.length;
    File file = new File(dirs[index], path);
    File dir = file.getParentFile();
    if (dir.exists() || dir.mkdirs()) {
    return file;
    }
    }
    throw new IOException("No valid local directories in property: "+dirsProp);
    }
    Exception in thread "main" java.lang.NullPointerException
    at org.apache.hadoop.conf.Configuration.getFile(Configuration.java:397)
    at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.newBackupFile(DFSClient.java:913)
    at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.<init>(DFSClient.java:903)
    at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:276)
    at org.apache.hadoop.dfs.DistributedFileSystem.createRaw(DistributedFileSystem.java:104)
    at org.apache.hadoop.fs.FSDataOutputStream$Summer.<init>(FSDataOutputStream.java:56)
    at org.apache.hadoop.fs.FSDataOutputStream$Summer.<init>(FSDataOutputStream.java:45)
    at org.apache.hadoop.fs.FSDataOutputStream.<init>(FSDataOutputStream.java:146)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:271)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:178)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:77)
    at org.apache.hadoop.dfs.DistributedFileSystem.copyFromLocalFile(DistributedFileSystem.java:186)
    at org.apache.hadoop.dfs.DFSShell.copyFromLocal(DFSShell.java:45)
    at org.apache.hadoop.dfs.DFSShell.run(DFSShell.java:516)
    at org.apache.hadoop.util.ToolBase.doMain(ToolBase.java:187)
    at org.apache.hadoop.dfs.DFSShell.main(DFSShell.java:570)
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see: http://www.atlassian.com/software/jira
  • Marco Nicosia (JIRA) at Feb 9, 2007 at 3:14 am
    [ https://issues.apache.org/jira/browse/HADOOP-612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12471569 ]

    Marco Nicosia commented on HADOOP-612:
    --------------------------------------

    Is it possible to close this bug?

    DFs copyFromLocal fails with NullPointerException for a single file
    -------------------------------------------------------------------

    Key: HADOOP-612
    URL: https://issues.apache.org/jira/browse/HADOOP-612
    Project: Hadoop
    Issue Type: Bug
    Components: dfs
    Affects Versions: 0.7.1
    Reporter: Sanjay Dahiya
    Assigned To: Sameer Paranjpye
    Priority: Blocker

    DFS copyFromLocal fails with NullPointerException when copying a single file to DFS. Copying a directory works.
    public File getFile(String dirsProp, String path)
    throws IOException {
    String[] dirs = getStrings(dirsProp); <===== returns null for a single file
    int hashCode = path.hashCode();
    for (int i = 0; i < dirs.length; i++) { // try each local dir <==== Throws NullPointerException
    int index = (hashCode+i & Integer.MAX_VALUE) % dirs.length;
    File file = new File(dirs[index], path);
    File dir = file.getParentFile();
    if (dir.exists() || dir.mkdirs()) {
    return file;
    }
    }
    throw new IOException("No valid local directories in property: "+dirsProp);
    }
    Exception in thread "main" java.lang.NullPointerException
    at org.apache.hadoop.conf.Configuration.getFile(Configuration.java:397)
    at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.newBackupFile(DFSClient.java:913)
    at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.<init>(DFSClient.java:903)
    at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:276)
    at org.apache.hadoop.dfs.DistributedFileSystem.createRaw(DistributedFileSystem.java:104)
    at org.apache.hadoop.fs.FSDataOutputStream$Summer.<init>(FSDataOutputStream.java:56)
    at org.apache.hadoop.fs.FSDataOutputStream$Summer.<init>(FSDataOutputStream.java:45)
    at org.apache.hadoop.fs.FSDataOutputStream.<init>(FSDataOutputStream.java:146)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:271)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:178)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:77)
    at org.apache.hadoop.dfs.DistributedFileSystem.copyFromLocalFile(DistributedFileSystem.java:186)
    at org.apache.hadoop.dfs.DFSShell.copyFromLocal(DFSShell.java:45)
    at org.apache.hadoop.dfs.DFSShell.run(DFSShell.java:516)
    at org.apache.hadoop.util.ToolBase.doMain(ToolBase.java:187)
    at org.apache.hadoop.dfs.DFSShell.main(DFSShell.java:570)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • dhruba borthakur (JIRA) at May 1, 2007 at 11:40 pm
    [ https://issues.apache.org/jira/browse/HADOOP-612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    dhruba borthakur resolved HADOOP-612.
    -------------------------------------

    Resolution: Fixed

    I am closing this one because this does not happen anymore.
    DFs copyFromLocal fails with NullPointerException for a single file
    -------------------------------------------------------------------

    Key: HADOOP-612
    URL: https://issues.apache.org/jira/browse/HADOOP-612
    Project: Hadoop
    Issue Type: Bug
    Components: dfs
    Affects Versions: 0.7.1
    Reporter: Sanjay Dahiya
    Assigned To: Sameer Paranjpye
    Priority: Blocker

    DFS copyFromLocal fails with NullPointerException when copying a single file to DFS. Copying a directory works.
    public File getFile(String dirsProp, String path)
    throws IOException {
    String[] dirs = getStrings(dirsProp); <===== returns null for a single file
    int hashCode = path.hashCode();
    for (int i = 0; i < dirs.length; i++) { // try each local dir <==== Throws NullPointerException
    int index = (hashCode+i & Integer.MAX_VALUE) % dirs.length;
    File file = new File(dirs[index], path);
    File dir = file.getParentFile();
    if (dir.exists() || dir.mkdirs()) {
    return file;
    }
    }
    throw new IOException("No valid local directories in property: "+dirsProp);
    }
    Exception in thread "main" java.lang.NullPointerException
    at org.apache.hadoop.conf.Configuration.getFile(Configuration.java:397)
    at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.newBackupFile(DFSClient.java:913)
    at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.<init>(DFSClient.java:903)
    at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:276)
    at org.apache.hadoop.dfs.DistributedFileSystem.createRaw(DistributedFileSystem.java:104)
    at org.apache.hadoop.fs.FSDataOutputStream$Summer.<init>(FSDataOutputStream.java:56)
    at org.apache.hadoop.fs.FSDataOutputStream$Summer.<init>(FSDataOutputStream.java:45)
    at org.apache.hadoop.fs.FSDataOutputStream.<init>(FSDataOutputStream.java:146)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:271)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:178)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:77)
    at org.apache.hadoop.dfs.DistributedFileSystem.copyFromLocalFile(DistributedFileSystem.java:186)
    at org.apache.hadoop.dfs.DFSShell.copyFromLocal(DFSShell.java:45)
    at org.apache.hadoop.dfs.DFSShell.run(DFSShell.java:516)
    at org.apache.hadoop.util.ToolBase.doMain(ToolBase.java:187)
    at org.apache.hadoop.dfs.DFSShell.main(DFSShell.java:570)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedOct 18, '06 at 11:35a
activeMay 1, '07 at 11:40p
posts5
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

dhruba borthakur (JIRA): 5 posts

People

Translate

site design / logo © 2022 Grokbase