FAQ
Calling listPaths() on a dfs:// path doesn't always preserve the dfs relationship
---------------------------------------------------------------------------------

Key: HADOOP-1107
URL: https://issues.apache.org/jira/browse/HADOOP-1107
Project: Hadoop
Issue Type: Bug
Components: dfs
Reporter: Bryan Pendleton


If you create a Path object from a URI string (ie, dfs://namenode:port/filename), and your default filesystem *isn't* that same dfs namenode, when you call listPaths() on that Path object, the objects you get back are DfsPath() objects which haven't been constructed with the right details to connect to the same host.

I tried this on a debug machine that hadn't been configured to use our DFS configuration. The listPaths() completes, populating with the actual contents of the Path in DFS - but the Path objects no longer refer to the dfs://namenode:port/ that they need to.

This appears to be a problem with DistributedFileSystem.listPaths(), and/or DfsPath(). When you create a DfsPath(), it takes a DfsInfo() object, which isn't derived from any object which remembers the name or port of the namenode.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Doug Cutting (JIRA) at May 29, 2007 at 5:55 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12499864 ]

    Doug Cutting commented on HADOOP-1107:
    --------------------------------------

    Was this fixed by HADOOP-1226?
    Calling listPaths() on a dfs:// path doesn't always preserve the dfs relationship
    ---------------------------------------------------------------------------------

    Key: HADOOP-1107
    URL: https://issues.apache.org/jira/browse/HADOOP-1107
    Project: Hadoop
    Issue Type: Bug
    Components: dfs
    Reporter: Bryan Pendleton

    If you create a Path object from a URI string (ie, dfs://namenode:port/filename), and your default filesystem *isn't* that same dfs namenode, when you call listPaths() on that Path object, the objects you get back are DfsPath() objects which haven't been constructed with the right details to connect to the same host.
    I tried this on a debug machine that hadn't been configured to use our DFS configuration. The listPaths() completes, populating with the actual contents of the Path in DFS - but the Path objects no longer refer to the dfs://namenode:port/ that they need to.
    This appears to be a problem with DistributedFileSystem.listPaths(), and/or DfsPath(). When you create a DfsPath(), it takes a DfsInfo() object, which isn't derived from any object which remembers the name or port of the namenode.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedMar 10, '07 at 2:08a
activeMay 29, '07 at 5:55p
posts2
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Doug Cutting (JIRA): 2 posts

People

Translate

site design / logo © 2022 Grokbase