FAQ
hadoop dfs -ls, -get, -mv command's source/destination URI are inconsistent
---------------------------------------------------------------------------

Key: HADOOP-1967
URL: https://issues.apache.org/jira/browse/HADOOP-1967
Project: Hadoop
Issue Type: Bug
Components: dfs
Affects Versions: 0.14.1
Reporter: lohit vijayarenu


While specifying source/destination path for hadoop dfs -ls, -get, -mv, -cp commands, we have some inconsistency related to 'hdfs://' scheme.

Particularly, few of the commands accept both formats
[1] hdfs:///user/lohit/testfile
[2] hdfs://myhost:8020/user/lohit/testfile

and few commands accept only paths, which have authority (host:port)
[2] hdfs://myhost:8020/user/lohit/testfile

below are examples
hadoop dfs -ls (works for both formats)
{quote}
[lohit@krygw1000 ~]$ hadoop dfs -ls hdfs://kry-nn1:8020/user/lohit/ranges
Found 1 items
/user/lohit/ranges <r 3> 24 1970-01-01 00:00
[lohit@krygw1000 ~]$ hadoop dfs -ls hdfs:///user/lohit/ranges
Found 1 items
{quote}


hadoop dfs -get (works for only format [2])
{quote}
[lohit@krygw1000 ~]$ hadoop dfs -get hdfs:///user/lohit/ranges .
Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS:
hdfs:/user/lohit/ranges, expected: hdfs://kry-nn1:8020
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:204)
at
org.apache.hadoop.dfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:108)
at
org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:104)
at
org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:319)
at org.apache.hadoop.fs.FileSystem.isDirectory(FileSystem.java:423)
at org.apache.hadoop.fs.FsShell.copyToLocal(FsShell.java:177)
at org.apache.hadoop.fs.FsShell.copyToLocal(FsShell.java:155)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:1233)
at org.apache.hadoop.util.ToolBase.doMain(ToolBase.java:187)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:1342)
[lohit@krygw1000 ~]$ hadoop dfs -get hdfs://kry-nn1:8020/user/lohit/ranges .
[lohit@krygw1000 ~]$ ls ./ranges
./ranges
[lohit@krygw1000 ~]$
{quote}

hadoop dfs -mv / -cp command. source path accepts both format [1] and [2], while destination accepts only [2].
{quote}
[lohit@krygw1000 ~]$ hadoop dfs -cp hdfs://kry-nn1:8020/user/lohit/ranges.test2
hdfs:///user/lohit/ranges.test
Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS:
hdfs:/user/lohit/ranges.test, expected: hdfs://kry-nn1:8020
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:204)
at
org.apache.hadoop.dfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:108)
at
org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:104)
at
org.apache.hadoop.dfs.DistributedFileSystem.exists(DistributedFileSystem.java:162)
at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:269)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:117)
at org.apache.hadoop.fs.FsShell.copy(FsShell.java:691)
at org.apache.hadoop.fs.FsShell.copy(FsShell.java:727)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:1260)
at org.apache.hadoop.util.ToolBase.doMain(ToolBase.java:187)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:1342)
[lohit@krygw1000 ~]$ hadoop dfs -cp hdfs:///user/lohit/ranges.test2
hdfs://kry-nn1:8020/user/lohit/ranges.test
[lohit@krygw1000 ~]$
{quote}

We should have a consistent URI naming convention across all commands.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedSep 28, '07 at 9:33p
activeSep 28, '07 at 9:33p
posts1
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

lohit vijayarenu (JIRA): 1 post

People

Translate

site design / logo © 2022 Grokbase