Michele (aka pirroh) Catasta updated HADOOP-2366:
-------------------------------------------------
Affects Version/s: (was: 0.13.1)
Status: Patch Available (was: Open)
getStrings() trims leading and trailing whitespace by default now.
Patch includes also a simple unit test.
Space in the value for dfs.data.dir can cause great problems
------------------------------------------------------------
Key: HADOOP-2366
URL: https://issues.apache.org/jira/browse/HADOOP-2366
Project: Hadoop Core
Issue Type: Bug
Components: conf
Reporter: Ted Dunning
Assignee: Todd Lipcon
The following configuration causes problems:
<property>
<name>dfs.data.dir</name>
<value>/mnt/hstore2/hdfs, /home/foo/dfs</value>
<description>
Determines where on the local filesystem an DFS data node should store its bl
ocks. If this is a comma-delimited list of directories, then data will be stor
ed in all named directories, typically on different devices. Directories that
do not exist are ignored.
</description>
</property>
The problem is that the space after the comma causes the second directory for storage to be " /home/foo/dfs" which is in a directory named <SPACE> which contains a sub-dir named "home" in the hadoop datanodes default directory. This will typically cause the user's home partition to fill, but will be very hard for the user to understand since a directory with a whitespace name is hard to understand.
My proposed solution would be to trimLeft all path names from this and similar property after splitting on comma. This still allows spaces in file and directory names but avoids this problem.
--------------------------------------------------------------
Key: HADOOP-2366
URL: https://issues.apache.org/jira/browse/HADOOP-2366
Project: Hadoop Core
Issue Type: Bug
Components: conf
Reporter: Ted Dunning
Assignee: Todd Lipcon
The following configuration causes problems:
<property>
<name>dfs.data.dir</name>
<value>/mnt/hstore2/hdfs, /home/foo/dfs</value>
<description>
Determines where on the local filesystem an DFS data node should store its bl
ocks. If this is a comma-delimited list of directories, then data will be stor
ed in all named directories, typically on different devices. Directories that
do not exist are ignored.
</description>
</property>
The problem is that the space after the comma causes the second directory for storage to be " /home/foo/dfs" which is in a directory named <SPACE> which contains a sub-dir named "home" in the hadoop datanodes default directory. This will typically cause the user's home partition to fill, but will be very hard for the user to understand since a directory with a whitespace name is hard to understand.
My proposed solution would be to trimLeft all path names from this and similar property after splitting on comma. This still allows spaces in file and directory names but avoids this problem.
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
