FAQ
parameter dfs.replication is not reflected when put file into hadoop with fuse-dfs
----------------------------------------------------------------------------------

Key: HADOOP-4877
URL: https://issues.apache.org/jira/browse/HADOOP-4877
Project: Hadoop Core
Issue Type: Bug
Components: contrib/fuse-dfs
Affects Versions: 0.19.0
Environment: os:centos5.2
cpu:amd64
hadoop0.19.0
Reporter: zhuweimin


the $HADOOP_CONF_DIR is exist in the $CLASSPATH
and the dfs.replication is set to 3 with the following in hadoop-site.xml
<property>
<name>dfs.replication</name>
<value>1</value>
</property>

The file's replication is 3 when it be put into hadoop.

I think the reason is :
there is a hardcoding in the src\contrib\fuse-dfs\src\fuse_dfs.c

line 1337
if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 3, 0)) == NULL) {
line 1591
if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 3, 0)) == NULL) {

the fifth parameter is a hardcoding when call the function of hdfsOpenFile.

It is should set to 0. It is as follows.
line 1337
if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 0, 0)) == NULL) {
line 1591
if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 0, 0)) == NULL) {






--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • zhuweimin (JIRA) at Dec 19, 2008 at 2:10 am
    [ https://issues.apache.org/jira/browse/HADOOP-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    zhuweimin updated HADOOP-4877:
    ------------------------------

    Status: Patch Available (was: Open)
    parameter dfs.replication is not reflected when put file into hadoop with fuse-dfs
    ----------------------------------------------------------------------------------

    Key: HADOOP-4877
    URL: https://issues.apache.org/jira/browse/HADOOP-4877
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/fuse-dfs
    Affects Versions: 0.19.0
    Environment: os:centos5.2
    cpu:amd64
    hadoop0.19.0
    Reporter: zhuweimin

    the $HADOOP_CONF_DIR is exist in the $CLASSPATH
    and the dfs.replication is set to 3 with the following in hadoop-site.xml
    <property>
    <name>dfs.replication</name>
    <value>1</value>
    </property>
    The file's replication is 3 when it be put into hadoop.
    I think the reason is :
    there is a hardcoding in the src\contrib\fuse-dfs\src\fuse_dfs.c
    line 1337
    if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 3, 0)) == NULL) {
    line 1591
    if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 3, 0)) == NULL) {
    the fifth parameter is a hardcoding when call the function of hdfsOpenFile.
    It is should set to 0. It is as follows.
    line 1337
    if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 0, 0)) == NULL) {
    line 1591
    if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 0, 0)) == NULL) {
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • zhuweimin (JIRA) at Dec 19, 2008 at 2:14 am
    [ https://issues.apache.org/jira/browse/HADOOP-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    zhuweimin updated HADOOP-4877:
    ------------------------------

    Status: Open (was: Patch Available)
    parameter dfs.replication is not reflected when put file into hadoop with fuse-dfs
    ----------------------------------------------------------------------------------

    Key: HADOOP-4877
    URL: https://issues.apache.org/jira/browse/HADOOP-4877
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/fuse-dfs
    Affects Versions: 0.19.0
    Environment: os:centos5.2
    cpu:amd64
    hadoop0.19.0
    Reporter: zhuweimin

    the $HADOOP_CONF_DIR is exist in the $CLASSPATH
    and the dfs.replication is set to 3 with the following in hadoop-site.xml
    <property>
    <name>dfs.replication</name>
    <value>1</value>
    </property>
    The file's replication is 3 when it be put into hadoop.
    I think the reason is :
    there is a hardcoding in the src\contrib\fuse-dfs\src\fuse_dfs.c
    line 1337
    if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 3, 0)) == NULL) {
    line 1591
    if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 3, 0)) == NULL) {
    the fifth parameter is a hardcoding when call the function of hdfsOpenFile.
    It is should set to 0. It is as follows.
    line 1337
    if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 0, 0)) == NULL) {
    line 1591
    if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 0, 0)) == NULL) {
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • zhuweimin (JIRA) at Dec 22, 2008 at 2:40 am
    [ https://issues.apache.org/jira/browse/HADOOP-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    zhuweimin updated HADOOP-4877:
    ------------------------------

    Status: Patch Available (was: Open)
    parameter dfs.replication is not reflected when put file into hadoop with fuse-dfs
    ----------------------------------------------------------------------------------

    Key: HADOOP-4877
    URL: https://issues.apache.org/jira/browse/HADOOP-4877
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/fuse-dfs
    Affects Versions: 0.19.0
    Environment: os:centos5.2
    cpu:amd64
    hadoop0.19.0
    Reporter: zhuweimin

    the $HADOOP_CONF_DIR is exist in the $CLASSPATH
    and the dfs.replication is set to 3 with the following in hadoop-site.xml
    <property>
    <name>dfs.replication</name>
    <value>1</value>
    </property>
    The file's replication is 3 when it be put into hadoop.
    I think the reason is :
    there is a hardcoding in the src\contrib\fuse-dfs\src\fuse_dfs.c
    line 1337
    if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 3, 0)) == NULL) {
    line 1591
    if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 3, 0)) == NULL) {
    the fifth parameter is a hardcoding when call the function of hdfsOpenFile.
    It is should set to 0. It is as follows.
    line 1337
    if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 0, 0)) == NULL) {
    line 1591
    if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 0, 0)) == NULL) {
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • zhuweimin (JIRA) at Dec 22, 2008 at 2:42 am
    [ https://issues.apache.org/jira/browse/HADOOP-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    zhuweimin updated HADOOP-4877:
    ------------------------------

    Attachment: HADOOP-4877.txt.0.19
    parameter dfs.replication is not reflected when put file into hadoop with fuse-dfs
    ----------------------------------------------------------------------------------

    Key: HADOOP-4877
    URL: https://issues.apache.org/jira/browse/HADOOP-4877
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/fuse-dfs
    Affects Versions: 0.19.0
    Environment: os:centos5.2
    cpu:amd64
    hadoop0.19.0
    Reporter: zhuweimin
    Attachments: HADOOP-4877.txt.0.19


    the $HADOOP_CONF_DIR is exist in the $CLASSPATH
    and the dfs.replication is set to 3 with the following in hadoop-site.xml
    <property>
    <name>dfs.replication</name>
    <value>1</value>
    </property>
    The file's replication is 3 when it be put into hadoop.
    I think the reason is :
    there is a hardcoding in the src\contrib\fuse-dfs\src\fuse_dfs.c
    line 1337
    if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 3, 0)) == NULL) {
    line 1591
    if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 3, 0)) == NULL) {
    the fifth parameter is a hardcoding when call the function of hdfsOpenFile.
    It is should set to 0. It is as follows.
    line 1337
    if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 0, 0)) == NULL) {
    line 1591
    if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 0, 0)) == NULL) {
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Craig Macdonald (JIRA) at Jan 6, 2009 at 8:24 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Craig Macdonald updated HADOOP-4877:
    ------------------------------------

    Attachment: HADOOP-4877.txt.trunk

    Here is a patch for trunk. My only thoughts are that for truncate, it might be more uniform if the original file replication number was kept.
    parameter dfs.replication is not reflected when put file into hadoop with fuse-dfs
    ----------------------------------------------------------------------------------

    Key: HADOOP-4877
    URL: https://issues.apache.org/jira/browse/HADOOP-4877
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/fuse-dfs
    Affects Versions: 0.19.0
    Environment: os:centos5.2
    cpu:amd64
    hadoop0.19.0
    Reporter: zhuweimin
    Attachments: HADOOP-4877.txt.0.19, HADOOP-4877.txt.trunk


    the $HADOOP_CONF_DIR is exist in the $CLASSPATH
    and the dfs.replication is set to 3 with the following in hadoop-site.xml
    <property>
    <name>dfs.replication</name>
    <value>1</value>
    </property>
    The file's replication is 3 when it be put into hadoop.
    I think the reason is :
    there is a hardcoding in the src\contrib\fuse-dfs\src\fuse_dfs.c
    line 1337
    if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 3, 0)) == NULL) {
    line 1591
    if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 3, 0)) == NULL) {
    the fifth parameter is a hardcoding when call the function of hdfsOpenFile.
    It is should set to 0. It is as follows.
    line 1337
    if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 0, 0)) == NULL) {
    line 1591
    if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 0, 0)) == NULL) {
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedDec 16, '08 at 5:39a
activeJan 6, '09 at 8:24p
posts6
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Craig Macdonald (JIRA): 6 posts

People

Translate

site design / logo © 2022 Grokbase