Grokbase › Groups › Hadoop › common-user › May 2009
FAQ
Hi, I have a problem to use hdfs.

I mounted hdfs using fuse-dfs.

I created a dummy file for 'Xen' in hdfs and then formated the dummy file using 'mke2fs'.

But the operation was faced error. The error message is as follows.

[[email protected] hdfs]# mke2fs -j -F ./file_dumy
mke2fs 1.40.2 (12-Jul-2007)
./file_dumy: Input/output error while setting up superblock
Also, I copyed an image file of xen to hdfs. But Xen couldn't the image files in hdfs.

[email protected] hdfs]# fdisk -l fedora6_demo.img
last_lba(): I don't know how to handle files with mode 81a4
You must set cylinders.
You can do this from the extra functions menu.

Disk fedora6_demo.img: 0 MB, 0 bytes
255 heads, 63 sectors/track, 0 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
fedora6_demo.img1 * 1 156 1253038+ 83 Linux

Could you answer me anything about this problem.

Thank you.

Search Discussions

  • Taeho Kang at May 22, 2009 at 9:07 am ⇧
    I don't think HDFS is a good place to store your Xen image file as it will
    likely be updated/appended frequently in small blocks. With the way HDFS is
    designed for, you can't quite use it like a regular filesystem (e.g. ones
    that support frequent small block appends/updates in files). My suggestion
    is to use another filesystem like NAS or SAN.

    /Taeho

    2009/5/22 신승엽 <[email protected]>
    Hi, I have a problem to use hdfs.

    I mounted hdfs using fuse-dfs.

    I created a dummy file for 'Xen' in hdfs and then formated the dummy file
    using 'mke2fs'.

    But the operation was faced error. The error message is as follows.

    [[email protected] hdfs]# mke2fs -j -F ./file_dumy
    mke2fs 1.40.2 (12-Jul-2007)
    ./file_dumy: Input/output error while setting up superblock
    Also, I copyed an image file of xen to hdfs. But Xen couldn't the image
    files in hdfs.

    [email protected] hdfs]# fdisk -l fedora6_demo.img
    last_lba(): I don't know how to handle files with mode 81a4
    You must set cylinders.
    You can do this from the extra functions menu.

    Disk fedora6_demo.img: 0 MB, 0 bytes
    255 heads, 63 sectors/track, 0 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot Start End Blocks Id System
    fedora6_demo.img1 * 1 156 1253038+ 83 Linux

    Could you answer me anything about this problem.

    Thank you.
  • Aaron Kimball at May 22, 2009 at 6:10 pm ⇧
    More specifically:

    HDFS does not support operations such as opening a file for write/append
    after it has already been closed, or seeking to a new location in a writer.
    You can only write files linearly; all other operations will return a "not
    supported" error.

    You'll also find that random-access read performance, while implemented, is
    not particularly high-throughput. For serving Xen images even in read-only
    mode, you'll likely have much better luck with a different FS.

    - Aaron


    2009/5/22 Taeho Kang <[email protected]>
    I don't think HDFS is a good place to store your Xen image file as it will
    likely be updated/appended frequently in small blocks. With the way HDFS is
    designed for, you can't quite use it like a regular filesystem (e.g. ones
    that support frequent small block appends/updates in files). My suggestion
    is to use another filesystem like NAS or SAN.

    /Taeho

    2009/5/22 신승엽 <[email protected]>
    Hi, I have a problem to use hdfs.

    I mounted hdfs using fuse-dfs.

    I created a dummy file for 'Xen' in hdfs and then formated the dummy file
    using 'mke2fs'.

    But the operation was faced error. The error message is as follows.

    [[email protected] hdfs]# mke2fs -j -F ./file_dumy
    mke2fs 1.40.2 (12-Jul-2007)
    ./file_dumy: Input/output error while setting up superblock
    Also, I copyed an image file of xen to hdfs. But Xen couldn't the image
    files in hdfs.

    [email protected] hdfs]# fdisk -l fedora6_demo.img
    last_lba(): I don't know how to handle files with mode 81a4
    You must set cylinders.
    You can do this from the extra functions menu.

    Disk fedora6_demo.img: 0 MB, 0 bytes
    255 heads, 63 sectors/track, 0 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot Start End Blocks Id System
    fedora6_demo.img1 * 1 156 1253038+ 83 Linux

    Could you answer me anything about this problem.

    Thank you.
  • Amr Awadallah at May 22, 2009 at 9:43 pm ⇧
    You should take a look at GlusterFS, it should be a good fit for this
    kind of workload:

    http://www.gluster.org/

    -- amr

    Aaron Kimball wrote:
    More specifically:

    HDFS does not support operations such as opening a file for write/append
    after it has already been closed, or seeking to a new location in a writer.
    You can only write files linearly; all other operations will return a "not
    supported" error.

    You'll also find that random-access read performance, while implemented, is
    not particularly high-throughput. For serving Xen images even in read-only
    mode, you'll likely have much better luck with a different FS.

    - Aaron


    2009/5/22 Taeho Kang <[email protected]>

    I don't think HDFS is a good place to store your Xen image file as it will
    likely be updated/appended frequently in small blocks. With the way HDFS is
    designed for, you can't quite use it like a regular filesystem (e.g. ones
    that support frequent small block appends/updates in files). My suggestion
    is to use another filesystem like NAS or SAN.

    /Taeho

    2009/5/22 신승엽 <[email protected]>

    Hi, I have a problem to use hdfs.

    I mounted hdfs using fuse-dfs.

    I created a dummy file for 'Xen' in hdfs and then formated the dummy file
    using 'mke2fs'.

    But the operation was faced error. The error message is as follows.

    [[email protected] hdfs]# mke2fs -j -F ./file_dumy
    mke2fs 1.40.2 (12-Jul-2007)
    ./file_dumy: Input/output error while setting up superblock
    Also, I copyed an image file of xen to hdfs. But Xen couldn't the image
    files in hdfs.

    [email protected] hdfs]# fdisk -l fedora6_demo.img
    last_lba(): I don't know how to handle files with mode 81a4
    You must set cylinders.
    You can do this from the extra functions menu.

    Disk fedora6_demo.img: 0 MB, 0 bytes
    255 heads, 63 sectors/track, 0 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot Start End Blocks Id System
    fedora6_demo.img1 * 1 156 1253038+ 83 Linux

    Could you answer me anything about this problem.

    Thank you.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedMay 22, '09 at 8:55a
activeMay 22, '09 at 9:43p
posts4
users4
websitehadoop.apache.org...
irc#hadoop

4 users in discussion

Amr Awadallah: 1 post Aaron Kimball: 1 post 신승엽: 1 post Taeho Kang: 1 post

Content

People

Support

Translate

site design / logo © 2023 Grokbase