FAQ
hello
i have 16tb storage. 8x2tb sata raided.
i want to share it on network via nfs.
which file system is better for it?
thank you
???
Ashkan R

Search Discussions

  • Nux! at Aug 4, 2012 at 10:09 am

    On 04.08.2012 15:01, ashkab rahmani wrote:
    hello
    i have 16tb storage. 8x2tb sata raided.
    i want to share it on network via nfs.
    which file system is better for it?
    thank you
    ???
    Ashkan R
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
    No redundancy? That's a lot of data to lose. :-)

    As for your question, I'd use ext4. It has caught up a lot with XFS and
    it's THE file system supported by RHEL and Fedora.

    --
    Sent from the Delta quadrant using Borg technology!

    Nux!
    www.nux.ro
  • Ashkab rahmani at Aug 4, 2012 at 10:19 am
    thank you i have redundancy but i have simplified scenario.
    but i think ext4 is notbas fast as others. is it true?

    ???
    Ashkan R
    On Aug 4, 2012 6:39 PM, "Nux!" wrote:
    On 04.08.2012 15:01, ashkab rahmani wrote:
    hello
    i have 16tb storage. 8x2tb sata raided.
    i want to share it on network via nfs.
    which file system is better for it?
    thank you
    ???
    Ashkan R
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
    No redundancy? That's a lot of data to lose. :-)

    As for your question, I'd use ext4. It has caught up a lot with XFS and
    it's THE file system supported by RHEL and Fedora.

    --
    Sent from the Delta quadrant using Borg technology!

    Nux!
    www.nux.ro
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
  • Nux! at Aug 4, 2012 at 10:28 am

    On 04.08.2012 15:19, ashkab rahmani wrote:
    thank you i have redundancy but i have simplified scenario.
    but i think ext4 is notbas fast as others. is it true?

    ???
    Ashkan R
    On Aug 4, 2012 6:39 PM, "Nux!" wrote:
    On 04.08.2012 15:01, ashkab rahmani wrote:
    hello
    i have 16tb storage. 8x2tb sata raided.
    i want to share it on network via nfs.
    which file system is better for it?
    thank you
    ???
    Ashkan R
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
    No redundancy? That's a lot of data to lose. :-)

    As for your question, I'd use ext4. It has caught up a lot with XFS
    and
    it's THE file system supported by RHEL and Fedora.

    --
    Sent from the Delta quadrant using Borg technology!

    Nux!
    www.nux.ro
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
    Well, I think ext4 is pretty fast. Maybe XFS has a slight edge over it
    in some scenarios.
    ZFS on linux is still highly experimental and has received close to no
    testing.
    If you are in mood for experiments EL6.3 includes BTRFS as technology
    preview for 64bit machines. Give it a try and let us know how it goes.

    --
    Sent from the Delta quadrant using Borg technology!

    Nux!
    www.nux.ro
  • Ashkab rahmani at Aug 4, 2012 at 10:36 am
    thank you. very usefull
    i think i'll try btrfs or jfs,
    i'll send you btrfs result for you.
    On Sat, Aug 4, 2012 at 6:58 PM, Nux! wrote:
    On 04.08.2012 15:19, ashkab rahmani wrote:
    thank you i have redundancy but i have simplified scenario.
    but i think ext4 is notbas fast as others. is it true?

    ???
    Ashkan R
    On Aug 4, 2012 6:39 PM, "Nux!" wrote:
    On 04.08.2012 15:01, ashkab rahmani wrote:
    hello
    i have 16tb storage. 8x2tb sata raided.
    i want to share it on network via nfs.
    which file system is better for it?
    thank you
    ???
    Ashkan R
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
    No redundancy? That's a lot of data to lose. :-)

    As for your question, I'd use ext4. It has caught up a lot with XFS
    and
    it's THE file system supported by RHEL and Fedora.

    --
    Sent from the Delta quadrant using Borg technology!

    Nux!
    www.nux.ro
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
    Well, I think ext4 is pretty fast. Maybe XFS has a slight edge over it
    in some scenarios.
    ZFS on linux is still highly experimental and has received close to no
    testing.
    If you are in mood for experiments EL6.3 includes BTRFS as technology
    preview for 64bit machines. Give it a try and let us know how it goes.

    --
    Sent from the Delta quadrant using Borg technology!

    Nux!
    www.nux.ro
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
  • Nux! at Aug 4, 2012 at 10:46 am

    On 04.08.2012 15:36, ashkab rahmani wrote:
    thank you. very usefull
    i think i'll try btrfs or jfs,
    i'll send you btrfs result for you.
    Ilsistemista.net seems to have some good articles about filesystems.
    e.g.
    http://www.ilsistemista.net/index.php/linux-a-unix/33-btrfs-vs-ext3-vs-ext4-vs-xfs-performance-on-fedora-17.html
    Check them out.

    --
    Sent from the Delta quadrant using Borg technology!

    Nux!
    www.nux.ro
  • Johnny Hughes at Aug 4, 2012 at 11:21 am

    On 08/04/2012 09:36 AM, ashkab rahmani wrote:
    thank you. very usefull
    i think i'll try btrfs or jfs,
    i'll send you btrfs result for you.
    On Sat, Aug 4, 2012 at 6:58 PM, Nux! wrote:
    On 04.08.2012 15:19, ashkab rahmani wrote:
    thank you i have redundancy but i have simplified scenario.
    but i think ext4 is notbas fast as others. is it true?
    On 04.08.2012 15:01, ashkab rahmani wrote:
    hello
    i have 16tb storage. 8x2tb sata raided.
    i want to share it on network via nfs.
    which file system is better for it?
    thank you
    No redundancy? That's a lot of data to lose. :-)

    As for your question, I'd use ext4. It has caught up a lot with XFS
    and
    it's THE file system supported by RHEL and Fedora.

    Well, I think ext4 is pretty fast. Maybe XFS has a slight edge over it
    in some scenarios.
    ZFS on linux is still highly experimental and has received close to no
    testing.
    If you are in mood for experiments EL6.3 includes BTRFS as technology
    preview for 64bit machines. Give it a try and let us know how it goes.
    Personally, I would use ext4 ... faster is not always better.

    As Nux! initially said, ext4 is the OS that RHEL and Fedora support as
    their main file system. I would (and do) use that. The 6.3 kernel does
    support xfs and CentOS has the jfs tools in our extras directory, but I
    like tried and true over experimental.

    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: signature.asc
    Type: application/pgp-signature
    Size: 262 bytes
    Desc: OpenPGP digital signature
    Url : http://lists.centos.org/pipermail/centos/attachments/20120804/6ea8ebf2/attachment.bin
  • Keith Keller at Aug 4, 2012 at 11:05 pm

    On 2012-08-04, Johnny Hughes wrote:
    As Nux! initially said, ext4 is the OS that RHEL and Fedora support as
    their main file system. I would (and do) use that. The 6.3 kernel does
    support xfs and CentOS has the jfs tools in our extras directory, but I
    like tried and true over experimental.
    Isn't XFS on linux tried and true by now? It's always worked great for
    me.

    Does ext4 resolve the issue of slow fsck? Recently I had a ~500GB ext3
    filesystem that hadn't been checked in a while; it took over 20 minutes
    to fsck. Meanwhile, a few months ago I had a problematic ~10TB XFS
    filesystem, and it took about 1-2 hours to fsck (IIRC 1.5 hrs). This
    was also a reason I switched away from reiserfs (this was well before
    Hans Reiser's personal problems)--a reiserfsck of a relatively modest
    filesystem took much longer than even an ext3 fsck.

    If I get some time I will try it on some spare filesystems, but I'm
    curious what other people's experiences are.

    I've looked into ZFS on linux, but it still seems not quite ready for
    real production use. I'd love to test it on a less crucial server when
    I get the chance. Their FAQ claims RHEL 6.0 support:

    http://zfsonlinux.org/faq.html

    --keith


    --
    kkeller at wombat.san-francisco.ca.us
  • Johnny Hughes at Aug 5, 2012 at 10:59 am

    On 08/04/2012 10:05 PM, Keith Keller wrote:
    On 2012-08-04, Johnny Hughes wrote:
    As Nux! initially said, ext4 is the OS that RHEL and Fedora support as
    their main file system. I would (and do) use that. The 6.3 kernel does
    support xfs and CentOS has the jfs tools in our extras directory, but I
    like tried and true over experimental.
    Isn't XFS on linux tried and true by now? It's always worked great for
    me.
    I suppose that depends on your definition. It has only JUST become
    supported on RHEL ...
    Does ext4 resolve the issue of slow fsck? Recently I had a ~500GB ext3
    filesystem that hadn't been checked in a while; it took over 20 minutes
    to fsck. Meanwhile, a few months ago I had a problematic ~10TB XFS
    filesystem, and it took about 1-2 hours to fsck (IIRC 1.5 hrs). This
    was also a reason I switched away from reiserfs (this was well before
    Hans Reiser's personal problems)--a reiserfsck of a relatively modest
    filesystem took much longer than even an ext3 fsck.

    If I get some time I will try it on some spare filesystems, but I'm
    curious what other people's experiences are.

    I've looked into ZFS on linux, but it still seems not quite ready for
    real production use. I'd love to test it on a less crucial server when
    I get the chance. Their FAQ claims RHEL 6.0 support:

    http://zfsonlinux.org/faq.html
    My experience is that I normally do not have any issues with ext3 on EL5
    or ext4 on EL6 ... problems being that I can not use the system because
    the IO is too slow, for example.

    Other people might want to pick and prod and get the top 5 or 6%
    performance they can out of a machine ... I would rather use what is
    supported unless there is a reason that I can not do it in production.

    That is why it's your machine ... you get to do whatever you want ...
    and keep all the pieces :D

    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: signature.asc
    Type: application/pgp-signature
    Size: 262 bytes
    Desc: OpenPGP digital signature
    Url : http://lists.centos.org/pipermail/centos/attachments/20120805/a8a4f8e6/attachment.bin
  • Eero Volotinen at Aug 5, 2012 at 2:20 pm

    2012/8/5 Johnny Hughes <johnny@centos.org>:
    On 08/04/2012 10:05 PM, Keith Keller wrote:
    On 2012-08-04, Johnny Hughes wrote:
    As Nux! initially said, ext4 is the OS that RHEL and Fedora support as
    their main file system. I would (and do) use that. The 6.3 kernel does
    support xfs and CentOS has the jfs tools in our extras directory, but I
    like tried and true over experimental.
    Isn't XFS on linux tried and true by now? It's always worked great for
    me.
    I suppose that depends on your definition. It has only JUST become
    supported on RHEL ...
    Just? http://www.redhat.com/products/enterprise-linux-add-ons/file-systems/

    At least it is supported on RHEL 5 also with extra price tag..



    --
    Eero
  • Karanbir Singh at Aug 5, 2012 at 2:59 pm

    On 08/05/2012 04:05 AM, Keith Keller wrote:
    I've looked into ZFS on linux, but it still seems not quite ready for
    real production use. I'd love to test it on a less crucial server when
    I get the chance. Their FAQ claims RHEL 6.0 support:
    Excellent! Do share your test / play experience.


    --
    Karanbir Singh
    +44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh
    ICQ: 2522219 | Yahoo IM: z00dax | Gtalk: z00dax
    GnuPG Key : http://www.karan.org/publickey.asc
  • Warren Young at Aug 6, 2012 at 6:44 pm

    On 8/4/2012 9:21 AM, Johnny Hughes wrote:
    ext4 is the OS that RHEL and Fedora support as
    their main file system. I would (and do) use that. The 6.3 kernel does
    support xfs and CentOS has the jfs tools in our extras directory, but I
    like tried and true over experimental.
    xfs still has at least one big advantage over ext4 on EL6, AFAIK:
    supporting filesystems over 16 TB.

    I am aware that ext4 is supposed to support 1 EiB filesystems[1], but
    due to a bug in e2fsprogs 1.41 (what EL6 ships) there is an artificial
    16 TB limit[2].

    I know we all like to pride ourselves on "stable" package versions, but
    this bug was fixed in the very next feature release of e2fsprogs, 1.42.
    Johnny, do you have any insight into why upstream isn't making an
    exception here, as they have for, say, Firefox? Surely they aren't
    going to hold it for EL7 and tout it as a "feature"?

    Perhaps I am being unfair. Did they backport the bug fix only, and my
    failure to mkfs.ext4 a 22 TB partition was due to some other problem?

    If someone wants me to try something, it'll have to wait at least a few
    weeks. That's the earliest I'm likely to have a > 16 TB RAID sitting
    around ready to be nuke-and-paved at a whim.

    [1] https://en.wikipedia.org/wiki/Ext4
    [2] http://e2fsprogs.sourceforge.net/e2fsprogs-release.html#1.42
  • Adam Tauno Williams at Aug 8, 2012 at 9:17 am

    On Sat, 2012-08-04 at 10:21 -0500, Johnny Hughes wrote:
    On 08/04/2012 09:36 AM, ashkab rahmani wrote:
    thank you. very usefull
    i think i'll try btrfs or jfs,
    i'll send you btrfs result for you.
    On Sat, Aug 4, 2012 at 6:58 PM, Nux! wrote:
    On 04.08.2012 15:19, ashkab rahmani wrote:
    thank you i have redundancy but i have simplified scenario.
    but i think ext4 is notbas fast as others. is it true?
    On 04.08.2012 15:01, ashkab rahmani wrote:
    hello
    i have 16tb storage. 8x2tb sata raided.
    i want to share it on network via nfs.
    which file system is better for it?
    thank you
    No redundancy? That's a lot of data to lose. :-)

    As for your question, I'd use ext4. It has caught up a lot with XFS
    and
    it's THE file system supported by RHEL and Fedora.

    Well, I think ext4 is pretty fast. Maybe XFS has a slight edge over it
    in some scenarios.
    ZFS on linux is still highly experimental and has received close to no
    testing.
    If you are in mood for experiments EL6.3 includes BTRFS as technology
    preview for 64bit machines. Give it a try and let us know how it goes.
    Personally, I would use ext4 ... faster is not always better.
    +1 ext4 is 'plenty good', bullet proof, and robustly supported.

    What ext4 suffers most from is hangover impressions of its quality that
    have followed it from early ext3 [even later versions of ext3 were
    considerably better than early ext3; especially with the introduction
    of dir_index that solved a lot of big-folders-are-very-slow problems].

    ext4 uses extents, just like XFS. It can pre-allocate just like XFS.
    It does delated allocation, like XFS.

    If you want really good performance than putting your journal external
    to the filesystem, preferably on really fast storage, will probably help
    more than anything else. Certainly more than type-of-filesystem.
    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: not available
    Type: application/pgp-signature
    Size: 198 bytes
    Desc: This is a digitally signed message part
    Url : http://lists.centos.org/pipermail/centos/attachments/20120808/c27bf172/attachment.bin
  • Morten Stevens at Aug 4, 2012 at 11:26 am

    On 04.08.2012 16:36, ashkab rahmani wrote:
    thank you. very usefull
    i think i'll try btrfs or jfs,
    i'll send you btrfs result for you.
    Please note: The Btrfs code of CentOS 6.3 is based on kernel 2.6.32.
    This is very experimental.

    If you want to try Btrfs, then use kernel 3.2 or higher. (there are
    thousands of bug fixes and improvements since 2.6.32)

    Anyway, I still recommend ext4 or xfs.

    Best regards,

    Morten
  • Joerg Schilling at Aug 4, 2012 at 12:06 pm

    Nux! wrote:

    ZFS on linux is still highly experimental and has received close to no
    testing.
    If you are in mood for experiments EL6.3 includes BTRFS as technology
    preview for 64bit machines. Give it a try and let us know how it goes.
    Using BTRFS now is like using ZFS in 2005.

    ZFS is adult now, BTRFS is not.

    Nux! wrote:
    ZFS on linux is still highly experimental and has received close to no
    testing.
    If you are in mood for experiments EL6.3 includes BTRFS as technology
    preview for 64bit machines. Give it a try and let us know how it goes.
    Using BTRFS now is like using ZFS in 2005.

    ZFS is adult now, BTRFS is not.

    J?rg

    --
    EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
    js at cs.tu-berlin.de (uni)
    joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
    URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
  • Joerg Schilling at Aug 4, 2012 at 12:38 pm

    Reindl Harald wrote:

    face the truth!

    there is no ZFS for linux
    there will never be

    that you do not like GPL, Linux etc. at all will
    not change anything, not now and not in the future
    What do you expect from spreading lies against me?

    You are off topic, so please stop this nonsense.

    J?rg

    --
    EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
    js at cs.tu-berlin.de (uni)
    joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
    URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
  • Karanbir Singh at Aug 4, 2012 at 2:32 pm

    On 08/04/2012 05:06 PM, Joerg Schilling wrote:
    Using BTRFS now is like using ZFS in 2005.
    ZFS is adult now, BTRFS is not
    Can you quantify this in an impartial format as relevant to CentOS ? At
    the moment your statement is just a rant, and having come across your
    work in the past, I know you can do better than this.

    Regards,

    --
    Karanbir Singh
    +44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh
    ICQ: 2522219 | Yahoo IM: z00dax | Gtalk: z00dax
    GnuPG Key : http://www.karan.org/publickey.asc
  • Joerg Schilling at Aug 4, 2012 at 3:32 pm

    Karanbir Singh wrote:
    On 08/04/2012 05:06 PM, Joerg Schilling wrote:
    Using BTRFS now is like using ZFS in 2005.
    ZFS is adult now, BTRFS is not
    Can you quantify this in an impartial format as relevant to CentOS ? At
    the moment your statement is just a rant, and having come across your
    work in the past, I know you can do better than this.
    I would not call it a rant but a food for thought.

    ZFS was distributed to the public after it turned 4.
    ZFS is now in public use since more than 7 years.

    What is the age of BTRFS?

    The experience with various filesystems tells that it takes 8-10 years to make
    a new filesystem mature.

    Also the OP did not ask for CentOS, but for a filesystem comparison.

    So comparing filesystems seems to be the question. For ZFS, I know that it
    took until three years ago to get rid of nasty bugs. At that time, ZFS was 8.

    So be careful with BTRFS until it was in wide use for at least 4 years.

    ZFS is the best I know for filesystems >= 2 TB and in case you need flexible
    snapshots. ZFS has just one single problem, it is slow in case you ask it to
    verify a stable FS state, UFS is much faster here, but this ZFS "problem" is
    true for all filesystems on Linux because of the implementation of the Linux
    buffer cache.

    And BTW: ZFS is based on the COW ideas I made in 1988 and the NetApp patents
    are also just based on my master thesis without giving me credit ;-)


    There are few fs use cases where COW is not the best.

    J?rg

    --
    EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
    js at cs.tu-berlin.de (uni)
    joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
    URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
  • Nux! at Aug 4, 2012 at 3:39 pm

    On 04.08.2012 20:32, Joerg.Schilling at fokus.fraunhofer.de wrote:
    Karanbir Singh wrote:
    On 08/04/2012 05:06 PM, Joerg Schilling wrote:
    Using BTRFS now is like using ZFS in 2005.
    ZFS is adult now, BTRFS is not
    ZFS is the best I know for filesystems >= 2 TB and in case you need
    flexible
    snapshots. ZFS has just one single problem, it is slow in case you
    ask it to
    verify a stable FS state, UFS is much faster here, but this ZFS
    "problem" is
    true for all filesystems on Linux because of the implementation of
    the Linux
    buffer cache.

    And BTW: ZFS is based on the COW ideas I made in 1988 and the NetApp
    patents
    are also just based on my master thesis without giving me credit ;-)
    Jorg,

    Given your expertise then, can you say how mature/stable/usable is ZFS
    on Linux, specifically CentOS?
    That's what everybody is probably most interested in.

    --
    Sent from the Delta quadrant using Borg technology!

    Nux!
    www.nux.ro
  • Joerg Schilling at Aug 5, 2012 at 5:50 am

    Nux! wrote:
    On 04.08.2012 20:32, Joerg.Schilling at fokus.fraunhofer.de wrote:
    Karanbir Singh wrote:
    On 08/04/2012 05:06 PM, Joerg Schilling wrote:
    Using BTRFS now is like using ZFS in 2005.
    ZFS is adult now, BTRFS is not
    ZFS is the best I know for filesystems >= 2 TB and in case you need
    flexible
    snapshots. ZFS has just one single problem, it is slow in case you
    ask it to
    verify a stable FS state, UFS is much faster here, but this ZFS
    "problem" is
    true for all filesystems on Linux because of the implementation of
    the Linux
    buffer cache.
    Given your expertise then, can you say how mature/stable/usable is ZFS
    on Linux, specifically CentOS?
    That's what everybody is probably most interested in.
    ZFS is stable on FreeBSD since aprox. 3 years.

    ZFS itself is also stable.

    I cannot speak for the stability of Linux, but I've read that there is a group
    that works on a ZFS integration. The problem in this area is that Linux comes
    with a very limited VFS interface and porters would either need to reduce ZFS
    functionality or ignore the VFS interface from Linux.

    J?rg

    --
    EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
    js at cs.tu-berlin.de (uni)
    joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
    URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
  • Ashkab rahmani at Aug 4, 2012 at 3:48 pm
    thank you very much. what do you think abou jfs??
    is it comparable with others??
    ???
    Ashkan R
    On Aug 5, 2012 12:02 AM, "Joerg Schilling" wrote:

    Karanbir Singh wrote:
    On 08/04/2012 05:06 PM, Joerg Schilling wrote:
    Using BTRFS now is like using ZFS in 2005.
    ZFS is adult now, BTRFS is not
    Can you quantify this in an impartial format as relevant to CentOS ? At
    the moment your statement is just a rant, and having come across your
    work in the past, I know you can do better than this.
    I would not call it a rant but a food for thought.

    ZFS was distributed to the public after it turned 4.
    ZFS is now in public use since more than 7 years.

    What is the age of BTRFS?

    The experience with various filesystems tells that it takes 8-10 years to
    make
    a new filesystem mature.

    Also the OP did not ask for CentOS, but for a filesystem comparison.

    So comparing filesystems seems to be the question. For ZFS, I know that it
    took until three years ago to get rid of nasty bugs. At that time, ZFS was
    8.

    So be careful with BTRFS until it was in wide use for at least 4 years.

    ZFS is the best I know for filesystems >= 2 TB and in case you need
    flexible
    snapshots. ZFS has just one single problem, it is slow in case you ask it
    to
    verify a stable FS state, UFS is much faster here, but this ZFS "problem"
    is
    true for all filesystems on Linux because of the implementation of the
    Linux
    buffer cache.

    And BTW: ZFS is based on the COW ideas I made in 1988 and the NetApp
    patents
    are also just based on my master thesis without giving me credit ;-)


    There are few fs use cases where COW is not the best.

    J?rg

    --
    EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353
    Berlin
    js at cs.tu-berlin.de (uni)
    joerg.schilling at fokus.fraunhofer.de (work) Blog:
    http://schily.blogspot.com/
    URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
  • John R Pierce at Aug 4, 2012 at 4:14 pm

    On 08/04/12 12:48 PM, ashkab rahmani wrote:
    thank you very much. what do you think abou jfs??
    is it comparable with others??
    it works very well on IBM AIX, but I see very little support or usage
    from the Linux community.



    --
    john r pierce N 37, W 122
    santa cruz ca mid-left coast
  • Fernando Cassia at Aug 4, 2012 at 11:26 pm

    On Sat, Aug 4, 2012 at 4:48 PM, ashkab rahmani wrote:
    thank you very much. what do you think abou jfs??
    is it comparable with others??
    I was very pro-JFS... until I lost 10gig of very important data, and
    back then (2002) there was no way to recover a JFS volume (the data
    was in RAID, but some corruption ocurred and I lost the whole drive, I
    mean, I ended up with a blank root).

    Back in 2004 I asked one of the IBMers at the JFS team about it and he
    had this to say:

    -----
    "IBM will continue to invest in jfs as long as we feel that our customers get
    value from it."

    Q: Will JFS be enhanced eventually with features from ReiserFS 4? (can it
    be done without a complete rewrite?).

    "Possibly some. Samba has been asking for streams support for a while,
    and if reiser4 leads the way in an implementation that does not break
    unix file semantics, jfs (and possibly other file systems) may follow."

    -----

    Dunno if IBM did much to JFS after that... haven?t been following
    their work wrt JFS...

    FC
  • John R Pierce at Aug 4, 2012 at 11:32 pm

    On 08/04/12 8:26 PM, Fernando Cassia wrote:
    Dunno if IBM did much to JFS after that... haven?t been following
    their work wrt JFS...
    JFS is the primary file system for AIX on their big Power servers, and
    on those, it performs very very well. the utilities are are fully
    integrated so growing a file system is a one step process that takes
    care of both the LVM and JFS online in a single command.

    # chfs -size=+10G /home

    hard to be much simpler than that!



    --
    john r pierce N 37, W 122
    santa cruz ca mid-left coast
  • Joerg Schilling at Aug 5, 2012 at 6:40 am

    John R Pierce wrote:

    integrated so growing a file system is a one step process that takes
    care of both the LVM and JFS online in a single command.

    # chfs -size=+10G /home

    hard to be much simpler than that!
    ZFS is simpler than that ;-)

    If you enabled the zpool autoexpand feature, you just need to start replacing
    disks by bigger ones. Once you are ready with replacing all disks from a RAID
    system, the filesytem shows the new size.

    BTW: where do you expect the additional 10G to come from in your example?



    J?rg

    --
    EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
    js at cs.tu-berlin.de (uni)
    joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
    URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
  • John R Pierce at Aug 5, 2012 at 2:14 pm

    On 08/05/12 3:40 AM, Joerg Schilling wrote:
    ZFS is simpler than that ;-)
    well aware, I run ZFS on Solaris.

    BTW: where do you expect the additional 10G to come from in your example?
    from the LVM pool containing /home ... Linux LVM also came from IBM, and was based on the LVM of AIX



    --
    john r pierce N 37, W 122
    santa cruz ca mid-left coast
  • Karanbir Singh at Aug 5, 2012 at 2:33 pm

    On 08/05/2012 07:14 PM, John R Pierce wrote:
    from the LVM pool containing /home ... Linux LVM also came from IBM, and was based on the LVM of AIX

    AIX had a LogicalVolume Manager, sure - but I dont think thats where the
    linux LVM came from - the Sistina guys had a fairly independent
    implementation. And the Linux LVM looks a lot more like the HP variant
    than the IBM one.

    - KB
    --
    Karanbir Singh
    +44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh
    GnuPG Key : http://www.karan.org/publickey.asc
  • Fernando Cassia at Aug 5, 2012 at 2:40 pm

    On Sun, Aug 5, 2012 at 3:33 PM, Karanbir Singh wrote:
    AIX had a LogicalVolume Manager, sure - but I dont think thats where the
    linux LVM came from - the Sistina guys had a fairly independent
    implementation. And the Linux LVM looks a lot more like the HP variant
    than the IBM one.
    And all LVM implementations become obsolete with Btrfs ;-P

    http://www.youtube.com/watch?v=hxWuaozpe2I

    FC
  • Karanbir Singh at Aug 5, 2012 at 2:53 pm

    On 08/05/2012 07:40 PM, Fernando Cassia wrote:
    AIX had a LogicalVolume Manager, sure - but I dont think thats where the
    linux LVM came from - the Sistina guys had a fairly independent
    implementation. And the Linux LVM looks a lot more like the HP variant
    than the IBM one.
    And all LVM implementations become obsolete with Btrfs ;-P

    http://www.youtube.com/watch?v=hxWuaozpe2I
    you seem confused about what a filesystem and volume management is.


    --
    Karanbir Singh
    +44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh
    ICQ: 2522219 | Yahoo IM: z00dax | Gtalk: z00dax
    GnuPG Key : http://www.karan.org/publickey.asc
  • Fernando Cassia at Aug 5, 2012 at 3:04 pm

    On Sun, Aug 5, 2012 at 3:53 PM, Karanbir Singh wrote:
    you seem confused about what a filesystem and volume management is.
    http://www.funtoo.org/wiki/BTRFS_Fun

    ----
    Btrfs, often compared to ZFS, is offering some interesting features like:
    (snip)
    Built-in storage pool capabilities (no need for LVM)
    ----

    FC
  • SilverTip257 at Aug 5, 2012 at 8:25 pm
    XFS: Recent and Future Adventures in Filesystem Scalability - Dave Chinner
    Uploaded by linuxconfau2012 on Jan 19, 2012
    http://www.youtube.com/watch?NR=1&feature=endscreen&v=FegjLbCnoBw

    ---~~.~~---
    Mike
    // SilverTip257 //

    On Sun, Aug 5, 2012 at 2:40 PM, Fernando Cassia wrote:
    On Sun, Aug 5, 2012 at 3:33 PM, Karanbir Singh wrote:
    AIX had a LogicalVolume Manager, sure - but I dont think thats where the
    linux LVM came from - the Sistina guys had a fairly independent
    implementation. And the Linux LVM looks a lot more like the HP variant
    than the IBM one.
    And all LVM implementations become obsolete with Btrfs ;-P

    http://www.youtube.com/watch?v=hxWuaozpe2I

    FC
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
  • Fernando Cassia at Aug 5, 2012 at 10:14 pm

    On Sun, Aug 5, 2012 at 9:25 PM, SilverTip257 wrote:
    Recent and Future Adventures in Filesystem Scalability - Dave Chinner
    Thanks for that vid!

    FC
  • John R Pierce at Aug 5, 2012 at 2:43 pm

    On 08/05/12 11:33 AM, Karanbir Singh wrote:
    And the Linux LVM looks a lot more like the HP variant
    than the IBM one.
    ah, you're right, googing and wikipedia says, the linux implementation
    was based on HPUX, I was mistaken thinking IBM had provided their LVM code.



    --
    john r pierce N 37, W 122
    santa cruz ca mid-left coast
  • Fernando Cassia at Aug 5, 2012 at 1:46 pm

    On Sun, Aug 5, 2012 at 12:32 AM, John R Pierce wrote:
    JFS is the primary file system for AIX on their big Power servers, and
    on those, it performs very very well. the utilities are are fully
    integrated so growing a file system is a one step process that takes
    care of both the LVM and JFS online in a single command.
    Yes, however my data loss experience was with IBM?s OS/2 port of JFS.
    Probably related to one of these
    http://www.os2voice.org/warpcast/1999-08/37CC5F9D.htm

    Needless to say I learned the hard way that filesystems can be buggy. ;)
    FC
    FC
  • Karanbir Singh at Aug 5, 2012 at 2:38 pm

    On 08/05/2012 06:46 PM, Fernando Cassia wrote:
    Yes, however my data loss experience was with IBM?s OS/2 port of JFS.
    Probably related to one of these
    http://www.os2voice.org/warpcast/1999-08/37CC5F9D.htm
    I think its safe to assume that OS/2 experience from 1998 is pretty much
    irrelevant to the conversation here, and JFS on linux. Just looking at
    the kernel tree its easy to spot plenty of changes in that part of the
    codebase.

    There is jfs support available for CentOS - feel free to install it and
    quantify issues around it.


    --
    Karanbir Singh
    +44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh
    ICQ: 2522219 | Yahoo IM: z00dax | Gtalk: z00dax
    GnuPG Key : http://www.karan.org/publickey.asc
  • Fernando Cassia at Aug 5, 2012 at 2:42 pm

    On Sun, Aug 5, 2012 at 3:38 PM, Karanbir Singh wrote:
    I think its safe to assume that OS/2 experience from 1998 is pretty much
    irrelevant to the conversation here, and JFS on linux
    My data loss was in 2002. :-p

    You are putting words in my mouth. Re-read what I posted before you
    jump to conclusions. I also do not like your patronising tone.

    FC
  • Joerg Schilling at Aug 5, 2012 at 6:35 am

    Fernando Cassia wrote:
    "Possibly some. Samba has been asking for streams support for a while,
    and if reiser4 leads the way in an implementation that does not break
    unix file semantics, jfs (and possibly other file systems) may follow."
    Microsoft tried to advertize their "stream" concept to POSIX in summer 2001.

    They failed because they used a userinterface that is in conflict with POSIX
    rules (e.g. by forbidding ':' to be a normal character in filenames or by
    trying to introduce a ew special directory "...").

    In August 2001, Sun came up with the extended attribute file concept that is a
    superset of the Microsoft stream concept and in addition compatile to POSIX.

    In August 2001, the implementation was only usable on UFS outside from Sun, but
    it was implemented in ZFS from the beginning.

    Given the fact that the extended attribute file concept is part of the NFSv4
    standard, Linux should implement it in case it offers a full blown NFSv4. I am
    not sure whether this applies to Linux, as my last information say that support
    for NFSv4 ACLs is also missing on Linux. Note that NFSv4 ACLs are bitwise
    identical to NTFS ACLs and to ZFS ACLs.

    But if you like to offer SMB exports, you should better use Solaris as Solaris
    comes with an in-kernel SMB server that supports all features from SMB. This
    includes support for atomar ACL create support that can only be supported with
    an enhanced VFS interface.

    J?rg

    --
    EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
    js at cs.tu-berlin.de (uni)
    joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
    URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
  • Fernando Cassia at Aug 4, 2012 at 11:16 pm

    On Sat, Aug 4, 2012 at 4:32 PM, Joerg Schilling wrote:
    What is the age of BTRFS?
    BTRFS presentation, mid-2007
    https://oss.oracle.com/projects/btrfs/dist/documentation/btrfs-ukuug.pdf

    That makes it 6 years in development. Next...

    FC

    --
    During times of Universal Deceit, telling the truth becomes a revolutionary act
    Durante ?pocas de Enga?o Universal, decir la verdad se convierte en un
    Acto Revolucionario
    - George Orwell
  • Joerg Schilling at Aug 5, 2012 at 6:14 am

    Fernando Cassia wrote:

    On Sat, Aug 4, 2012 at 4:32 PM, Joerg Schilling
    wrote:
    What is the age of BTRFS?
    BTRFS presentation, mid-2007
    https://oss.oracle.com/projects/btrfs/dist/documentation/btrfs-ukuug.pdf

    That makes it 6 years in development. Next...
    So BTRFS is 6 years younger than ZFs.

    Comparing todays's bugreports for BTRFS with ZFS bugreports from 2006 let me
    asume that you should not consider to use BTRFS during the next 2-3 years in
    real production.



    J?rg

    --
    EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
    js at cs.tu-berlin.de (uni)
    joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
    URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
  • Fernando Cassia at Aug 4, 2012 at 11:21 pm

    On Sat, Aug 4, 2012 at 4:32 PM, Joerg Schilling wrote:

    So be careful with BTRFS until it was in wide use for at least 4 years.
    FUD alert...

    https://events.linuxfoundation.org/events/linuxcon-japan/bo
    ---
    LinuxCon Japan 2012 | Presentations
    On The Way to a Healthy Btrfs Towards Enterprise

    Btrfs has been on full development for about 5 years and it does make
    lots of progress on both feature and performance, but why does
    everybody keep tagging it with ""experimental""? And why do people
    still think of it as a vulnerable one for production use? As a goal of
    production use, we have been strengthening several features, making
    improvements on performance and keeping fixing bugs to make btrfs
    stable, for instance, ""snapshot aware defrag"", ""extent buffer
    cache"", ""rbtree lock contention"", etc. This talk will cover the
    above and will also show problems we are facing with, solutions we are
    seeking for and a blueprint we are planning to lay out. For this
    session, I'll focus on its features and performance, so for the target
    audience, it'd be better to have a basic knowledge base of filesystem.


    Liu Bo, Fujitsu

    Liu Bo has been working on linux kernel development since late 2010 as
    a Fujitsu engineer. He has been working on filesystem field and he's
    now focusing on btrfs development.
    ----

    FC

    --
    During times of Universal Deceit, telling the truth becomes a revolutionary act
    Durante ?pocas de Enga?o Universal, decir la verdad se convierte en un
    Acto Revolucionario
    - George Orwell
  • Karanbir Singh at Aug 5, 2012 at 2:51 pm

    On 08/04/2012 08:32 PM, Joerg Schilling wrote:
    I would not call it a rant but a food for thought. agreed!
    ZFS was distributed to the public after it turned 4.
    ZFS is now in public use since more than 7 years.
    but ZFS has not had a stable release in Linux as yet, making it still be
    negative in years. The codebase is likely to take a lot longer to get
    into a stable status than btrfs.
    What is the age of BTRFS?
    My personal experience with btrfs is from the 2.8.2x tree, at which
    point btrfs struggled to notify on disk-full situation. Making it quite
    academic as to what the maturity state of the system was! Admittedly,
    things have moved on and at this time btrfs has an exponentially higher
    contributor and adopter base on Linux than Zfs does.

    btw, I dont totally agree that btrfs and zfs are feature identical, so
    there will always been scope for one over the other in terms of how it
    fits the problem domain a user might have.
    Also the OP did not ask for CentOS, but for a filesystem comparison.
    This is the CentOS Linux list :) I think its safe to assume answers here
    should be directed towards that.
    So be careful with BTRFS until it was in wide use for at least 4 years.
    So, I'm all for mature and tested systems - always. But given the
    problem domain, I think its wrong to generalise to that level. Lots of
    people will use technology on the cutting edge, and lots more people
    will adopt for feature matching with app domains. I, for one, am
    grateful to these people for picking up the stuff in its early days and
    working to find and then even fix issues as they come up.
    ZFS is the best I know for filesystems >= 2 TB and in case you need flexible
    snapshots. ZFS has just one single problem, it is slow in case you ask it to
    verify a stable FS state, UFS is much faster here, but this ZFS "problem" is
    true for all filesystems on Linux because of the implementation of the Linux
    buffer cache.
    the other problem with zfs is its large RAM requirements - and extremely
    poor 32bit support.
    There are few fs use cases where COW is not the best.
    agreed.

    --
    Karanbir Singh
    +44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh
    ICQ: 2522219 | Yahoo IM: z00dax | Gtalk: z00dax
    GnuPG Key : http://www.karan.org/publickey.asc
  • Joerg Schilling at Aug 6, 2012 at 6:00 am

    Karanbir Singh wrote:
    On 08/04/2012 08:32 PM, Joerg Schilling wrote:
    I would not call it a rant but a food for thought. agreed!
    ZFS was distributed to the public after it turned 4.
    ZFS is now in public use since more than 7 years.
    but ZFS has not had a stable release in Linux as yet, making it still be
    negative in years. The codebase is likely to take a lot longer to get
    into a stable status than btrfs.
    The ZFS code base is stable, the problem is the VFS interface in Linux and that
    applies to all filesystems....
    So be careful with BTRFS until it was in wide use for at least 4 years.
    So, I'm all for mature and tested systems - always. But given the
    problem domain, I think its wrong to generalise to that level. Lots of
    people will use technology on the cutting edge, and lots more people
    will adopt for feature matching with app domains. I, for one, am
    grateful to these people for picking up the stuff in its early days and
    working to find and then even fix issues as they come up.
    The ZFS developers have been forced by Sun to put their home directoriers on
    ZFS in 2001. Does this apply to BTRFS too?

    ZFS is the best I know for filesystems >= 2 TB and in case you need flexible
    snapshots. ZFS has just one single problem, it is slow in case you ask it to
    verify a stable FS state, UFS is much faster here, but this ZFS "problem" is
    true for all filesystems on Linux because of the implementation of the Linux
    buffer cache.
    the other problem with zfs is its large RAM requirements - and extremely
    poor 32bit support.
    This is not a problem as nobody encourages you to put ZFS into toys or phones.
    With the typical amout of RAM in today's machines, ZFS is happy.

    J?rg

    --
    EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
    js at cs.tu-berlin.de (uni)
    joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
    URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
  • Pasi Kärkkäinen at Aug 7, 2012 at 5:57 pm

    On Mon, Aug 06, 2012 at 12:00:22PM +0200, Joerg Schilling wrote:
    Karanbir Singh wrote:
    On 08/04/2012 08:32 PM, Joerg Schilling wrote:
    I would not call it a rant but a food for thought. agreed!
    ZFS was distributed to the public after it turned 4.
    ZFS is now in public use since more than 7 years.
    but ZFS has not had a stable release in Linux as yet, making it still be
    negative in years. The codebase is likely to take a lot longer to get
    into a stable status than btrfs.
    The ZFS code base is stable, the problem is the VFS interface in Linux and that
    applies to all filesystems....
    Hello,

    Care to explain what's the problem in Linux VFS layer ?

    -- Pasi
  • Joerg Schilling at Aug 8, 2012 at 6:11 am

    Pasi K?rkk?inen wrote:

    The ZFS code base is stable, the problem is the VFS interface in Linux and that
    applies to all filesystems....
    Hello,

    Care to explain what's the problem in Linux VFS layer ?
    The VFS layer was introduced in 1980 by Bill Joy when he started the UFS
    project (his master thesis).

    The VFS layer was enhanced around 1984/1985 to support NFS.

    The VFS layer was enhanced again in 1987 to support mmap() and the new momory
    model from SunOS-4.0 that has been copied by all major OS.

    In 1993, VFS was enhanced to support ACLs.

    In 2007, VFS was enhanced to support CIFS (e.g. atomic create with ACL, switch
    into case insensitive mode, ...).

    VFS has a 30 year history and offers a clean abstract design. It is even
    independent from typical changes in the proc or user structures. It uses vnodes
    instead of inodes to grant a clean abstraction level.


    Linux VFS looks not very abstract, it is directly bound to user/proc structures
    and to Linux I/O. There is no support for NFS (get fid from vnode and get vnode
    from fid). There is no support for ACLs (let the filesystem decide whether
    access should be granted).

    The deviating I/O model in Linux will cause a lot of work for adaptation....

    What about extended attribute files (the superset of Win-DOS streams)?

    J?rg

    --
    EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
    js at cs.tu-berlin.de (uni)
    joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
    URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
  • John R Pierce at Aug 4, 2012 at 4:12 pm

    On 08/04/12 7:01 AM, ashkab rahmani wrote:
    hello
    i have 16tb storage. 8x2tb sata raided.
    i want to share it on network via nfs.
    which file system is better for it?

    we are using XFS with CentOS 6.latest on 80TB file systems, works quite
    well. handles a mix of many tiny files and very large files without
    any special tuning.

    Theres one big issue with NFS that requires a workaround... XFS requires
    64 bit inodes on a large file system ('inode64'), and by default, NFS
    wants to use the inode as the unique ID for the export, this doesn't
    work as that unique ID has to be 32 bits, so you have to manually
    specify a unique identifier for each share from a given server. I
    can't remember offhand what the specific option is, but you can specify
    1, 2, 3, 4 for the share identifiers, or any other unique integer. if
    you only export the root of a file system, tis is not a problem. this
    problem is squarely an NFS implementation problem, that code should have
    been fixed eons ago.


    --
    john r pierce N 37, W 122
    santa cruz ca mid-left coast
  • SilverTip257 at Aug 4, 2012 at 9:47 pm
    One disadvantage I've seen with XFS is that you cannot shrink [0] the
    file system.
    For a box dedicated to network storage this shouldn't be a problem.
    But in my instance I made /var a bit too large and needed to reclaim
    space for /.

    [0] http://xfs.org/index.php/Shrinking_Support

    ---~~.~~---
    Mike
    // SilverTip257 //

    On Sat, Aug 4, 2012 at 4:12 PM, John R Pierce wrote:
    On 08/04/12 7:01 AM, ashkab rahmani wrote:
    hello
    i have 16tb storage. 8x2tb sata raided.
    i want to share it on network via nfs.
    which file system is better for it?

    we are using XFS with CentOS 6.latest on 80TB file systems, works quite
    well. handles a mix of many tiny files and very large files without
    any special tuning.

    Theres one big issue with NFS that requires a workaround... XFS requires
    64 bit inodes on a large file system ('inode64'), and by default, NFS
    wants to use the inode as the unique ID for the export, this doesn't
    work as that unique ID has to be 32 bits, so you have to manually
    specify a unique identifier for each share from a given server. I
    can't remember offhand what the specific option is, but you can specify
    1, 2, 3, 4 for the share identifiers, or any other unique integer. if
    you only export the root of a file system, tis is not a problem. this
    problem is squarely an NFS implementation problem, that code should have
    been fixed eons ago.


    --
    john r pierce N 37, W 122
    santa cruz ca mid-left coast

    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
  • Joerg Schilling at Aug 5, 2012 at 6:06 am

    John R Pierce wrote:
    Theres one big issue with NFS that requires a workaround... XFS requires
    64 bit inodes on a large file system ('inode64'), and by default, NFS
    wants to use the inode as the unique ID for the export, this doesn't
    work as that unique ID has to be 32 bits, so you have to manually
    specify a unique identifier for each share from a given server. I
    This is wrong.

    Your claim is aproximately correct for NFSv2 (1988) but wrong for other NFS
    versions.

    On NFSv2, te file handle is not able to handle more than 32 bit inode numbers.

    NFSv3 changed this (I believe this is from 1990).

    Unfortunately, NFSv2 and NFSv3 have been implemented in the same server/client
    code an thus it was not recommended to use large NFS file handles to retain
    NFSv2 compatibility.

    NFSv4 (since 2004) by default uses large NFS filehandles.

    Your problem may be caused by the quality of the NFS code in Linux, so it is
    worth to make a bug report.

    J?rg

    --
    EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
    js at cs.tu-berlin.de (uni)
    joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
    URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
  • John R Pierce at Aug 5, 2012 at 2:20 pm

    On 08/05/12 3:06 AM, Joerg Schilling wrote:
    Your claim is aproximately correct for NFSv2 (1988) but wrong for other NFS
    versions.
    The server was using NFS V3/V4 in CentOS 6.2 earlier this year, and
    various clients, including Solaris 10. The problems were reported from
    our overseas manufacturing operations so I only got them 3rd hand, and
    don't know all the specifics. In my lab I had only shared the root of
    the file system as thats the model I use, but apparently operations
    likes to have lots of different shares, MS Windows style. This was a
    'stop production' kind of error, so the most expedient fix was to
    manually specify the export ID.



    --
    john r pierce N 37, W 122
    santa cruz ca mid-left coast
  • Joerg Schilling at Aug 5, 2012 at 6:18 pm

    John R Pierce wrote:
    On 08/05/12 3:06 AM, Joerg Schilling wrote:
    Your claim is aproximately correct for NFSv2 (1988) but wrong for other NFS
    versions.
    The server was using NFS V3/V4 in CentOS 6.2 earlier this year, and
    various clients, including Solaris 10. The problems were reported from
    our overseas manufacturing operations so I only got them 3rd hand, and
    don't know all the specifics. In my lab I had only shared the root of
    the file system as thats the model I use, but apparently operations
    likes to have lots of different shares, MS Windows style. This was a
    'stop production' kind of error, so the most expedient fix was to
    manually specify the export ID.
    If you suffer from bugs in Linux filesystem implementations, you should make a
    bug report against the related code. Only a bug report ans a willing maintainer
    can help you.

    The problem you describe does not exist on Solaris nor on other systems with
    bug-free NFS and I know why I try to avoid Linux when NFS is important. It is a
    pity that after many years, there are still NFS problems in Linux.

    Again:

    - NFSv2 (from 1988) allows 32 Bytes for a NFS file handle

    - NFSv3 (from 1990) allows 64 Bytes for a NFS file handle

    - NFSv4 (from 2004) has no hard limit here

    With the 32 byte file handle, there are still 12 bytes (including a 2 byte
    length indicator) for the file id in the file handle.

    If your filesystem could use 44 and more bytes in the case you describe, there
    is no problem - except when the code is not OK.

    It is of course nice to still support SunOS-4.0 clients, but in case that the
    client supports NFSv3 or newer, why not use longer file id's?

    J?rg

    --
    EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
    js at cs.tu-berlin.de (uni)
    joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
    URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
  • John R Pierce at Aug 5, 2012 at 6:31 pm

    On 08/05/12 3:18 PM, Joerg Schilling wrote:
    John R Pierce wrote:
    On 08/05/12 3:06 AM, Joerg Schilling wrote:
    Your claim is aproximately correct for NFSv2 (1988) but wrong for other NFS
    versions.
    The server was using NFS V3/V4 in CentOS 6.2 earlier this year, and
    various clients, including Solaris 10. The problems were reported from
    our overseas manufacturing operations so I only got them 3rd hand, and
    don't know all the specifics. In my lab I had only shared the root of
    the file system as thats the model I use, but apparently operations
    likes to have lots of different shares, MS Windows style. This was a
    'stop production' kind of error, so the most expedient fix was to
    manually specify the export ID.
    If you suffer from bugs in Linux filesystem implementations, you should make a
    bug report against the related code. Only a bug report ans a willing maintainer
    can help you.

    The problem you describe does not exist on Solaris nor on other systems with
    bug-free NFS and I know why I try to avoid Linux when NFS is important. It is a
    pity that after many years, there are still NFS problems in Linux.

    Again:

    - NFSv2 (from 1988) allows 32 Bytes for a NFS file handle

    - NFSv3 (from 1990) allows 64 Bytes for a NFS file handle

    - NFSv4 (from 2004) has no hard limit here

    With the 32 byte file handle, there are still 12 bytes (including a 2 byte
    length indicator) for the file id in the file handle.

    If your filesystem could use 44 and more bytes in the case you describe, there
    is no problem - except when the code is not OK.

    It is of course nice to still support SunOS-4.0 clients, but in case that the
    client supports NFSv3 or newer, why not use longer file id's?
    we had both solaris 10 aka sunos 5.10 clients and EL5/6 clients. the
    error is "Stale NFS file handle"


    anyways, this refers to the fsid problem,
    http://xfs.org/index.php/XFS_FAQ#Q:_Why_doesn.27t_NFS-exporting_subdirectories_of_inode64-mounted_filesystem_work.3F

    I discussed this problem on this list back in march, and got little
    useful feedback

    I see related issues here.
    http://oss.sgi.com/archives/xfs/2009-11/msg00161.html
    so this problem has been known for awhile.

    we were unable to make the 'fsid=uuid' option work (or we didn't
    understand it), but using fsid=## for unique integers for each export
    works fine, so thats what we went with.

    are these fsid's the same as your 32 vs 64 bit file handles ? doesn't
    sound like it to me, unless I'm misunderstanding what you're referring
    to as a file handle.



    --
    john r pierce N 37, W 122
    santa cruz ca mid-left coast
  • Joerg Schilling at Aug 6, 2012 at 6:26 am

    John R Pierce wrote:

    Again:

    - NFSv2 (from 1988) allows 32 Bytes for a NFS file handle

    - NFSv3 (from 1990) allows 64 Bytes for a NFS file handle

    - NFSv4 (from 2004) has no hard limit here

    With the 32 byte file handle, there are still 12 bytes (including a 2 byte
    length indicator) for the file id in the file handle.

    If your filesystem could use 44 and more bytes in the case you describe, there
    is no problem - except when the code is not OK.

    It is of course nice to still support SunOS-4.0 clients, but in case that the
    client supports NFSv3 or newer, why not use longer file id's?
    we had both solaris 10 aka sunos 5.10 clients and EL5/6 clients. the
    error is "Stale NFS file handle"
    So the bug is in the NFS server code.


    A NFS file handle is made of a filesystem ID, a file ID for the export
    directory and a file ID for the current file.

    A file ID is (for a writable filesystem) typically made of the inode number and
    a file generation number that is incremented every time a destructive operation
    on the file appears.

    "Stale NFS file handle" is the error code if the referred file (inode number +
    file generation number) no longer exists.

    I asume that in your case, the server did send out invalid file IDs that do not
    point to a valid file. For this reason, the client gets a "Stale NFS file
    handle" when it uses the NFS file handle returned by the NFS server for a
    open() operation.

    This seems to be an "explanation" from the people who wrote the non-working NFS
    code.
    we were unable to make the 'fsid=uuid' option work (or we didn't
    understand it), but using fsid=## for unique integers for each export
    works fine, so thats what we went with.

    are these fsid's the same as your 32 vs 64 bit file handles ? doesn't
    sound like it to me, unless I'm misunderstanding what you're referring
    to as a file handle.
    If you introduce new words, you would need to explain them. I can only explain
    how NFS works and that it is possible to use NFS to export filesystems with
    more than 32 bit inode numbers.

    Just a note: even with NFSv2, you could have an 8 Byte inode number + 2 Byte
    file generation number or 7 Byte inode number + 3 byte file generation number
    or 6 Byte inode number + 4 Byte file generation number.

    It is most unlikely that a filesystem designed for NFSv2 export will use the
    full 8 byte address space that 64 bit inode number could allow.


    J?rg

    --
    EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
    js at cs.tu-berlin.de (uni)
    joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
    URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Related Discussions

People

Translate

site design / logo © 2022 Grokbase