FAQ
G'day,

I'm involved in a project to migrate a 4TB database from HP/UX 11 and Oracle 9i to a brand-new Sun M5000 server with Oracle 11.2.0.3. This database suffers insert transactions in the order of 70 tx/sec. The daily redo production is in the order of 45GB. Management reports are also run with great vigour (i.e. large volumes of disk read IOPS). Two further tiny instances (5-7GB each) also live in the same environment.

The original plan was to install Solaris 10 on the new server and create a big ZFS pool on the san, as proposed by Oracle Sales. However, doubts have arisen as to the performance of ZFS with Oracle databases, and we now lean towards using UFS for the database files. All discussions and white papers that I have been able to find on the subject stress to closely follow the upgrade path, as ZFS is continuously being improved still. Some blogs give pointers on how to make ZFS perform "almost the same as UFS", which sounds to me as a lot of extra effort for no gain. I struggle to find any validation for choosing ZFS over UFS.

Today, the boss was told by a relation who used to work for Sun that that relation would no longer install boxes with UFS. He would also enable direct IO instead of totally relying on ZFS. The SAN disks should according to this relation be presented as raw disks, rather than striped-and-mirrored LUNs, to be RAIDed in ZFS. Apparently there are desirable features in ZFS that make this worthwhile. It should be noted that the SAN is (almost) completely dedicated to this one database machine and has block copy capabilities, built-in raid, etc.

To me it seems a bit back-to-front to disable the SAN functionality, effectively turning it into an expensive external disk array, and at the same time shifting all the work that the SAN would have done to the database machine CPU where it competes for resources with the Oracle instances. What advantages, if any, exist that make using ZFS in this way is preferable over UFS? Do you have any experience with it?

The Solaris version was bought before Oracle certified 11.2.0.3 on Solaris 11, but now it seems silly not to upgrade Solaris before this system goes life. It will quite possibly not be able to be upgraded any time soon, possibly not until after Oracle 14x is released.. ;) The same relation however also insisted that "there are certification issues with Solaris 11" and he would never install Oracle 11g database on Solaris 11. However, MOS clearly shows that 11.2.0.3 is fully certified on Solaris 11. Do you happen to know what issues could exist that pre-empt the use of Solaris 11, even if that might mean that the client will be on Solaris 10 for the next decade?

I would like to hear about your experiences and thoughts.

Cheers,
Tony

Search Discussions

  • Przemyslaw Bak at Mar 29, 2012 at 7:43 am
    Hi Tony,

    - regarding UFS vs ZFS. The best thing is to do any (!) sort of tests. But if you really cannot do This is really difficult question since:
    - zfs is brand new filesystem which will be improved more and more. UFS will not be improved.
    - but does it matter for you if you will keep this system for years without change ?
    - zfs has much more features which are unknown for UFS (and never will be)
    - but do you really need them ?
    - if you insist on using ZFS for Oracle read the following URL: http://www.solarisinternals.com/wiki/index.php/ZFS_for_Databases#Oracle_Considerations
    - there are a lot of knowledge (in terms of people experience) in the internet about using UFS + Oracle
    - if you can do any sort of tests (ZFS vs UFS):
    - you can do it on both ZFS and UFS using:
    - Orion (Oracle tool to test storage performance)
    - of course don't relay on just one tool and its results
    - Oracle 11 IO calibration (new feature)
    and just compare the results
    - a couple of URLs:
    - https://blogs.oracle.com/roch/entry/zfs_and_directio

    - regarding Solaris 10 vs 11
    - Solaris 11 has many new features. You can read WPs about What's new, etc. It's worth reading.
    - but do you need them for typical OLTP (DSS ?) environment ?
    - Solaris 10 is stable and predictable
    - but does it matter for you ? Maybe you like new environments ? New features ?
    - having Solaris 10 does not mean that you cannot upgrade to Solaris 11 in the future. Live Upgrade is a feature which helps you in this area.
    - if you happen to have a bug in Solaris 11 Oracle support is not known to be the best on this planet regarding fixing new bugs ...

    - regarding SAN
    - if you have typical hardware array I would not mirror at the filesystem level - don't complicate this.
    - quite old but anyway ... http://storagemojo.com/2007/04/23/new-zfs-performance-numbers/

    Best regards
    Przemyslaw Bak (przemol)
    --
    http://przemol.blogspot.com/
    On Wed, Mar 28, 2012 at 11:34:25PM +1000, De DBA wrote:
    G'day,

    I'm involved in a project to migrate a 4TB database from HP/UX 11 and Oracle 9i to a brand-new Sun M5000 server with Oracle 11.2.0.3. This database suffers insert transactions in the order of 70 tx/sec. The daily redo production is in the order of 45GB. Management reports are also run with great vigour (i.e. large volumes of disk read IOPS). Two further tiny instances (5-7GB each) also live in the same environment.

    The original plan was to install Solaris 10 on the new server and create a big ZFS pool on the san, as proposed by Oracle Sales. However, doubts have arisen as to the performance of ZFS with Oracle databases, and we now lean towards using UFS for the database files. All discussions and white papers that I have been able to find on the subject stress to closely follow the upgrade path, as ZFS is continuously being improved still. Some blogs give pointers on how to make ZFS perform "almost the same as UFS", which sounds to me as a lot of extra effort for no gain. I struggle to find any validation for choosing ZFS over UFS.

    Today, the boss was told by a relation who used to work for Sun that that relation would no longer install boxes with UFS. He would also enable direct IO instead of totally relying on ZFS. The SAN disks should according to this relation be presented as raw disks, rather than striped-and-mirrored LUNs, to be RAIDed in ZFS. Apparently there are desirable features in ZFS that make this worthwhile. It should be noted that the SAN is (almost) completely dedicated to this one database machine and has block copy capabilities, built-in raid, etc.

    To me it seems a bit back-to-front to disable the SAN functionality, effectively turning it into an expensive external disk array, and at the same time shifting all the work that the SAN would have done to the database machine CPU where it competes for resources with the Oracle instances. What advantages, if any, exist that make using ZFS in this way is preferable over UFS? Do you have any experience with it?

    The Solaris version was bought before Oracle certified 11.2.0.3 on Solaris 11, but now it seems silly not to upgrade Solaris before this system goes life. It will quite possibly not be able to be upgraded any time soon, possibly not until after Oracle 14x is released.. ;) The same relation however also insisted that "there are certification issues with Solaris 11" and he would never install Oracle 11g database on Solaris 11. However, MOS clearly shows that 11.2.0.3 is fully certified on Solaris 11. Do you happen to know what issues could exist that pre-empt the use of Solaris 11, even if that might mean that the client will be on Solaris 10 for the next decade?

    I would like to hear about your experiences and thoughts.

    Cheers,
    Tony

    --
    http://www.freelists.org/webpage/oracle-l


















































    --
    http://www.freelists.org/webpage/oracle-l
  • De DBA at Mar 29, 2012 at 10:22 am
    Thanks, Przemyslaw
    When I said that the system won't be able to be upgraded, I was not thinking of technical complications. Rather, the importance of this 24/7 database and the jitter factor associated with an upgrade project this size will encourage the management to sit it out as long as possible. Live Upgrade is not going to help there.. ;)

    We need to strike a balance between two conflicting interests here:

    1. Install the newest possible software, so that the system will have the longest possible period into the future of active vendor support
    2. Deliver the most stable and robust solution possible so that the system will suffer as few hickups as possible.

    In this light ZFS sounded not like there was an overriding reason to use it, as we do not need any of the extra features that it offers. It is young and not necessarily stable, and therefore an unknown risk. In the meantime we have decided to give the older and better understood ASM a go (thanks Frits, for reminding me of it).

    You are right, Solaris 11 is also squeakingly new, just out of the shrinkwrap. But, if I understand correctly, its new features have had a good flogging in the wild through the OpenSolaris Project and Solaris 11 Express (which is now replaced by Sol11), have they not? So the risk is smaller than it seems at first glance, and it stands to reason that Oracle/Sun will put more effort towards fixing OS bugs in this new OS, rather than old, trusty Sol10. Our project will include a good volume testing phase, which should expose serious flaws (if any) and the entire project will not end until the second half of this year. Based on previous experience, I would expect that Oracle/Sun release an update before that.. ;)

    Cheers,
    Tony

    On 29/03/12 17:41, przemolicc@poczta.fm wrote:
    Hi Tony,

    - regarding UFS vs ZFS. The best thing is to do any (!) sort of tests. But if you really cannot do This is really difficult question since:
    - zfs is brand new filesystem which will be improved more and more. UFS will not be improved.
    - but does it matter for you if you will keep this system for years without change ?
    - zfs has much more features which are unknown for UFS (and never will be)
    - but do you really need them ?
    - if you insist on using ZFS for Oracle read the following URL: http://www.solarisinternals.com/wiki/index.php/ZFS_for_Databases#Oracle_Considerations
    - there are a lot of knowledge (in terms of people experience) in the internet about using UFS + Oracle
    - if you can do any sort of tests (ZFS vs UFS):
    - you can do it on both ZFS and UFS using:
    - Orion (Oracle tool to test storage performance)
    - of course don't relay on just one tool and its results
    - Oracle 11 IO calibration (new feature)
    and just compare the results
    - a couple of URLs:
    - https://blogs.oracle.com/roch/entry/zfs_and_directio

    - regarding Solaris 10 vs 11
    - Solaris 11 has many new features. You can read WPs about What's new, etc. It's worth reading.
    - but do you need them for typical OLTP (DSS ?) environment ?
    - Solaris 10 is stable and predictable
    - but does it matter for you ? Maybe you like new environments ? New features ?
    - having Solaris 10 does not mean that you cannot upgrade to Solaris 11 in the future. Live Upgrade is a feature which helps you in this area.
    - if you happen to have a bug in Solaris 11 Oracle support is not known to be the best on this planet regarding fixing new bugs ...

    - regarding SAN
    - if you have typical hardware array I would not mirror at the filesystem level - don't complicate this.
    - quite old but anyway ... http://storagemojo.com/2007/04/23/new-zfs-performance-numbers/

    Best regards
    Przemyslaw Bak (przemol)
    --
    http://przemol.blogspot.com/


    --
    http://www.freelists.org/webpage/oracle-l
  • Przemyslaw Bak at Mar 29, 2012 at 12:24 pm

    On Thu, Mar 29, 2012 at 08:21:22PM +1000, De DBA wrote:
    Thanks, Przemyslaw

    When I said that the system won't be able to be upgraded, I was not thinking of technical complications. Rather, the importance of this 24/7 database and the jitter factor associated with an upgrade project this size will encourage the management to sit it out as long as possible. Live Upgrade is not going to help there.. ;)

    We need to strike a balance between two conflicting interests here:

    1. Install the newest possible software, so that the system will have the longest possible period into the future of active vendor support
    Until you find a bug (don't expect no bugs in a long periods). And the effort to install fixes is almost the same as in upgrade to newer release. In both
    cases you use LU ...
    2. Deliver the most stable and robust solution possible so that the system will suffer as few hickups as possible.

    In this light ZFS sounded not like there was an overriding reason to use it, as we do not need any of the extra features that it offers. It is young and not necessarily stable, and therefore an unknown risk. In the meantime we have decided to give the older and better understood ASM a go (thanks Frits, for reminding me of it).

    You are right, Solaris 11 is also squeakingly new, just out of the shrinkwrap. But, if I understand correctly, its new features have had a good flogging in the wild through the OpenSolaris Project and Solaris 11 Express (which is now replaced by Sol11), have they not? So the risk is smaller than it seems at first glance, and it stands to reason that Oracle/Sun will put more effort towards fixing OS bugs in this new OS, rather than old, trusty Sol10. Our project will include a good volume testing phase, which should expose serious flaws (if any) and the entire project will not end until the second half of this year. Based on previous experience, I would expect that Oracle/Sun release an update before that.. ;)
    Well, because of OpenSolaris/Solaris 11 Express I believe that it is better tested then any other OS which gets to the market. But I don't believe that Solaris 11 + Oracle 11 is also so well tested ...
    You don't want to use ZFS beacuse it is young. ZFS is much more mature then Solaris 11 ... ;-) (just kidding but ...)
    You insist on stability so Solaris 10 is IMHO better choice.


    Regards
    Przemyslaw Bak (przemol)
    --
    http://przemol.blogspot.com/
    Cheers,
    Tony

    On 29/03/12 17:41, przemolicc@poczta.fm wrote:
    Hi Tony,

    - regarding UFS vs ZFS. The best thing is to do any (!) sort of tests. But if you really cannot do This is really difficult question since:
    - zfs is brand new filesystem which will be improved more and more. UFS will not be improved.
    - but does it matter for you if you will keep this system for years without change ?
    - zfs has much more features which are unknown for UFS (and never will be)
    - but do you really need them ?
    - if you insist on using ZFS for Oracle read the following URL: http://www.solarisinternals.com/wiki/index.php/ZFS_for_Databases#Oracle_Considerations
    - there are a lot of knowledge (in terms of people experience) in the internet about using UFS + Oracle
    - if you can do any sort of tests (ZFS vs UFS):
    - you can do it on both ZFS and UFS using:
    - Orion (Oracle tool to test storage performance)
    - of course don't relay on just one tool and its results
    - Oracle 11 IO calibration (new feature)
    and just compare the results
    - a couple of URLs:
    - https://blogs.oracle.com/roch/entry/zfs_and_directio

    - regarding Solaris 10 vs 11
    - Solaris 11 has many new features. You can read WPs about What's new, etc. It's worth reading.
    - but do you need them for typical OLTP (DSS ?) environment ?
    - Solaris 10 is stable and predictable
    - but does it matter for you ? Maybe you like new environments ? New features ?
    - having Solaris 10 does not mean that you cannot upgrade to Solaris 11 in the future. Live Upgrade is a feature which helps you in this area.
    - if you happen to have a bug in Solaris 11 Oracle support is not known to be the best on this planet regarding fixing new bugs ...

    - regarding SAN
    - if you have typical hardware array I would not mirror at the filesystem level - don't complicate this.
    - quite old but anyway ... http://storagemojo.com/2007/04/23/new-zfs-performance-numbers/

    Best regards
    Przemyslaw Bak (przemol)
    --
    http://przemol.blogspot.com/


















































    --
    http://www.freelists.org/webpage/oracle-l
  • De DBA at Mar 29, 2012 at 9:50 am
    Thanks, Frits.
    The client is happy to go with ASM, as it is a tried and tested solution :)

    I have no actual experience with ASM, only theoretical knowledge and playing around with test installations.. Given that the database consists of a mountain of 30GB files, and the method of migration will be transportable tablespaces, would you have a suggestion as to how large the SAN slices should ideally be? The SAN administrator is suggesting to present LUNs of 3TB or 5TB (on SATA and SAS disks respectively). Is there a best practice for this kind of installations?

    Cheers,
    Tony
    On 28/03/12 23:46, Frits Hoogland wrote:


    Frits Hoogland

    http://fritshoogland.wordpress.com <http://fritshoogland.wordpress.com/>
    mailto:frits.hoogland@gmail.com
    cell: +31 6 53569942
    On Mar 28, 2012, at 3:34 PM, De DBA wrote:

    G'day,

    I'm involved in a project to migrate a 4TB database from HP/UX 11 and Oracle 9i to a brand-new Sun M5000 server with Oracle 11.2.0.3. This database suffers insert transactions in the order of 70 tx/sec. The daily redo production is in the order of 45GB. Management reports are also run with great vigour (i.e. large volumes of disk read IOPS). Two further tiny instances (5-7GB each) also live in the same environment.
    Because (probably) both the HPUX and the Sun M5000 box have an approximate same CPU speed, you will not suffer from the "slowness" of the SPAC CPU when compared with Intel. Still Intel (x64) will perform much better in terms of latency because of much higher CPU speed. M5000 has some advantages for concurrency (more threads on the CPU's, probably more memory), aka bandwidth.
    The original plan was to install Solaris 10 on the new server and create a big ZFS pool on the san, as proposed by Oracle Sales. However, doubts have arisen as to the performance of ZFS with Oracle databases, and we now lean towards using UFS for the database files. All discussions and white papers that I have been able to find on the subject stress to closely follow the upgrade path, as ZFS is continuously being improved still. Some blogs give pointers on how to make ZFS perform "almost the same as UFS", which sounds to me as a lot of extra effort for no gain. I struggle to find any validation for choosing ZFS over UFS.
    ZFS does not support DIO. So you effectively double buffer all IO.

    ZFS is made with flexiblity in mind /for system administrators, not for DBA's. /So ZFS is a resolution for a problem /you do not have/.
    Today, the boss was told by a relation who used to work for Sun that that relation would no longer install boxes with UFS. He would also enable direct IO instead of totally relying on ZFS. The SAN disks should according to this relation be presented as raw disks, rather than striped-and-mirrored LUNs, to be RAIDed in ZFS. Apparently there are desirable features in ZFS that make this worthwhile. It should be noted that the SAN is (almost) completely dedicated to this one database machine and has block copy capabilities, built-in raid, etc.
    Use ASM with SAN slices which are RAID 10. Period.

    This requires to learn ASM, but it will keep your data balanced across all slices. Make slices of a handy, moderate size, so you can add slices.
    To me it seems a bit back-to-front to disable the SAN functionality, effectively turning it into an expensive external disk array, and at the same time shifting all the work that the SAN would have done to the database machine CPU where it competes for resources with the Oracle instances. What advantages, if any, exist that make using ZFS in this way is preferable over UFS? Do you have any experience with it?
    See ZFS comments above.
    The Solaris version was bought before Oracle certified 11.2.0.3 on Solaris 11, but now it seems silly not to upgrade Solaris before this system goes life. It will quite possibly not be able to be upgraded any time soon, possibly not until after Oracle 14x is released.. ;) The same relation however also insisted that "there are certification issues with Solaris 11" and he would never install Oracle 11g database on Solaris 11. However, MOS clearly shows that 11.2.0.3 is fully certified on Solaris 11. Do you happen to know what issues could exist that pre-empt the use of Solaris 11, even if that might mean that the client will be on Solaris 10 for the next decade?

    I would like to hear about your experiences and thoughts.

    Cheers,
    Tony

    --
    http://www.freelists.org/webpage/oracle-l


    --
    http://www.freelists.org/webpage/oracle-l
  • Mark W. Farnham at Mar 29, 2012 at 10:17 am
    G'day - I'm just curious why there is no question about using ASM with
    external redundancy, putting the bulk of your storage requirement into a
    format that is the path forward for Oracle while letting the SAN box take
    the load on the stuff it is good at, opening the possibility of moving to
    RAC without reloading everything, and making the total contents size in file
    systems on UFS or ZFS small enough that moving file systems or changing your
    mind is no big whoop.

    Along with the other poster to this thread, I'd suggest testing and
    especially testing that a proposed configuration provides the i/o throughput
    you require with a margin for growth.

    Regards,

    mwf
    -----Original Message-----
    From: oracle-l-bounce@freelists.org
    On Behalf Of De DBA
    Sent: Wednesday, March 28, 2012 9:34 AM
    To: oracle-l@freelists.org
    Subject: ZFS or UFS? Solaris 11 or better stay with Solaris 10?

    G'day,

    I'm involved in a project to migrate a 4TB database from HP/UX 11 and Oracle
    9i to a brand-new Sun M5000 server with Oracle 11.2.0.3. This database
    suffers insert transactions in the order of 70 tx/sec. The daily redo
    production is in the order of 45GB. Management reports are also run with
    great vigour (i.e. large volumes of disk read IOPS). Two further tiny
    instances (5-7GB each) also live in the same environment.

    The original plan was to install Solaris 10 on the new server and create a
    big ZFS pool on the san, as proposed by Oracle Sales. However, doubts have
    arisen as to the performance of ZFS with Oracle databases, and we now lean
    towards using UFS for the database files. All discussions and white papers
    that I have been able to find on the subject stress to closely follow the
    upgrade path, as ZFS is continuously being improved still. Some blogs give
    pointers on how to make ZFS perform "almost the same as UFS", which sounds
    to me as a lot of extra effort for no gain. I struggle to find any
    validation for choosing ZFS over UFS.

    Today, the boss was told by a relation who used to work for Sun that that
    relation would no longer install boxes with UFS. He would also enable direct
    IO instead of totally relying on ZFS. The SAN disks should according to this
    relation be presented as raw disks, rather than striped-and-mirrored LUNs,
    to be RAIDed in ZFS. Apparently there are desirable features in ZFS that
    make this worthwhile. It should be noted that the SAN is (almost) completely
    dedicated to this one database machine and has block copy capabilities,
    built-in raid, etc.

    To me it seems a bit back-to-front to disable the SAN functionality,
    effectively turning it into an expensive external disk array, and at the
    same time shifting all the work that the SAN would have done to the database
    machine CPU where it competes for resources with the Oracle instances. What
    advantages, if any, exist that make using ZFS in this way is preferable over
    UFS? Do you have any experience with it?

    The Solaris version was bought before Oracle certified 11.2.0.3 on Solaris
    11, but now it seems silly not to upgrade Solaris before this system goes
    life. It will quite possibly not be able to be upgraded any time soon,
    possibly not until after Oracle 14x is released.. ;) The same relation
    however also insisted that "there are certification issues with Solaris 11"
    and he would never install Oracle 11g database on Solaris 11. However, MOS
    clearly shows that 11.2.0.3 is fully certified on Solaris 11. Do you happen
    to know what issues could exist that pre-empt the use of Solaris 11, even if
    that might mean that the client will be on Solaris 10 for the next decade?

    I would like to hear about your experiences and thoughts.

    Cheers,
    Tony

    --
    http://www.freelists.org/webpage/oracle-l


    --
    http://www.freelists.org/webpage/oracle-l
  • Mark W. Farnham at Mar 29, 2012 at 10:21 am
    Of course while I was typing this the response from Frits came through. At
    least we didn't disagree about anything....

    -----Original Message-----
    From: oracle-l-bounce@freelists.org
    On Behalf Of Mark W. Farnham
    Sent: Thursday, March 29, 2012 6:16 AM
    To: oracle-l@freelists.org
    Subject: RE: ZFS or UFS? Solaris 11 or better stay with Solaris 10?

    G'day - I'm just curious why there is no question about using ASM with
    external redundancy, putting the bulk of your storage requirement into a
    format that is the path forward for Oracle while letting the SAN box take
    the load on the stuff it is good at, opening the possibility of moving to
    RAC without reloading everything, and making the total contents size in file
    systems on UFS or ZFS small enough that moving file systems or changing your
    mind is no big whoop.

    Along with the other poster to this thread, I'd suggest testing and
    especially testing that a proposed configuration provides the i/o throughput
    you require with a margin for growth.

    Regards,

    mwf
    -----Original Message-----
    From: oracle-l-bounce@freelists.org
    On Behalf Of De DBA
    Sent: Wednesday, March 28, 2012 9:34 AM
    To: oracle-l@freelists.org
    Subject: ZFS or UFS? Solaris 11 or better stay with Solaris 10?

    G'day,

    I'm involved in a project to migrate a 4TB database from HP/UX 11 and Oracle
    9i to a brand-new Sun M5000 server with Oracle 11.2.0.3. This database
    suffers insert transactions in the order of 70 tx/sec. The daily redo
    production is in the order of 45GB. Management reports are also run with
    great vigour (i.e. large volumes of disk read IOPS). Two further tiny
    instances (5-7GB each) also live in the same environment.

    The original plan was to install Solaris 10 on the new server and create a
    big ZFS pool on the san, as proposed by Oracle Sales. However, doubts have
    arisen as to the performance of ZFS with Oracle databases, and we now lean
    towards using UFS for the database files. All discussions and white papers
    that I have been able to find on the subject stress to closely follow the
    upgrade path, as ZFS is continuously being improved still. Some blogs give
    pointers on how to make ZFS perform "almost the same as UFS", which sounds
    to me as a lot of extra effort for no gain. I struggle to find any
    validation for choosing ZFS over UFS.

    Today, the boss was told by a relation who used to work for Sun that that
    relation would no longer install boxes with UFS. He would also enable direct
    IO instead of totally relying on ZFS. The SAN disks should according to this
    relation be presented as raw disks, rather than striped-and-mirrored LUNs,
    to be RAIDed in ZFS. Apparently there are desirable features in ZFS that
    make this worthwhile. It should be noted that the SAN is (almost) completely
    dedicated to this one database machine and has block copy capabilities,
    built-in raid, etc.

    To me it seems a bit back-to-front to disable the SAN functionality,
    effectively turning it into an expensive external disk array, and at the
    same time shifting all the work that the SAN would have done to the database
    machine CPU where it competes for resources with the Oracle instances. What
    advantages, if any, exist that make using ZFS in this way is preferable over
    UFS? Do you have any experience with it?

    The Solaris version was bought before Oracle certified 11.2.0.3 on Solaris
    11, but now it seems silly not to upgrade Solaris before this system goes
    life. It will quite possibly not be able to be upgraded any time soon,
    possibly not until after Oracle 14x is released.. ;) The same relation
    however also insisted that "there are certification issues with Solaris 11"
    and he would never install Oracle 11g database on Solaris 11. However, MOS
    clearly shows that 11.2.0.3 is fully certified on Solaris 11. Do you happen
    to know what issues could exist that pre-empt the use of Solaris 11, even if
    that might mean that the client will be on Solaris 10 for the next decade?

    I would like to hear about your experiences and thoughts.

    Cheers,
    Tony

    --
    http://www.freelists.org/webpage/oracle-l


    --
    http://www.freelists.org/webpage/oracle-l


    --
    http://www.freelists.org/webpage/oracle-l
  • GG at Mar 29, 2012 at 4:56 pm

    W dniu 2012-03-28 15:34, De DBA pisze:
    G'day,
    ZFS is very mature and database friendly filesystem as long as You
    follow the rules :) mentioned here
    http://developers.sun.com/solaris/docs/wp-oraclezfsconfig-0510_ds_ac2.pdf
    :)

    Regards
    GregG
  • kyle Hailey at Mar 30, 2012 at 9:39 pm
    The Oracle paper is good.
    I'm involved exclusively with Oracle databases on ZFS, so in general it
    works well. I've listed below the issues I've run into and the solutions
    for each of them. There are many cool options that are only available on
    ZFS. The ZFS community is quite active in improving, fixing and adding
    functionality. There is quite a bit going on in the way it works. I'm still
    quite new to all the possibilities in ZFS.

    A few things to keep in mind:

    For best write performance the pool should be less than 80% free.

    ZFS by default double writes, similar to Oracle. Oracle writes to Redo and
    the datafile. Similarly ZFS writes to the ZIL (redo) and the files
    themselves. This can double the amount of writes which can be confusing
    when benchmarking the I/O. This can be modified with logbias as shown in
    the Oracle paper. Set the Oracle datafiles to throughput and the metadata
    will get written to the ZIL (like nologging operations in Oracle) but the
    data will be written to the data files. Put the redo in latency logbias
    mode so it gets committed through the ZIL. This is faster but will cause
    double writes.

    My experience is that ZFS read ahead and caching works well and especially
    for DSS queries. I can't say for sure but it seems ZFS is aggressive with
    read ahead and caching. If by chance you want less read ahead, ie you are
    only doing random 8k reads, you can turn read ahead off with
    zfs_prefetch_disable
    http://forums.freenas.org/archive/index.php/t-1076.html


    Two rare but problematic issues come to mind

    1. the ZFS ARC (like file system cache) stopped caching and spent all it's
    time kicking pages out. There is a fix, but not sure if it's out the open
    source community yet. To monitor, run
    echo '::arc ! egrep -w "c_min|c_max|size|arc_no_grow"' | pfexec mdb -k
    the problem manifests by a arc_no_grow set to 1 (ie don't grow it) and the
    arc size been well under the available memory.
    Seen this happen 4 times on maybe 100s of systems
    If happens then it might require a reboot.

    2. Write throughput dropped drastically after a flurry of disk errors. Disk
    errors were the core problem but it turns out that ZFS in became a bit
    overly protective, throttled writes down too far. Write throttling can be
    turned off with zfs_no_write_throttle
    Seen this happen once, but was quite confusing at the time.
    you should be able to monitor what ZFS thinks the write speed is with
    dtrace:
    dsl_pool_sync:entry
    /stringof(args[0]->dp_spa->spa_name) == "domain0"/
    {
    self->dp = args[0];
    }

    dsl_pool_sync:return
    /self->dp/
    {
    printf("write_limit %d, write_throughput %d\n",
    self->dp->dp_write_limit,
    self->dp->dp_throughput);
    self->dp = NULL;
    }


    Other things to be aware of, is that ZFS scrub can show up as a lot of
    reads when the filesystem would otherwise be idle. The ZFS scrub should
    back off strongly when user load comes on to give priority to other I/O.
    The scrub can be turned off with
    zpool scrub -s poolname
    The should only be temporary as its crucial that scrub gets run regularly
    (like weekly)

    There is no direct I/O per say. Data will get cached in the ARC. If by
    chance you want to turn off caching and simulate direct I/O (not suggesting
    this but it's useful for testing of the actual back end disks) you set
    caching off:
    zfs set primarycache=none poolname
    Note that this will still leave things cached that are already in the
    cache. You'd have to export the pool to clear the cache of an existing pool.

    ZFS will also call to the disks to flush them if the have cache like an
    internal array usually has. This can cause problems if that cache is
    battery backed and it interprets the flush as a force write to disk. The
    call to flush can be turned off with
    zfs_nocacheflush
    I think the Oracle paper discusses this.

    The parameters you want to set per pool are

    - compression - on/off, up to you
    - logbias - latency for redo, throughput datafiles
    - recordsize - blocksize for datafiles, 128K for others. Oracle paper
    gives recommendations
    - primarycache - all except archive (and UNDO) set to metadata
    - secondarycache - all for datafiles, none for others (probably)


    ZFS also bases calculations on LUNs so if your give a few LUNs to ZFS that
    represent many back end spindles, some of the I/O queue calculations can be
    off. I believe the Oracle paper goes into this. Never been a problem that
    I've directly seen though heard about it.

    As in all I/O systems block alignment is important. Unfortunately that is
    the point in this list I'm weakest on.

    Comments, additions, corrections welcome as this is all new to me :)


    - Kyle








    On Thu, Mar 29, 2012 at 9:49 AM, GG wrote:

    W dniu 2012-03-28 15:34, De DBA pisze:
    G'day,
    ZFS is very mature and database friendly filesystem as long as You
    follow the rules :) mentioned here
    http://developers.sun.com/solaris/docs/wp-oraclezfsconfig-0510_ds_ac2.pdf
    :)

    Regards
    GregG

    --
    http://www.freelists.org/webpage/oracle-l


    --
    http://www.freelists.org/webpage/oracle-l
  • kyle Hailey at Apr 2, 2012 at 6:47 pm
    That's for pointing that out.
    The pool should have a least 20% free for best write performance. (ie when
    it get's over 80% full, it's time to start thinking about reorg to keep
    write performance at it's maximum)


    - Kyle

    http://dboptimizer.com
    On Mon, Apr 2, 2012 at 5:24 AM, wrote:

    Di you mean "more than 80% free"?

    Thanks for the comments

    Joel Patterson
    Database Administrator
    904 727-2546

    --
    http://www.freelists.org/webpage/oracle-l

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouporacle-l @
categoriesoracle
postedMar 28, '12 at 1:35p
activeApr 2, '12 at 6:47p
posts10
users5
websiteoracle.com

People

Translate

site design / logo © 2022 Grokbase