FAQ
We are planning to setup RAC with two node on Redhat Linux. We want to
use RAW devices with multipath using Hitachi 9970. We could see shared
storage from both the nodes. Cluster Verify fails with following error
message.



[oracle_at_soadbstg01 cluvfy]$ ./runcluvfy.sh comp ssa -n
soadbstg01,soadbstg02

Verifying shared storage accessibility

Checking shared storage accessibility...

WARNING:

Unable to determine the sharedness of /dev/sda on nodes:
soadbstg02,soadbstg01

Shared storage check failed on nodes "soadbstg02,soadbstg01".
Oracle (cvuqdisk) is looking for /dev/sd* location. RAW Partitions are
configured as /dev/oracleocr. Is there a way to bypass default location?

/usr/sbin/cvuqdisk seems to look for /dev/sd*
trying to pass /dev/oracleocr/ocr01 doesn't seem to work.
Is there a way to pass an alternate device path?



Thanks,

Rama



This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.

Search Discussions

  • Rajeev Prabhakar at Aug 29, 2007 at 9:31 pm
    Have you already tried the by-name device names ? Are you folks using udev ?
    On 8/29/07, rama.ari_at_accenture.com wrote:

    We are planning to setup RAC with two node on Redhat Linux. We want to
    use RAW devices with multipath using Hitachi 9970. We could see shared
    storage from both the nodes. Cluster Verify fails with following error
    message.



    [oracle_at_soadbstg01 cluvfy]$ ./runcluvfy.sh comp ssa -n
    soadbstg01,soadbstg02

    Verifying shared storage accessibility

    Checking shared storage accessibility...

    WARNING:
    Unable to determine the sharedness of /dev/sda on nodes:
    soadbstg02,soadbstg01

    Shared storage check failed on nodes "soadbstg02,soadbstg01".
    Oracle (cvuqdisk) is looking for /dev/sd* location. RAW Partitions are
    configured as /dev/oracleocr. Is there a way to bypass default location?

    /usr/sbin/cvuqdisk seems to look for /dev/sd*
    trying to pass /dev/oracleocr/ocr01 doesn't seem to work.
    Is there a way to pass an alternate device path?



    Thanks,

    Rama



    This message is for the designated recipient only and may contain
    privileged, proprietary, or otherwise private information. If you have
    received it in error, please notify the sender immediately and delete the
    original. Any other use of the email by you is prohibited.
    --
    http://www.freelists.org/webpage/oracle-l
  • Charles Schultz at Aug 29, 2007 at 10:57 pm
    I sent this to Rama earlier, but wanted to include the list:

    We had similar issues on different hardware. ASMLib is looking for the first
    shared path in your mpath, which is not what you want. I suggest putting the
    EXCLUDE line in oracleasm as follows:

    /u01/app/oracle: cat /etc/sysconfig/oracleasm

    #
    # This is a configuration file for automatic loading of the Oracle
    # Automatic Storage Management library kernel driver. It is generated
    # By running /etc/init.d/oracleasm configure. Please use that method
    # to modify this file
    #

    # ORACLEASM_ENABELED: 'true' means to load the driver on boot.
    ORACLEASM_ENABLED=true

    # ORACLEASM_UID: Default user owning the /dev/oracleasm mount point.
    ORACLEASM_UID=oracle

    # ORACLEASM_GID: Default group owning the /dev/oracleasm mount point.
    ORACLEASM_GID=dba

    # ORACLEASM_SCANBOOT: 'true' means fix disk perms on boot
    ORACLEASM_SCANBOOT=true

    # ORACLEASM_CLEARBOOT: 'true' means clean old disk perms on boot
    ORACLEASM_CLEARBOOT=true

    # ORACLEASM_SCANORDER: Matching patterns to order disk scanning
    ORACLEASM_SCANORDER=

    # ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan
    ORACLEASM_SCANEXCLUDE=sd

    We are planning to setup RAC with two node on Redhat Linux. We want to use
    RAW devices with multipath using Hitachi 9970. We could see shared storage
    from both the nodes. Cluster Verify fails with following error message.



    [oracle_at_soadbstg01 cluvfy]$ ./runcluvfy.sh comp ssa -n
    soadbstg01,soadbstg02

    Verifying shared storage accessibility

    Checking shared storage accessibility...

    WARNING:
    Unable to determine the sharedness of /dev/sda on nodes:
    soadbstg02,soadbstg01

    Shared storage check failed on nodes "soadbstg02,soadbstg01".
    Oracle (cvuqdisk) is looking for /dev/sd* location. RAW Partitions are
    configured as /dev/oracleocr. Is there a way to bypass default location?

    /usr/sbin/cvuqdisk seems to look for /dev/sd*
    trying to pass /dev/oracleocr/ocr01 doesn't seem to work.
    Is there a way to pass an alternate device path?



    Thanks,

    Rama



    This message is for the designated recipient only and may contain
    privileged, proprietary, or otherwise private information. If you have
    received it in error, please notify the sender immediately and delete the
    original. Any other use of the email by you is prohibited.
    On 8/29/07, rama.ari_at_accenture.com wrote:
    --
    Charles Schultz

    --
    http://www.freelists.org/webpage/oracle-l
  • Rama.ari_at_accenture.com at Aug 30, 2007 at 3:01 pm
    Chrles,



    We tried your suggestion, it didn't work for us.



    Thanks,

    Rama

    From: Charles Schultz
    Sent: Wednesday, August 29, 2007 6:57 PM
    To: Ari, Rama
    Cc: oracle-l_at_freelists.org
    Subject: Re: RAC -- RAW Device Multipath (Shared Storage)



    I sent this to Rama earlier, but wanted to include the list:

    We had similar issues on different hardware. ASMLib is looking for the
    first shared path in your mpath, which is not what you want. I suggest
    putting the EXCLUDE line in oracleasm as follows:



    /u01/app/oracle: cat /etc/sysconfig/oracleasm
    #
    # This is a configuration file for automatic loading of the Oracle
    # Automatic Storage Management library kernel driver. It is generated
    # By running /etc/init.d/oracleasm configure. Please use that method
    # to modify this file
    #



    # ORACLEASM_ENABELED: 'true' means to load the driver on boot.
    ORACLEASM_ENABLED=true



    # ORACLEASM_UID: Default user owning the /dev/oracleasm mount point.
    ORACLEASM_UID=oracle



    # ORACLEASM_GID: Default group owning the /dev/oracleasm mount point.
    ORACLEASM_GID=dba



    # ORACLEASM_SCANBOOT: 'true' means fix disk perms on boot
    ORACLEASM_SCANBOOT=true



    # ORACLEASM_CLEARBOOT: 'true' means clean old disk perms on boot
    ORACLEASM_CLEARBOOT=true



    # ORACLEASM_SCANORDER: Matching patterns to order disk scanning
    ORACLEASM_SCANORDER=



    # ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan
    ORACLEASM_SCANEXCLUDE=sd

    We are planning to setup RAC with two node on Redhat
    Linux. We want to use RAW devices with multipath using Hitachi 9970. We
    could see shared storage from both the nodes. Cluster Verify fails with
    following error message.



    [oracle_at_soadbstg01 cluvfy]$ ./runcluvfy.sh comp ssa -n
    soadbstg01,soadbstg02


    Verifying shared storage accessibility


    Checking shared storage accessibility...


    WARNING:
    Unable to determine the sharedness of /dev/sda on nodes:
    soadbstg02,soadbstg01

    Shared storage check failed on nodes
    "soadbstg02,soadbstg01".
    Oracle (cvuqdisk) is looking for /dev/sd* location. RAW

    Partitions are configured as /dev/oracleocr. Is there a way to bypass
    default location?

    /usr/sbin/cvuqdisk seems to look for /dev/sd*
    trying to pass /dev/oracleocr/ocr01 doesn't seem to
    work.
    Is there a way to pass an alternate device path?

    Thanks,

    Rama

    This message is for the designated recipient only and

    may contain privileged, proprietary, or otherwise private information.
    If you have received it in error, please notify the sender immediately
    and delete the original. Any other use of the email by you is
    prohibited.

    On 8/29/07, rama.ari_at_accenture.com < rama.ari_at_accenture.com>
    wrote:

    --
    Charles Schultz

    This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.

    --
    http://www.freelists.org/webpage/oracle-l
  • Alex Gorbachev at Aug 30, 2007 at 1:05 am
    Known problem. ML Note 358617.1.

    Cluvfy currently does not work with devices other than scsi devices.

    So devices like emc powerpath, volume groups etc will fail when
    checked using cluvfy for shared accessibility.

    I usually check it manually - dd to it on one node and dd from it on
    another and check that mapping is the same.
    On 8/29/07, rama.ari_at_accenture.com wrote:




    We are planning to setup RAC with two node on Redhat Linux. We want to use
    RAW devices with multipath using Hitachi 9970. We could see shared storage
    from both the nodes. Cluster Verify fails with following error message.



    [oracle_at_soadbstg01 cluvfy]$ ./runcluvfy.sh comp ssa -n
    soadbstg01,soadbstg02

    Verifying shared storage accessibility

    Checking shared storage accessibility...

    WARNING:
    Unable to determine the sharedness of /dev/sda on nodes:
    soadbstg02,soadbstg01

    Shared storage check failed on nodes "soadbstg02,soadbstg01".
    Oracle (cvuqdisk) is looking for /dev/sd* location. RAW Partitions are
    configured as /dev/oracleocr. Is there a way to bypass default location?

    /usr/sbin/cvuqdisk seems to look for /dev/sd*
    trying to pass /dev/oracleocr/ocr01 doesn't seem to work.
    Is there a way to pass an alternate device path?



    Thanks,

    Rama






    This message is for the designated recipient only and may contain
    privileged, proprietary, or otherwise private information. If you have
    received it in error, please notify the sender immediately and delete the
    original. Any other use of the email by you is prohibited.
    --
    Alex Gorbachev, Oracle DBA Brewer, The Pythian Group
    http://www.pythian.com/blogs/author/alex http://www.oracloid.com
    BAAG party - www.BattleAgainstAnyGuess.com
    --
    http://www.freelists.org/webpage/oracle-l
  • Rama.ari_at_accenture.com at Aug 30, 2007 at 3:04 pm
    Alex,

    Thanks for your prompt reply. Oracle suggested applying patch 4679769 as
    per Note # 343092.1

    Thanks,
    Rama
    -----Original Message-----
    From: gorbyx_at_gmail.com On Behalf Of Alex
    Gorbachev
    Sent: Wednesday, August 29, 2007 9:05 PM
    To: Ari, Rama
    Cc: oracle-l_at_freelists.org
    Subject: Re: RAC -- RAW Device Multipath (Shared Storage)

    Known problem. ML Note 358617.1.

    Cluvfy currently does not work with devices other than scsi devices.

    So devices like emc powerpath, volume groups etc will fail when
    checked using cluvfy for shared accessibility.

    I usually check it manually - dd to it on one node and dd from it on
    another and check that mapping is the same.
    On 8/29/07, rama.ari_at_accenture.com wrote:




    We are planning to setup RAC with two node on Redhat Linux. We want to use
    RAW devices with multipath using Hitachi 9970. We could see shared storage
    from both the nodes. Cluster Verify fails with following error message.


    [oracle_at_soadbstg01 cluvfy]$ ./runcluvfy.sh comp ssa -n
    soadbstg01,soadbstg02

    Verifying shared storage accessibility

    Checking shared storage accessibility...

    WARNING:
    Unable to determine the sharedness of /dev/sda on nodes:
    soadbstg02,soadbstg01

    Shared storage check failed on nodes "soadbstg02,soadbstg01".
    Oracle (cvuqdisk) is looking for /dev/sd* location. RAW Partitions are
    configured as /dev/oracleocr. Is there a way to bypass default location?
    /usr/sbin/cvuqdisk seems to look for /dev/sd*
    trying to pass /dev/oracleocr/ocr01 doesn't seem to work.
    Is there a way to pass an alternate device path?



    Thanks,

    Rama






    This message is for the designated recipient only and may contain
    privileged, proprietary, or otherwise private information. If you have
    received it in error, please notify the sender immediately and delete the
    original. Any other use of the email by you is prohibited.
    --
    Alex Gorbachev, Oracle DBA Brewer, The Pythian Group
    http://www.pythian.com/blogs/author/alex http://www.oracloid.com
    BAAG party - www.BattleAgainstAnyGuess.com

    This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.
    --
    http://www.freelists.org/webpage/oracle-l
  • Alex Gorbachev at Aug 30, 2007 at 3:46 pm
    That's odd. These are the steps specific to MD Linus multipathing. You
    are using Hitachi multipathing drivers as I understand (I never had a
    chance to work with it). Patch 4679769 seems to be universal to all
    multipathing (it's discovered with power path). However, I have never
    applied it and always worked fine. Maybe it solves problem with block
    devices and not multipathing? For that, you should find Note 401132.1
    useful.
    On 8/30/07, rama.ari_at_accenture.com wrote:
    Alex,

    Thanks for your prompt reply. Oracle suggested applying patch 4679769 as
    per Note # 343092.1
    --
    Alex Gorbachev, Oracle DBA Brewer, The Pythian Group
    http://www.pythian.com/blogs/author/alex http://www.oracloid.com
    BAAG party - www.BattleAgainstAnyGuess.com
    --
    http://www.freelists.org/webpage/oracle-l
  • Rama.ari_at_accenture.com at Sep 17, 2007 at 9:06 pm
    Fyi

    Patch # 4679769 fixed our problem.

    -----Original Message-----
    From: oracle-l-bounce_at_freelists.org
    On Behalf Of
    rama.ari_at_accenture.com
    Sent: Thursday, August 30, 2007 11:04 AM
    To: ag_at_oracloid.com
    Cc: oracle-l_at_freelists.org
    Subject: RE: RAC -- RAW Device Multipath (Shared Storage)

    Alex,

    Thanks for your prompt reply. Oracle suggested applying patch 4679769 as
    per Note # 343092.1

    Thanks,
    Rama
    -----Original Message-----
    From: gorbyx_at_gmail.com On Behalf Of Alex
    Gorbachev
    Sent: Wednesday, August 29, 2007 9:05 PM
    To: Ari, Rama
    Cc: oracle-l_at_freelists.org
    Subject: Re: RAC -- RAW Device Multipath (Shared Storage)

    Known problem. ML Note 358617.1.

    Cluvfy currently does not work with devices other than scsi devices.

    So devices like emc powerpath, volume groups etc will fail when
    checked using cluvfy for shared accessibility.

    I usually check it manually - dd to it on one node and dd from it on
    another and check that mapping is the same.
    On 8/29/07, rama.ari_at_accenture.com wrote:




    We are planning to setup RAC with two node on Redhat Linux. We want to use
    RAW devices with multipath using Hitachi 9970. We could see shared storage
    from both the nodes. Cluster Verify fails with following error message.


    [oracle_at_soadbstg01 cluvfy]$ ./runcluvfy.sh comp ssa -n
    soadbstg01,soadbstg02

    Verifying shared storage accessibility

    Checking shared storage accessibility...

    WARNING:
    Unable to determine the sharedness of /dev/sda on nodes:
    soadbstg02,soadbstg01

    Shared storage check failed on nodes "soadbstg02,soadbstg01".
    Oracle (cvuqdisk) is looking for /dev/sd* location. RAW Partitions are
    configured as /dev/oracleocr. Is there a way to bypass default location?
    /usr/sbin/cvuqdisk seems to look for /dev/sd*
    trying to pass /dev/oracleocr/ocr01 doesn't seem to work.
    Is there a way to pass an alternate device path?



    Thanks,

    Rama






    This message is for the designated recipient only and may contain
    privileged, proprietary, or otherwise private information. If you have
    received it in error, please notify the sender immediately and delete the
    original. Any other use of the email by you is prohibited.
    --
    Alex Gorbachev, Oracle DBA Brewer, The Pythian Group
    http://www.pythian.com/blogs/author/alex http://www.oracloid.com
    BAAG party - www.BattleAgainstAnyGuess.com

    This message is for the designated recipient only and may contain
    privileged, proprietary, or otherwise private information. If you have
    received it in error, please notify the sender immediately and delete
    the original. Any other use of the email by you is prohibited.
    --
    http://www.freelists.org/webpage/oracle-l

    This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.
    --
    http://www.freelists.org/webpage/oracle-l
  • LS Cheng at Aug 30, 2007 at 4:58 am
    cluvfy is quite hopeless with shared storage, try what alex auggested

    thanks

    --
    LSC
    On 8/29/07, rama.ari_at_accenture.com wrote:

    We are planning to setup RAC with two node on Redhat Linux. We want to
    use RAW devices with multipath using Hitachi 9970. We could see shared
    storage from both the nodes. Cluster Verify fails with following error
    message.



    [oracle_at_soadbstg01 cluvfy]$ ./runcluvfy.sh comp ssa -n
    soadbstg01,soadbstg02

    Verifying shared storage accessibility

    Checking shared storage accessibility...

    WARNING:
    Unable to determine the sharedness of /dev/sda on nodes:
    soadbstg02,soadbstg01

    Shared storage check failed on nodes "soadbstg02,soadbstg01".
    Oracle (cvuqdisk) is looking for /dev/sd* location. RAW Partitions are
    configured as /dev/oracleocr. Is there a way to bypass default location?

    /usr/sbin/cvuqdisk seems to look for /dev/sd*
    trying to pass /dev/oracleocr/ocr01 doesn't seem to work.
    Is there a way to pass an alternate device path?



    Thanks,

    Rama



    This message is for the designated recipient only and may contain
    privileged, proprietary, or otherwise private information. If you have
    received it in error, please notify the sender immediately and delete the
    original. Any other use of the email by you is prohibited.
    --
    http://www.freelists.org/webpage/oracle-l
  • Alex Gorbachev at Aug 30, 2007 at 11:04 pm
    CCing the list just in case someone finds it useful.

    On one node run:
    echo LUN01 | dd of=/dev/emcpowera bs=5 count=1
    echo LUN02 | dd of=/dev/emcpowerb bs=5 count=1
    ...

    So you associate emcpowera with LUN1, emcpowerb with LUN2 and etc.
    On other nodes you read from every LUN:
    dd if=/dev/emcpowera bs=5 count=1 2>/dev/null; echo
    dd if=/dev/emcpowerb bs=5 count=1 2>/dev/null; echo
    ...

    This will help you to determine what is what and it's independent of
    which multipathing (if any) you use.

    Of course, another way is to get your System/Storage Admin to tell you
    but if you don't want to loose time or they can't be bothered or don't
    know or whatsover, you have this option.
    On 8/29/07, Ravi Gaur wrote:
    Alex,
    Can you provide an example on the dd command to check on the mappings? That
    will be very helpful.
    --
    Alex Gorbachev, Oracle DBA Brewer, The Pythian Group
    http://www.pythian.com/blogs/author/alex http://www.oracloid.com
    BAAG party - www.BattleAgainstAnyGuess.com
    --
    http://www.freelists.org/webpage/oracle-l

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouporacle-l @
categoriesoracle
postedAug 29, '07 at 6:31p
activeSep 17, '07 at 9:06p
posts10
users5
websiteoracle.com

People

Translate

site design / logo © 2022 Grokbase