FAQ
Hi all,

RAC on
Linux 2.6.18-92.el5 #1 SMP x86_64 x86_64 x86_64 GNU/Linux
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
ASMLib

We need to add disks to the Production environment for the first time.

Given the example below (querydisk on an existing disk):

# /etc/init.d/oracleasm querydisk -d BLOB
Disk "BLOB" is a valid ASM disk on device [66, 97]

# /etc/init.d/oracleasm querydisk -p BLOB
Disk "BLOB" is a valid ASM disk
/dev/sdam1: LABEL="BLOB" TYPE="oracleasm"
/dev/sdbc1: LABEL="BLOB" TYPE="oracleasm"
/dev/emcpowerz1: LABEL="BLOB" TYPE="oracleasm"

In /proc/partitions we have:

66 97 31457264 sdam1

ORACLEASM_SCANORDER and ORACLEASM_SCANEXCLUDE are both set to "" in
/etc/sysconfig/oracleasm.

So the question is: which path should we use with the createdisk
command (oracleasm createdisk )?
Is the multipathing used, given querydisk -d returns the major/minor
numbers of the phisical device?

After adding and marking the disks for asm from one node, we'll need to
run oracleasm scandisks on the remaining nodes,
is it safe to run this command (scandisks) in a live production
environment?

Thank you!

Dimitre

Search Discussions

  • Harel Safra at Apr 7, 2011 at 2:11 pm
    Use only /Dec/emcpower*
    Make sure to exclude /Dec/sd* - if you use them you won't be utilizing
    powerpath

    Harel Safra
    Sent from my phone.
    On Apr 7, 2011 5:04 PM, "Radoulov, Dimitre" wrote:
    Hi all,

    RAC on
    Linux 2.6.18-92.el5 #1 SMP x86_64 x86_64 x86_64 GNU/Linux
    Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit
    Production
    ASMLib

    We need to add disks to the Production environment for the first time.

    Given the example below (querydisk on an existing disk):

    # /etc/init.d/oracleasm querydisk -d BLOB
    Disk "BLOB" is a valid ASM disk on device [66, 97]

    # /etc/init.d/oracleasm querydisk -p BLOB
    Disk "BLOB" is a valid ASM disk
    /dev/sdam1: LABEL="BLOB" TYPE="oracleasm"
    /dev/sdbc1: LABEL="BLOB" TYPE="oracleasm"
    /dev/emcpowerz1: LABEL="BLOB" TYPE="oracleasm"

    In /proc/partitions we have:

    66 97 31457264 sdam1


    ORACLEASM_SCANORDER and ORACLEASM_SCANEXCLUDE are both set to "" in
    /etc/sysconfig/oracleasm.

    So the question is: which path should we use with the createdisk
    command (oracleasm createdisk )?
    Is the multipathing used, given querydisk -d returns the major/minor
    numbers of the phisical device?

    After adding and marking the disks for asm from one node, we'll need to
    run oracleasm scandisks on the remaining nodes,
    is it safe to run this command (scandisks) in a live production
    environment?


    Thank you!

    Dimitre



    --
    http://www.freelists.org/webpage/oracle-l
    --
    http://www.freelists.org/webpage/oracle-l
  • Martin Bach at Apr 7, 2011 at 2:14 pm
    Hi there,

    You need to actually exclude the native devices in the scan exclude variable, and use the pseudo-device instead (scan order variable). A great summary can be found here:

    Configuring Oracle ASMLib on Multipath Disks on Linux (Doc ID 394956.1)

    HTH,

    Martin
    -----Original Message-----
    From: oracle-l-bounce_at_freelists.org
    On Behalf Of Radoulov, Dimitre
    Sent: 07 April 2011 15:03
    To: ORACLE-L
    Subject: ASM and EMC PowerPath

    Hi all,

    RAC on
    ...
  • Radoulov, Dimitre at Apr 7, 2011 at 2:25 pm
    Thank you!
    I'm already reading all the notes I was able to find on MOS and google
    (including the below mentioned Configuring Oracle ASMLib on Multipath
    Disks on Linux).

    I was only unsure on what to do, given that the disks that are already
    configured point to the physical paths
    and the fact that this is a live production database.

    I doubt that we can change the paths of all disks online, could we?

    Set ORACLEASM_SCANEXCLUDE to "sd"
    oracleasm scandisks on all nodes.

    I suppose we need to stop all asm and client db instances.

    Best regards
    Dimitre
    On 07/04/2011 16:14, Martin Bach wrote:
    Hi there,

    You need to actually exclude the native devices in the scan exclude variable, and use the pseudo-device instead (scan order variable). A great summary can be found here:

    Configuring Oracle ASMLib on Multipath Disks on Linux (Doc ID 394956.1)

    HTH,
    Martin
    On 07/04/2011 16:11, Harel Safra wrote:

    Make sure to exclude /Dec/sd* - if you use them you won't be utilizing
    powerpath

    Harel Safra
    Sent from my phone.
    --
    http://www.freelists.org/webpage/oracle-l
  • Martin Bach at Apr 8, 2011 at 8:20 am
    Hi Dimitre,

    As you're live with production such a change shouldn't be made without proper testing-is your UAT environment a suitable candidate for this? You might create situation where not only the new disks aren't visible, but worse: the old ones won't be either creating the "perfect outage". I usually do such changes in DR first, or alternatively ensure that DR can be activated and stakeholders are aware of the risk. You'd then hope that you followed change management correctly and kept production and DR in sync .... But that keeps the job interesting!

    Also, out of curiosity, what's the output of v$asm_disk.library - does it show system or asmlib?

    Best regards,

    Martin

    Martin Bach
    Martin Bach Consulting Ltd
    http://www.linkedin.com/in/martincarstenbach
    http://martincarstenbach.wordpress.com
    -----Original Message-----
    From: oracle-l-bounce_at_freelists.org
    ....
  • Radoulov, Dimitre at Apr 8, 2011 at 8:42 am
    Hi Martin,
    thanks again for the valuable suggestions!

    No, there's neither UAT nor DR environment for this database ...

    Assuming no changes to the oracleasm configuration,
    could the following operations still break/have an impact on the live
    database?

    Add new disks to the servers.
    Create disks using the oracleasm createdisk command (using the first
    path found, in this case one of the two physical paths).
    Create new diskgroup using the new disks.

    I don't know if running scandisks, in order to make the newly created
    disks visible to the other nodes,
    could have a negative impact on the rest of the system.

    I'm only looking for information and suggestions,
    I realize these operations could be destructive and most probably we
    will call Oracle for this.

    The library shows as:

    ASM Library - Generic Linux, version 2.0.4 (KABI_V2)

    Best regards
    Dimitre
    On 08/04/2011 10:20, Martin Bach wrote:
    Hi Dimitre,

    As you're live with production such a change shouldn't be made without proper testing-is your UAT environment a suitable candidate for this? You might create situation where not only the new disks aren't visible, but worse: the old ones won't be either creating the "perfect outage". I usually do such changes in DR first, or alternatively ensure that DR can be activated and stakeholders are aware of the risk. You'd then hope that you followed change management correctly and kept production and DR in sync .... But that keeps the job interesting!

    Also, out of curiosity, what's the output of v$asm_disk.library - does it show system or asmlib?

    Best regards,

    Martin

    Martin Bach
    Martin Bach Consulting Ltd
    http://www.linkedin.com/in/martincarstenbach
    http://martincarstenbach.wordpress.com
    -----Original Message-----
    From: oracle-l-bounce_at_freelists.org
    ....
    --
    http://www.freelists.org/webpage/oracle-l
  • Radoulov, Dimitre at Apr 8, 2011 at 9:04 am
    One more question:

    oracleasm querydisk -p shows correctly all paths (physical and
    logical) to the devices.
    oracleasm querydisk -d shows the first one found during discovery

    How can we know which path was used originally, when the disk was created?

    Best regards
    Dimitre
    On 08/04/2011 10:42, Radoulov, Dimitre wrote:

    Hi Martin,
    thanks again for the valuable suggestions!

    No, there's neither UAT nor DR environment for this database ...

    Assuming no changes to the oracleasm configuration,
    could the following operations still break/have an impact on the live
    database?

    1. Add new disks to the servers.
    2. Create disks using the oracleasm createdisk command (using the
    first path found, in this case one of the two physical paths).
    3. Create new diskgroup using the new disks.

    I don't know if running scandisks, in order to make the newly created
    disks visible to the other nodes,
    could have a negative impact on the rest of the system.

    I'm only looking for information and suggestions,
    I realize these operations could be destructive and most probably we
    will call Oracle for this.


    The library shows as:

    ASM Library - Generic Linux, version 2.0.4 (KABI_V2)



    Best regards
    Dimitre

    On 08/04/2011 10:20, Martin Bach wrote:
    Hi Dimitre,

    As you're live with production such a change shouldn't be made
    without proper testing-is your UAT environment a suitable candidate
    for this? You might create situation where not only the new disks
    aren't visible, but worse: the old ones won't be either creating the
    "perfect outage". I usually do such changes in DR first, or
    alternatively ensure that DR can be activated and stakeholders are
    aware of the risk. You'd then hope that you followed change
    management correctly and kept production and DR in sync .... But that
    keeps the job interesting!

    Also, out of curiosity, what's the output of v$asm_disk.library -
    does it show system or asmlib?

    Best regards,

    Martin

    Martin Bach
    Martin Bach Consulting Ltd
    http://www.linkedin.com/in/martincarstenbach
    http://martincarstenbach.wordpress.com
    -----Original Message-----
    From: oracle-l-bounce_at_freelists.org
    ....
    --
    http://www.freelists.org/webpage/oracle-l
  • Andrew Kerber at Apr 8, 2011 at 8:22 pm
    I believe there are options with kfod to do some of this. The
    asm_diskstring parameter should show you how it is finding the disks
    currently. As I recall, if you try and change asm_diskstring, and current
    disks arent listed it wont let you.
    On Fri, Apr 8, 2011 at 4:04 AM, Radoulov, Dimitre wrote:


    One more question:

    - oracleasm querydisk -p shows correctly all paths (physical and logical)
    to the devices.
    - oracleasm querydisk -d shows the first one found during discovery

    How can we know which path was used originally, when the disk was created?



    Best regards
    Dimitre


    On 08/04/2011 10:42, Radoulov, Dimitre wrote:


    Hi Martin,
    thanks again for the valuable suggestions!

    No, there's neither UAT nor DR environment for this database ...

    Assuming no changes to the oracleasm configuration,
    could the following operations still break/have an impact on the live
    database?

    1. Add new disks to the servers.
    2. Create disks using the oracleasm createdisk command (using the first
    path found, in this case one of the two physical paths).
    3. Create new diskgroup using the new disks.

    I don't know if running scandisks, in order to make the newly created
    disks visible to the other nodes,
    could have a negative impact on the rest of the system.

    I'm only looking for information and suggestions,
    I realize these operations could be destructive and most probably we will
    call Oracle for this.


    The library shows as:

    ASM Library - Generic Linux, version 2.0.4 (KABI_V2)



    Best regards
    Dimitre

    On 08/04/2011 10:20, Martin Bach wrote:

    Hi Dimitre,

    As you're live with production such a change shouldn't be made without
    proper testing-is your UAT environment a suitable candidate for this? You
    might create situation where not only the new disks aren't visible, but
    worse: the old ones won't be either creating the "perfect outage". I usually
    do such changes in DR first, or alternatively ensure that DR can be
    activated and stakeholders are aware of the risk. You'd then hope that you
    followed change management correctly and kept production and DR in sync ....
    But that keeps the job interesting!

    Also, out of curiosity, what's the output of v$asm_disk.library - does it
    show system or asmlib?

    Best regards,

    Martin

    Martin Bach
    Martin Bach Consulting Ltd
    http://www.linkedin.com/in/martincarstenbach
    http://martincarstenbach.wordpress.com
    -----Original Message-----
    From: oracle-l-bounce_at_freelists.org [mailto:
    oracle-l-bounce@freelists.org]
    ....

    --
    http://www.freelists.org/webpage/oracle-l

    --
    Andrew W. Kerber

    'If at first you dont succeed, dont take up skydiving.'

    --
    http://www.freelists.org/webpage/oracle-l
  • Radoulov, Dimitre at Apr 9, 2011 at 6:47 am
    Hi Andrew,
    thanks! I didn't know about kfod, I'll check it's output on Monday (kfod
    disk=all -> path).

    I don't have an access to the systems right now, but if I recall
    correctly, v$asm_disk.path shows as ORCL:
    (so most probably asm_diskstring is set to ORCL:* or similar, I'll
    check that on Monday too).

    Thank you!

    Dimitre
    On 08/04/2011 22:22, Andrew Kerber wrote:
    I believe there are options with kfod to do some of this. The
    asm_diskstring parameter should show you how it is finding the disks
    currently. As I recall, if you try and change asm_diskstring, and
    current disks arent listed it wont let you.

    On Fri, Apr 8, 2011 at 4:04 AM, Radoulov, Dimitre
    wrote:

    One more question:

    - oracleasm querydisk -p shows correctly all paths (physical and
    logical) to the devices.
    - oracleasm querydisk -d shows the first one found during discovery

    How can we know which path was used originally, when the disk was
    created?


    [...]
    --
    http://www.freelists.org/webpage/oracle-l
  • Anonymous at Apr 7, 2011 at 5:33 pm
    Are there any metrics captured in AWR regarding network utilization- like what is the transfer rates, etc specifically to Data Guard ? We are trying to determine how much bandwidth DG is seeing and using on a network link. Thanks in advance.
  • Karl Arao at Apr 7, 2011 at 5:59 pm
    I will have to correlate that with the following:

    sysmetric view (Network Traffic Volume Per Sec) - see the SQL below
    also have a look at the Redo MB/s
    SAR data about the network transfer (txbyt/s) - I'll use kSar on this

    I had a client before that keeps on pointing on the Data Guard being the
    bottleneck because we are configured with Maximum Availability. When we
    started gathering performance data (systematically) on their problematic
    runs and proved them that it's not the network but an SQL problem....
    everybody believed on the hard data and the network, storage, development,
    dba teams began to be cooperative on solving/tuning the issue.

    (BTW, the network transfer rate is the same on their bad and good runs.. we
    were able to prove it from the gathered data..)

    In the end, tuning the network (OS & DB) + adding an index on the SQL solved
    the problem. Even made it faster than their prior runs.

    set echo off verify off

    ttitle center 'AWR Network Workload Report' skip 2
    set pagesize 50000
    set linesize 250

    col minval format 999990.00 heading "Network|Minvalue|(mb)/s"
    col maxval format 999990.00 heading "Network|Maxvalue|(mb)/s"
    col average format 999990.00 heading
    "Network|Average|value|(mb)/s"
    col std_dev format 999990.00 heading

    "Network|Std_dev|value|(mb)/s"

    select snap_id id, TO_CHAR(end_time,'YY/MM/DD HH24:MI') tm, instance_number
    inst, metric_name, minval/1024/1024 minval, maxval/1024/1024 maxval,
    average/1024/1024 average, standard_deviation/1024/1024 std_dev
    from dba_hist_sysmetric_summary
    where metric_name = 'Network Traffic Volume Per Sec'
    -- and snap_id in (338,339)
    /

    --
    Karl Arao
    karlarao.wordpress.com
    karlarao.tiddlyspot.com

    --
    http://www.freelists.org/webpage/oracle-l
  • Iotzov, Iordan at Apr 7, 2011 at 6:22 pm
    If you use OEM (Oracle Enterprise Manager) on all servers in your DataGaurd configuration, you might benefit from the performance information that is automatically collected by each OEM agent. The “Network Interfaces” group of metrics (things like Network Interface Read (MB/s) and Network Interface Read Utilization (%) ) might be of particular interest to you.

    Note that you can access the metrics information not only using the OEM GUI, but also directly from the respective MGMT$ tables/views in the OEM repository (an Oracle DB that stores all metrics and various other information).

    From: oracle-l-bounce_at_freelists.org On Behalf Of CRISLER, JON A (ATTCORP)
    Sent: Thursday, April 07, 2011 1:33 PM
    To: ORACLE-L
    Subject: Data Guard and network untilization

    Are there any metrics captured in AWR regarding network utilization- like what is the transfer rates, etc specifically to Data Guard ? We are trying to determine how much bandwidth DG is seeing and using on a network link. Thanks in advance.

    This message and its attachments may contain legally privileged or confidential information. It is intended solely for the named addressee. If you are not the addressee indicated in this message (or responsible for delivery of the message to the addressee), you may not copy or deliver this message or its attachments to anyone. Rather, you should permanently delete this message and its attachments and kindly notify the sender by reply e-mail. Any content of this message and its attachments that does not relate to the official business of News America Incorporated or its subsidiaries must be taken not to have been sent or endorsed by any of them. No warranty is made that the e-mail or attachment(s) are free from computer virus or other defect.
  • Pawel Smolarz at Apr 8, 2011 at 6:29 am
    Hi,
    Here is a great document about DG:
    http://www.oracle.com/technetwork/database/features/availability/maa-wp-10gr2-dataguardnetworkbestpr-134557.pdf

    Accurately described network configuration..

    Regards,
    Pawel

    W dniu 07.04.2011 19:33, CRISLER, JON A (ATTCORP) pisze:
    Are there any metrics captured in AWR regarding network utilization- like what is the transfer rates, etc specifically to Data Guard ? We are trying to determine how much bandwidth DG is seeing and using on a network link. Thanks in advance.

    Nordea Bank Polska S.A. z siedzibÄ… w Gdyni, ul. Kielecka 2, 81-303 Gdynia,
    wpisaną do Rejestru Przedsiębiorców Krajowego Rejestru Sądowego pod numerem: 0000021828,
    dla której dokumentację przechowuje Sąd Rejonowy Gdańsk - Północ w Gdańsku,
    VIII Wydział Gospodarczy Krajowego Rejestru Sądowego,
    o kapitale zakładowym i wpłaconym w wysokości: 277.493.500,00 złotych,
    NIP: 586-000-78-20, REGON: 190024711
  • Karl Arao at Apr 8, 2011 at 6:46 am
    In addition to that...

    �*Batch Processing in Disaster Recovery Configurations - Best Practices for
    Oracle Data Guard*� (http://goo.gl/hHhK)

    they did a really nice test case on that paper..

    --
    Karl Arao
    karlarao.wordpress.com
    karlarao.tiddlyspot.com

    --
    http://www.freelists.org/webpage/oracle-l
  • Anonymous at Apr 9, 2011 at 8:20 am
    For every disk discovered, the first block is reviewed and the
    ASMLIB label (ORCLDISK) is verified.
    If found, then the block device is created under the
    special directory /dev/oracleasm/disks.
    During the disk discovery, ASMLIB uses file /proc/partitions.

    If you are using EMC powerpath, you should explicitly set in
    ORACLEASM_SCANORDER="emcpower" and ORACLEASM_SCANEXCLUDE="sd" in
    /etc/sysconfig/oracleasm.

    You can use blkid utility on Linux to determine which disk you used
    for ASM, the result will tell you clearly which 2 disks are bind into
    1 emcpower device, and which emcpower device you can used in ASM to
    create diskgroups.
    # blkid | grep asm
    /dev/sdb1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdd1: LABEL="VOL2" TYPE="oracleasm"
    /dev/sde1: LABEL="VOL3" TYPE="oracleasm"
    /dev/sdg1: LABEL="VOL4" TYPE="oracleasm"
    /dev/sdo1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdq1: LABEL="VOL2" TYPE="oracleasm"
    /dev/sdr1: LABEL="VOL3" TYPE="oracleasm"
    /dev/sdt1: LABEL="VOL4" TYPE="oracleasm"
    /dev/emcpowerf1: LABEL="VOL4" TYPE="oracleasm"
    /dev/emcpowerp1: LABEL="VOL3" TYPE="oracleasm"
    /dev/emcpowero1: LABEL="VOL2" TYPE="oracleasm"
    /dev/emcpowern1: LABEL="VOL1" TYPE="oracleasm"

    You can use "oracleasm scandisks" in a live production environment,
    typically it's safe.

    There is no need to set "asm_diskstring" manually in most cases.

    --
    Kamus

    Visit my blog for more : http://www.dbform.com
    Join ACOUG: http://www.acoug.org
    On Sat, Apr 9, 2011 at 2:47 PM, Radoulov, Dimitre wrote:
    Hi Andrew,
    thanks! I didn't know about kfod, I'll check it's output on Monday (kfod
    disk=all -> path).

    I don't have an access to the systems right now, but if I recall correctly,
    v$asm_disk.path shows as ORCL:
    (so most probably asm_diskstring is set to ORCL:* or similar, I'll check
    that on Monday too).



    Thank you!

    Dimitre



    On 08/04/2011 22:22, Andrew Kerber wrote:

    I believe there are options with kfod to do some of this.  The
    asm_diskstring parameter should show you how it is finding the disks
    currently.  As I recall, if you try and change asm_diskstring, and current
    disks arent listed it wont let you.

    On Fri, Apr 8, 2011 at 4:04 AM, Radoulov, Dimitre
    wrote:
    One more question:

    - oracleasm querydisk -p shows correctly all paths (physical and logical)
    to the devices.
    - oracleasm querydisk -d shows the first one found during discovery

    How can we know which path was used originally, when the disk was created?


    [...]
    --
    http://www.freelists.org/webpage/oracle-l
  • Radoulov, Dimitre at Apr 9, 2011 at 7:50 pm
    Hi Leyi,
    this is an existing live production environment with EMC Powerpath in
    place,
    but ORACLEASM_SCANORDER and ORACLEASM_SCANEXCLUDE are not set.

    I know which emcpower devices should be used,
    but I want to know which ones was originally used with the
    createdisk command and which ones are currently in use.

    I think that at this point we should:

    Find the correct way to add disks to asm, considering the existing
    configuration
    (no ORACLEASM_SCANORDER or/and ORACLEASM_SCANEXCLUDE).

    Setup a test environment and try to fix the paths (use the emcpower
    devices,
    instead of the physical ones) there, before doing it in production.

    Best regards
    Dimitre
    On 09/04/2011 10:20, Leyi Zhang (Kamus) wrote:
    1. For every disk discovered, the first block is reviewed and the
    ASMLIB label (ORCLDISK) is verified.
    If found, then the block device is created under the
    special directory /dev/oracleasm/disks.
    During the disk discovery, ASMLIB uses file /proc/partitions.

    2. If you are using EMC powerpath, you should explicitly set in
    ORACLEASM_SCANORDER="emcpower" and ORACLEASM_SCANEXCLUDE="sd" in
    /etc/sysconfig/oracleasm.

    3. You can use blkid utility on Linux to determine which disk you used
    for ASM, the result will tell you clearly which 2 disks are bind into
    1 emcpower device, and which emcpower device you can used in ASM to
    create diskgroups.
    # blkid | grep asm
    /dev/sdb1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdd1: LABEL="VOL2" TYPE="oracleasm"
    /dev/sde1: LABEL="VOL3" TYPE="oracleasm"
    /dev/sdg1: LABEL="VOL4" TYPE="oracleasm"
    /dev/sdo1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdq1: LABEL="VOL2" TYPE="oracleasm"
    /dev/sdr1: LABEL="VOL3" TYPE="oracleasm"
    /dev/sdt1: LABEL="VOL4" TYPE="oracleasm"
    /dev/emcpowerf1: LABEL="VOL4" TYPE="oracleasm"
    /dev/emcpowerp1: LABEL="VOL3" TYPE="oracleasm"
    /dev/emcpowero1: LABEL="VOL2" TYPE="oracleasm"
    /dev/emcpowern1: LABEL="VOL1" TYPE="oracleasm"

    4. You can use "oracleasm scandisks" in a live production environment,
    typically it's safe.

    5. There is no need to set "asm_diskstring" manually in most cases.

    --
    Kamus

    Visit my blog for more : http://www.dbform.com
    Join ACOUG: http://www.acoug.org


    On Sat, Apr 9, 2011 at 2:47 PM, Radoulov, Dimitre wrote:
    Hi Andrew,
    thanks! I didn't know about kfod, I'll check it's output on Monday (kfod
    disk=all -> path).

    I don't have an access to the systems right now, but if I recall correctly,
    v$asm_disk.path shows as ORCL:
    (so most probably asm_diskstring is set to ORCL:* or similar, I'll check
    that on Monday too).



    Thank you!

    Dimitre



    On 08/04/2011 22:22, Andrew Kerber wrote:

    I believe there are options with kfod to do some of this. The
    asm_diskstring parameter should show you how it is finding the disks
    currently. As I recall, if you try and change asm_diskstring, and current
    disks arent listed it wont let you.

    On Fri, Apr 8, 2011 at 4:04 AM, Radoulov, Dimitre
    wrote:
    One more question:

    - oracleasm querydisk -p shows correctly all paths (physical and logical)
    to the devices.
    - oracleasm querydisk -d shows the first one found during discovery

    How can we know which path was used originally, when the disk was created?


    [...]
    --
    http://www.freelists.org/webpage/oracle-l
  • Andrew Kerber at Apr 9, 2011 at 8:01 pm
    Fyi, it is possible to list specific disks in asm_disktring, eg
    asm_diskstring='/dev/rdsk/emcpower1','/dev/rdsk/emcpower2','/dev/rdsk/emcpower3'

    I think you could add a wildcard on the end of that.
    On Sat, Apr 9, 2011 at 2:50 PM, Radoulov, Dimitre wrote:


    Hi Leyi,
    this is an existing live production environment with EMC Powerpath in
    place,
    but ORACLEASM_SCANORDER and ORACLEASM_SCANEXCLUDE are not set.

    I know which emcpower devices should be used,
    but I want to know which ones was originally used with the
    createdisk command and which ones are currently in use.

    I think that at this point we should:

    1. Find the correct way to add disks to asm, considering the existing
    configuration
    (no ORACLEASM_SCANORDER or/and ORACLEASM_SCANEXCLUDE).

    2. Setup a test environment and try to fix the paths (use the emcpower
    devices,
    instead of the physical ones) there, before doing it in production.



    Best regards
    Dimitre


    On 09/04/2011 10:20, Leyi Zhang (Kamus) wrote:

    1. For every disk discovered, the first block is reviewed and the
    ASMLIB label (ORCLDISK) is verified.
    If found, then the block device is created under the
    special directory /dev/oracleasm/disks.
    During the disk discovery, ASMLIB uses file /proc/partitions.

    2. If you are using EMC powerpath, you should explicitly set in
    ORACLEASM_SCANORDER="emcpower" and ORACLEASM_SCANEXCLUDE="sd" in
    /etc/sysconfig/oracleasm.

    3. You can use blkid utility on Linux to determine which disk you used
    for ASM, the result will tell you clearly which 2 disks are bind into
    1 emcpower device, and which emcpower device you can used in ASM to
    create diskgroups.
    # blkid | grep asm
    /dev/sdb1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdd1: LABEL="VOL2" TYPE="oracleasm"
    /dev/sde1: LABEL="VOL3" TYPE="oracleasm"
    /dev/sdg1: LABEL="VOL4" TYPE="oracleasm"
    /dev/sdo1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdq1: LABEL="VOL2" TYPE="oracleasm"
    /dev/sdr1: LABEL="VOL3" TYPE="oracleasm"
    /dev/sdt1: LABEL="VOL4" TYPE="oracleasm"
    /dev/emcpowerf1: LABEL="VOL4" TYPE="oracleasm"
    /dev/emcpowerp1: LABEL="VOL3" TYPE="oracleasm"
    /dev/emcpowero1: LABEL="VOL2" TYPE="oracleasm"
    /dev/emcpowern1: LABEL="VOL1" TYPE="oracleasm"

    4. You can use "oracleasm scandisks" in a live production environment,
    typically it's safe.

    5. There is no need to set "asm_diskstring" manually in most cases.

    --
    Kamus

    Visit my blog for more : http://www.dbform.com
    Join ACOUG: http://www.acoug.org



    On Sat, Apr 9, 2011 at 2:47 PM, Radoulov, Dimitre
    wrote:
    Hi Andrew,
    thanks! I didn't know about kfod, I'll check it's output on Monday (kfod
    disk=all -> path).

    I don't have an access to the systems right now, but if I recall
    correctly,
    v$asm_disk.path shows as ORCL:
    (so most probably asm_diskstring is set to ORCL:* or similar, I'll check
    that on Monday too).



    Thank you!

    Dimitre



    On 08/04/2011 22:22, Andrew Kerber wrote:

    I believe there are options with kfod to do some of this. The
    asm_diskstring parameter should show you how it is finding the disks
    currently. As I recall, if you try and change asm_diskstring, and
    current
    disks arent listed it wont let you.

    On Fri, Apr 8, 2011 at 4:04 AM, Radoulov, Dimitre
    wrote:
    One more question:

    - oracleasm querydisk -p shows correctly all paths (physical and
    logical)
    to the devices.
    - oracleasm querydisk -d shows the first one found during discovery

    How can we know which path was used originally, when the disk was
    created?


    [...]
    --
    http://www.freelists.org/webpage/oracle-l

    --
    Andrew W. Kerber

    'If at first you dont succeed, dont take up skydiving.'

    --
    http://www.freelists.org/webpage/oracle-l
  • Radoulov, Dimitre at Apr 10, 2011 at 9:13 am
    Hi Leyi,
    answers inline.
    What my point is, since you can get this result:
    # /etc/init.d/oracleasm querydisk -p BLOB
    Disk "BLOB" is a valid ASM disk
    /dev/sdam1: LABEL="BLOB" TYPE="oracleasm"
    /dev/sdbc1: LABEL="BLOB" TYPE="oracleasm"
    /dev/emcpowerz1: LABEL="BLOB" TYPE="oracleasm"
    Then it means emcpower device was used when creatediesk. Actually, no
    matter what device name was used when createdisk, at least now your
    "/dev/emcpowerz1" is recognized as ASM Disk BLOB. it's enough (If I'm
    wrong please correct me).
    Actually, that's what my original email is all about.

    I see that the emcpower devices are recognized as asm disks,
    I don't know if this is enough though ...

    I suppose that querydisk just reports all devices/paths that point
    to the physical disk. What I want to know is if asm is smart
    enough to pick the emcpower device in this case or it
    just uses the first one found (in ASCIIbetical order).

    querydisk -d returns the major/minor of one of the physical paths
    to the disk, not the emcpower path (I checked /proc/partitions for that),
    that's why I think it actually doesn't use the logical devices.
    I say "I think", because I'm not finding this anywhere in the docs for now.
    Query disk typically is safe, you can modify "ORACLEASM_SCANORDER" and
    "ORACLEASM_SCANEXCLUDE" when you get some downtime in database
    maintenance window then restart ASM, if disk scanned is not an ASM
    disk, will just simply raise an error, no data broken. (Of course,
    backup your database first)
    Thanks for the suggestion!
    Yes, most likely we'll try this first.
    BTW, can you post /proc/partitions content?
    Yes,
    I'm attaching the content of /proc/partitions.

    Thank you!

    Best regards
    Dimitre

    --
    http://www.freelists.org/webpage/oracle-l

    text/plain attachment: proc_partition.txt
  • Radoulov, Dimitre at Apr 10, 2011 at 9:50 am
    So the first question would be:

    what would happen if the path reported by querydisk -d for a given disk
    is not available?

    Would we get an error? Would asm switch to some of the other available
    paths to the same disk automatically?

    If asmlib could handle this, the choice of the path during disk creation
    should be irrelevant.

    Best regards
    Dimitre
    On 10/04/2011 11:13, Radoulov, Dimitre wrote:

    Hi Leyi,
    answers inline.
    What my point is, since you can get this result:
    # /etc/init.d/oracleasm querydisk -p BLOB
    Disk "BLOB" is a valid ASM disk
    /dev/sdam1: LABEL="BLOB" TYPE="oracleasm"
    /dev/sdbc1: LABEL="BLOB" TYPE="oracleasm"
    /dev/emcpowerz1: LABEL="BLOB" TYPE="oracleasm"
    Then it means emcpower device was used when creatediesk. Actually, no
    matter what device name was used when createdisk, at least now your
    "/dev/emcpowerz1" is recognized as ASM Disk BLOB. it's enough (If I'm
    wrong please correct me).
    Actually, that's what my original email is all about.

    I see that the emcpower devices are recognized as asm disks,
    I don't know if this is enough though ...

    I suppose that querydisk just reports all devices/paths that point
    to the physical disk. What I want to know is if asm is smart
    enough to pick the emcpower device in this case or it
    just uses the first one found (in ASCIIbetical order).

    querydisk -d returns the major/minor of one of the physical paths
    to the disk, not the emcpower path (I checked /proc/partitions for that),
    that's why I think it actually doesn't use the logical devices.
    I say "I think", because I'm not finding this anywhere in the docs for
    now.
    Query disk typically is safe, you can modify "ORACLEASM_SCANORDER" and
    "ORACLEASM_SCANEXCLUDE" when you get some downtime in database
    maintenance window then restart ASM, if disk scanned is not an ASM
    disk, will just simply raise an error, no data broken. (Of course,
    backup your database first)
    Thanks for the suggestion!
    Yes, most likely we'll try this first.
    BTW, can you post /proc/partitions content?
    Yes,
    I'm attaching the content of /proc/partitions.



    Thank you!


    Best regards
    Dimitre
    --
    http://www.freelists.org/webpage/oracle-l
  • Radoulov, Dimitre at Apr 10, 2011 at 10:12 am
    OK,
    as in the MOS note Martin suggested and the others have already mentioned,
    in order for asmlib to use multipathing we need to set scanexclude
    or/and scanorder. So even querydisk -p reports all available paths
    to a given disk, it uses the first one it finds.

    If we add new disks using the correct path (emcpower...)
    and we don't set the above mentioned asm configuration variables,
    after scanning the disks on the other node, asm will pick the first
    path found, not the one we used with the createdisk command.
    I don't know if this can cause problems.

    So I would first add disks (because this is urgent)
    using the first devices in ASCIIbetical order and asm will/should
    pick the same devices on both nodes (at least I hope so).

    After that, when downtime is possible (or better yet, if a test environment
    could be build) I'll try to fix the paths (use the emcpower paths).

    Best regards
    Dimitre
    On 10/04/2011 11:50, Radoulov, Dimitre wrote:

    So the first question would be:

    what would happen if the path reported by querydisk -d for a given
    disk is not available?

    Would we get an error? Would asm switch to some of the other available
    paths to the same disk automatically?

    If asmlib could handle this, the choice of the path during disk
    creation should be irrelevant.
    [...]
    On 10/04/2011 11:13, Radoulov, Dimitre wrote:

    Hi Leyi,
    answers inline.
    What my point is, since you can get this result:
    # /etc/init.d/oracleasm querydisk -p BLOB
    Disk "BLOB" is a valid ASM disk
    /dev/sdam1: LABEL="BLOB" TYPE="oracleasm"
    /dev/sdbc1: LABEL="BLOB" TYPE="oracleasm"
    /dev/emcpowerz1: LABEL="BLOB" TYPE="oracleasm"
    Then it means emcpower device was used when creatediesk. Actually, no
    matter what device name was used when createdisk, at least now your
    "/dev/emcpowerz1" is recognized as ASM Disk BLOB. it's enough (If I'm
    wrong please correct me).
    Actually, that's what my original email is all about.

    I see that the emcpower devices are recognized as asm disks,
    I don't know if this is enough though ...

    I suppose that querydisk just reports all devices/paths that point
    to the physical disk. What I want to know is if asm is smart
    enough to pick the emcpower device in this case or it
    just uses the first one found (in ASCIIbetical order).

    querydisk -d returns the major/minor of one of the physical paths
    to the disk, not the emcpower path (I checked /proc/partitions for
    that),
    that's why I think it actually doesn't use the logical devices.
    I say "I think", because I'm not finding this anywhere in the docs
    for now.
    Query disk typically is safe, you can modify "ORACLEASM_SCANORDER" and
    "ORACLEASM_SCANEXCLUDE" when you get some downtime in database
    maintenance window then restart ASM, if disk scanned is not an ASM
    disk, will just simply raise an error, no data broken. (Of course,
    backup your database first)
    Thanks for the suggestion!
    Yes, most likely we'll try this first.
    [...]
  • Goran bogdanovic at Apr 20, 2011 at 11:17 am

    querydisk -d returns the major/minor of one of the physical paths
    to the disk, not the emcpower path (I checked /proc/partitions for that),
    that's why I think it actually doesn't use the logical devices.
    I say "I think", because I'm not finding this anywhere in the docs for now.
    in my case:

    oracle_at_momos_at_+ASM: ~
    -bash-3.2 $ oracleasm querydisk -p vol1
    Disk "VOL1" is a valid ASM disk
    /dev/emcpowera1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdx1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdm1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdb1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdai1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdat1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdbe1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdbp1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdca1: LABEL="VOL1" TYPE="oracleasm"

    oracle_at_momos_at_+ASM: ~
    -bash-3.2 $ oracleasm querydisk -d vol1
    Disk "VOL1" is a valid ASM disk on device [120, 1]

    -bash-3.2 $ cat /proc/partitions |grep '120'
    120 48 281018368 emcpowerd
    120 49 281016981 emcpowerd1
    120 64 41943040 emcpowere
    120 65 41943024 emcpowere1

    120 0 281018368 emcpowera
    120 1 281001984 emcpowera1 <==

    120 16 281018368 emcpowerb
    120 17 281001984 emcpowerb1
    120 32 281018368 emcpowerc
    120 33 281001984 emcpowerc1
    120 144 281018368 emcpowerj
    120 145 281001984 emcpowerj1
    120 160 281018368 emcpowerk
    120 161 281001984 emcpowerk1
    120 80 281018368 emcpowerf
    120 81 281001984 emcpowerf1
    120 128 281018368 emcpoweri
    120 129 281001984 emcpoweri1
    120 112 281018368 emcpowerh
    120 113 281001984 emcpowerh1
    120 96 281018368 emcpowerg
    120 97 281001984 emcpowerg1

    regards,
    goran
  • Radoulov, Dimitre at Apr 20, 2011 at 11:27 am
    Thanks Goran,
    yes, in your case querydisk -d reports the pseudo-device (emcpowerXn).

    It seems that, at least with kernel 2.6, either path (emcpower or sd)
    could be used
    (see www.freelists.org/post/oracle-l/ASM-and-EMC-PowerPath,24).

    Just curious,
    did you configure ORACLEASM_SCANORDER in /etc/sysconfig/oracleasm?

    Best regards
    Dimitre
    On 20/04/2011 13:17, goran bogdanovic wrote:


    querydisk -d returns the major/minor of one of the physical paths
    to the disk, not the emcpower path (I checked /proc/partitions for
    that),
    that's why I think it actually doesn't use the logical devices.
    I say "I think", because I'm not finding this anywhere in the docs
    for now.


    in my case:

    oracle_at_momos_at_+ASM: ~
    -bash-3.2 $ oracleasm querydisk -p vol1
    Disk "VOL1" is a valid ASM disk
    /dev/emcpowera1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdx1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdm1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdb1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdai1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdat1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdbe1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdbp1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdca1: LABEL="VOL1" TYPE="oracleasm"

    oracle_at_momos_at_+ASM: ~
    -bash-3.2 $ oracleasm querydisk -d vol1
    Disk "VOL1" is a valid ASM disk on device [120, 1]

    -bash-3.2 $ cat /proc/partitions |grep '120'
    120 48 281018368 emcpowerd
    120 49 281016981 emcpowerd1
    120 64 41943040 emcpowere
    120 65 41943024 emcpowere1
    120 0 281018368 emcpowera
    120 1 281001984 emcpowera1 <==
    120 16 281018368 emcpowerb
    120 17 281001984 emcpowerb1
    120 32 281018368 emcpowerc
    120 33 281001984 emcpowerc1
    120 144 281018368 emcpowerj
    120 145 281001984 emcpowerj1
    120 160 281018368 emcpowerk
    120 161 281001984 emcpowerk1
    120 80 281018368 emcpowerf
    120 81 281001984 emcpowerf1
    120 128 281018368 emcpoweri
    120 129 281001984 emcpoweri1
    120 112 281018368 emcpowerh
    120 113 281001984 emcpowerh1
    120 96 281018368 emcpowerg
    120 97 281001984 emcpowerg1

    regards,
    goran
    --
    http://www.freelists.org/webpage/oracle-l
  • Goran bogdanovic at Apr 20, 2011 at 11:58 am
    ... as per note;-)

    -bash-3.2 $ cat /etc/sysconfig/oracleasm | egrep -i 'scano|exc'
    # ORACLEASM_SCANORDER: Matching patterns to order disk scanning
    ORACLEASM_SCANORDER="emcpower"
    # ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan
    ORACLEASM_SCANEXCLUDE="sd dm"

    regards,
    goran

    2011/4/20 Radoulov, Dimitre
    Thanks Goran,
    yes, in your case querydisk -d reports the pseudo-device (emcpowerXn).

    It seems that, at least with kernel 2.6, either path (emcpower or sd) could
    be used
    (see www.freelists.org/post/oracle-l/ASM-and-EMC-PowerPath,24).

    Just curious,
    did you configure ORACLEASM_SCANORDER in /etc/sysconfig/oracleasm?



    Best regards
    Dimitre



    On 20/04/2011 13:17, goran bogdanovic wrote:


    querydisk -d returns the major/minor of one of the physical paths
    to the disk, not the emcpower path (I checked /proc/partitions for that),
    that's why I think it actually doesn't use the logical devices.
    I say "I think", because I'm not finding this anywhere in the docs for
    now.
    in my case:

    oracle_at_momos_at_+ASM: ~
    -bash-3.2 $ oracleasm querydisk -p vol1
    Disk "VOL1" is a valid ASM disk
    /dev/emcpowera1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdx1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdm1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdb1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdai1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdat1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdbe1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdbp1: LABEL="VOL1" TYPE="oracleasm"
    /dev/sdca1: LABEL="VOL1" TYPE="oracleasm"

    oracle_at_momos_at_+ASM: ~
    -bash-3.2 $ oracleasm querydisk -d vol1
    Disk "VOL1" is a valid ASM disk on device [120, 1]

    -bash-3.2 $ cat /proc/partitions |grep '120'
    120 48 281018368 emcpowerd
    120 49 281016981 emcpowerd1
    120 64 41943040 emcpowere
    120 65 41943024 emcpowere1
    120 0 281018368 emcpowera
    120 1 281001984 emcpowera1 <==
    120 16 281018368 emcpowerb
    120 17 281001984 emcpowerb1
    120 32 281018368 emcpowerc
    120 33 281001984 emcpowerc1
    120 144 281018368 emcpowerj
    120 145 281001984 emcpowerj1
    120 160 281018368 emcpowerk
    120 161 281001984 emcpowerk1
    120 80 281018368 emcpowerf
    120 81 281001984 emcpowerf1
    120 128 281018368 emcpoweri
    120 129 281001984 emcpoweri1
    120 112 281018368 emcpowerh
    120 113 281001984 emcpowerh1
    120 96 281018368 emcpowerg
    120 97 281001984 emcpowerg1

    regards,
    goran


    --
    http://www.freelists.org/webpage/oracle-l
  • Karl Arao at Apr 10, 2011 at 10:33 am
    Hi Dimitre,

    See my answers below:

    Question: which path should we use with the createdisk command?
    Answer: You should use the /dev/emcpowerz1, note that as per the official
    doc you need to specify the whole disk partition

    Question: Is the multipathing used, given querydisk -d returns the
    major/minor numbers of the phisical device?
    Answer: Kindly check on this wiki to know the mapping of ASM to EMC Power
    Devices http://karlarao.tiddlyspot.com/index.html#ASMtoEMCPowerDevices
    This will show you a storage report that maps the relationship from the
    Oracle layer (ASM), Linux, and SAN storage. This info is very useful for you
    and the storage engineer. So you would know which is which in case of
    catastrophic problems..

    Question: After adding and marking the disks for asm from one node, we'll
    need to run oracleasm scandisks on the remaining nodes,
    is it safe to run this command (scandisks) in a live production environment?
    Answer:
    - First you have to make a whole disk partition on that shared EMC Power
    Device on the 1st node, after doing that you have to issue "partprobe" (as
    root) on the remaining nodes
    - Second, you'll execute the "createdisk" on the 1st node... after that, you
    have to issue "oracleasm scandisks" on the remaining nodes
    - Third, you should verify on all the nodes by executing "oracleasm
    listdisks"

    Is it safe to do this? mmm.. I would allocate a window for doing this
    activity just to be safe.. or do it first on test environment or R&D
    machines..

    --
    Karl Arao
    karlarao.wordpress.com
    karlarao.tiddlyspot.com

    --
    http://www.freelists.org/webpage/oracle-l
  • Radoulov, Dimitre at Apr 10, 2011 at 12:31 pm
    Hi Karl,
    thanks for the information (and thanks for adding partprobe to the list).

    The report you mention reports DBSO1DATA1 with major/minor 8, 33 (ls -l
    /dev/oracleasm/disks).
    This is the device sdc1 and emcpowerp is the logical path (the EMC Power
    path) to that device (if I understanding it right).
    But emcpowerp has different major/minor numbers, so I would expect to
    see its major/minor numbers,
    if the disk is accessed through the emcpower path.

    Of course, I could be totally wrong.

    Could you please check if on the RAC you produced the report on,
    ORACLEASM_SCANORDER or ORACLEASM_SCANEXCLUDE are set in
    /etc/sysconfig/oracleasm?

    Thanks
    Dimitre
    On 10/04/2011 12:33, Karl Arao wrote:

    Hi Dimitre,

    See my answers below:

    1) Question: which path should we use with the createdisk command?
    Answer: You should use the /dev/emcpowerz1, note that as per the
    official doc you need to specify the whole disk partition

    2) Question: Is the multipathing used, given querydisk -d returns the
    major/minor numbers of the phisical device?
    Answer: Kindly check on this wiki to know the mapping of ASM to EMC
    Power Devices
    http://karlarao.tiddlyspot.com/index.html#ASMtoEMCPowerDevices
    This will show you a storage report that maps the relationship from
    the Oracle layer (ASM), Linux, and SAN storage. This info is very
    useful for you and the storage engineer. So you would know which is
    which in case of catastrophic problems..

    3) Question: After adding and marking the disks for asm from one node,
    we'll need to run oracleasm scandisks on the remaining nodes,
    is it safe to run this command (scandisks) in a live production
    environment?
    Answer:
    - First you have to make a whole disk partition on that shared EMC
    Power Device on the 1st node, after doing that you have to
    issue "partprobe" (as root) on the remaining nodes
    - Second, you'll execute the "createdisk" on the 1st node... after
    that, you have to issue "oracleasm scandisks" on the remaining nodes
    - Third, you should verify on all the nodes by executing "oracleasm
    listdisks"

    Is it safe to do this? mmm.. I would allocate a window for doing this
    activity just to be safe.. or do it first on test environment or R&D
    machines..




    --
    Karl Arao
    karlarao.wordpress.com <http://karlarao.wordpress.com>
    karlarao.tiddlyspot.com <http://karlarao.tiddlyspot.com>
    --
    http://www.freelists.org/webpage/oracle-l
  • Karl Arao at Apr 10, 2011 at 1:19 pm
    Hi Dimitre,

    The major/minor of emcpowerp1 is 120 241...

    I would do this report every time there will be change on the storage side
    (let's say SAN Migration, powerpath upgrade)

    But take note, ASM is not really dependent on the major/minor numbers.. it
    is dependent on the ASM header information when you created that ASM disk.

    So let's say I'll be migrating to a new SAN storage.. I can do a SAN copy on
    a different set of LUNs on that new SAN and have totally new major/minor
    numbers but when you boot again the server with the new LUNs the ASM will
    still work as if nothing happened (no partprobe and scandisks needed). Cool
    right? :)

    BTW, we did no changes (default) on the ORACLEASM_SCANORDER and
    ORACLEASM_SCANEXCLUDE

    --
    Karl Arao
    karlarao.wordpress.com
    karlarao.tiddlyspot.com

    --
    http://www.freelists.org/webpage/oracle-l
  • Radoulov, Dimitre at Apr 10, 2011 at 5:47 pm
    Hi Karl,
    yes, I understand that asmlib will work with any valid path to the disks,
    but in order to actually use the multipath, you need to configure it
    correctly (check MOS Note 394956.1).

    You seem to have the default configuration (no scanorder and/or
    scanexclude),
    if on reboot (or after scandisk) asmlib chooses a path different than
    the multipath
    and the chosen path becomes unavailable, you won't be able to access
    those disks
    no more (you'll get errors).

    If you configure the access to the disks with asmlib (via scanorder or
    scanexclude)
    through the multipath, the disks will remain accessesible even if you
    loose one or more
    of the paths to those disks (assuming at least one path is still available).

    Best regards
    Dimitre
    On 10/04/2011 15:19, Karl Arao wrote:

    Hi Dimitre,

    The major/minor of emcpowerp1 is 120 241...

    I would do this report every time there will be change on the storage
    side (let's say SAN Migration, powerpath upgrade)

    But take note, ASM is not really dependent on the major/minor
    numbers.. it is dependent on the ASM header information when you
    created that ASM disk.

    So let's say I'll be migrating to a new SAN storage.. I can do a SAN
    copy on a different set of LUNs on that new SAN and have totally new
    major/minor numbers but when you boot again the server with the new
    LUNs the ASM will still work as if nothing happened (no partprobe and
    scandisks needed). Cool right? :)

    BTW, we did no changes (default) on the ORACLEASM_SCANORDER and
    ORACLEASM_SCANEXCLUDE

    --
    Karl Arao
    karlarao.wordpress.com <http://karlarao.wordpress.com>
    karlarao.tiddlyspot.com <http://karlarao.tiddlyspot.com>
    --
    http://www.freelists.org/webpage/oracle-l
  • Karl Arao at Apr 10, 2011 at 8:30 pm
    Hi Dimitre,

    See my answers below..

    but in order to actually use the multipath, you need to configure it
    correctly (check MOS Note 394956.1).

    ANSWER: Also see the following notes
    - Configuring Oracle ASMLib on Multipath Disks [ID 309815.1]
    "Unlike most multipath drivers, PowerPath will support full multipath
    capabilites even with the regular names. PowerPath and ASMLib now work
    together successfully."
    - ORA-15072 when creating a diskgroup with external redundancy [ID 396015.1]
    which pertains to the "Linux multipath" and not the "EMC Powerpath"

    2) if on reboot (or after scandisk) asmlib chooses a path different than the
    multipath
    and the chosen path becomes unavailable, you won't be able to access those
    disks
    no more (you'll get errors)

    ANSWER:

    - it will not choose a path other than the configured EMC Powerdevice if you
    have properly configured the persistent binding... so you must ensure that
    the storage engineer have properly configured the "multipathing" and
    "persistent binding"

    3) If you configure the access to the disks with asmlib (via scanorder or
    scanexclude)
    through the multipath, the disks will remain accessesible even if you loose
    one or more
    of the paths to those disks (assuming at least one path is still available).

    ANSWER:

    - again.. "Unlike most multipath drivers, PowerPath will support full
    multipath capabilites even with the regular names. PowerPath and ASMLib now
    work together successfully."

    The only error so far that I've seen on EMC multipathing is this.. which you
    can ignore as per powerlink
    http://karlarao.tiddlyspot.com/#[[Buffer%20IO%20error]]
    and the only time I have to exclude powerdevices is when I have to configure
    it for LVM ... http://karlarao.tiddlyspot.com/#EMCPowerDevicesOnLVM

    --
    Karl Arao
    karlarao.wordpress.com
    karlarao.tiddlyspot.com

    --
    http://www.freelists.org/webpage/oracle-l
  • Radoulov, Dimitre at Apr 11, 2011 at 8:03 am
    Karl,
    thank you for clarifying,
    "Unlike most multipath drivers, PowerPath will support full
    multipath capabilites
    even with the regular names.
    PowerPath and ASMLib now work together successfully."
    So it seems that we don't need to set any oracleasm variables with
    kernel 2.6
    (if I'm reading the note correctly) and multipathing will be used anyway,
    the path used (regular or multi) is irrelevant, as far as ASM Lib is
    concerned.

    So in our case, EMC Powerpath, no scanorder or scanexclude,
    we just need to configure the new disks with Powerpath
    and create them with oracleasm using one of the available paths
    (regular or multi, the choice is irrelevant).

    Is this right or I'm missing something again?

    Best regards
    Dimitre
    On 10/04/2011 22:30, Karl Arao wrote:
    Hi Dimitre,

    See my answers below..

    1) but in order to actually use the multipath, you need to configure
    it correctly (check MOS Note 394956.1).

    ANSWER: Also see the following notes
    - Configuring Oracle ASMLib on Multipath Disks [ID 309815.1]
    "Unlike most multipath drivers, PowerPath will support full
    multipath capabilites even with the regular names. PowerPath and
    ASMLib now work together successfully."
    - ORA-15072 when creating a diskgroup with external redundancy [ID
    396015.1] which pertains to the "Linux multipath" and not the "EMC
    Powerpath"


    2) if on reboot (or after scandisk) asmlib chooses a path different
    than the multipath
    and the chosen path becomes unavailable, you won't be able to access
    those disks
    no more (you'll get errors)

    ANSWER:
    - it will not choose a path other than the configured EMC
    Powerdevice if you have properly configured the persistent
    binding... so you must ensure that the storage engineer have
    properly configured the "multipathing" and "persistent binding"


    3) If you configure the access to the disks with asmlib (via scanorder
    or scanexclude)
    through the multipath, the disks will remain accessesible even if you
    loose one or more
    of the paths to those disks (assuming at least one path is still
    available).

    ANSWER:
    - again.. "Unlike most multipath drivers, PowerPath will support
    full multipath capabilites even with the regular names. PowerPath
    and ASMLib now work together successfully."



    The only error so far that I've seen on EMC multipathing is this..
    which you can ignore as per powerlink
    http://karlarao.tiddlyspot.com/#[[Buffer%20IO%20error]]
    and the only time I have to exclude powerdevices is when I have to
    configure it for LVM ...
    http://karlarao.tiddlyspot.com/#EMCPowerDevicesOnLVM


    --
    Karl Arao
    karlarao.wordpress.com <http://karlarao.wordpress.com>
    karlarao.tiddlyspot.com <http://karlarao.tiddlyspot.com>
    --
    http://www.freelists.org/webpage/oracle-l
  • Karl Arao at Apr 14, 2011 at 5:01 pm
    Hi Dimitre,

    I'm also curious about that.. especially on this

    "Unlike most multipath drivers, PowerPath will support full multipath
    capabilites even with the regular names. PowerPath and ASMLib now work
    together successfully."

    so I asked some friends from EMC. So we are still waiting for the EMC
    support for their answer.

    But for a really definitive answer.. if you have a test or r&d environment
    you can unplug the storage cable connection and see if the fail-over works
    even if there's no scanorder or scanexclude. And if this does not work, it
    will be just a simple configuration file change and restart of the nodes.

    --
    Karl Arao
    karlarao.wordpress.com
    karlarao.tiddlyspot.com

    --
    http://www.freelists.org/webpage/oracle-l
  • Radoulov, Dimitre at Apr 15, 2011 at 1:21 pm
    Hi Karl,
    thanks!

    Unfortunately I'm not able to run such a test, at least not for now.

    So please let me/us know, if/when you have updates on this from the EMC
    support.

    Best regards
    Dimitre
    On 14/04/2011 19:01, Karl Arao wrote:
    Hi Dimitre,

    I'm also curious about that.. especially on this

    "Unlike most multipath drivers, PowerPath will support full multipath
    capabilites even with the regular names. PowerPath and ASMLib now work
    together successfully."

    so I asked some friends from EMC. So we are still waiting for the EMC
    support for their answer.

    But for a really definitive answer.. if you have a test or r&d
    environment you can unplug the storage cable connection and see if
    the fail-over works even if there's no scanorder or scanexclude. And
    if this does not work, it will be just a simple configuration file
    change and restart of the nodes.


    --
    Karl Arao
    karlarao.wordpress.com <http://karlarao.wordpress.com>
    karlarao.tiddlyspot.com <http://karlarao.tiddlyspot.com>
    --
    http://www.freelists.org/webpage/oracle-l
  • Radoulov, Dimitre at Apr 10, 2011 at 5:47 pm
    Hi Karl,
    yes, I understand that asmlib will work with any valid path to the disks,
    but in order to actually use the multipath, you need to configure it
    correctly (check MOS Note 394956.1).

    You seem to have the default configuration (no scanorder and/or
    scanexclude),
    if on reboot (or after scandisk) asmlib chooses a path different than
    the multipath
    and the chosen path becomes unavailable, you won't be able to access
    those disks
    no more (you'll get errors).

    If you configure the access to the disks with asmlib (via scanorder or
    scanexclude)
    through the multipath, the disks will remain accessible even if you
    loose one or more
    of the paths to those disks (assuming at least one path is still available).

    Best regards
    Dimitre
    On 10/04/2011 15:19, Karl Arao wrote:

    Hi Dimitre,

    The major/minor of emcpowerp1 is 120 241...

    I would do this report every time there will be change on the storage
    side (let's say SAN Migration, powerpath upgrade)

    But take note, ASM is not really dependent on the major/minor
    numbers.. it is dependent on the ASM header information when you
    created that ASM disk.

    So let's say I'll be migrating to a new SAN storage.. I can do a SAN
    copy on a different set of LUNs on that new SAN and have totally new
    major/minor numbers but when you boot again the server with the new
    LUNs the ASM will still work as if nothing happened (no partprobe and
    scandisks needed). Cool right? :)

    BTW, we did no changes (default) on the ORACLEASM_SCANORDER and
    ORACLEASM_SCANEXCLUDE

    --
    Karl Arao
    karlarao.wordpress.com <http://karlarao.wordpress.com>
    karlarao.tiddlyspot.com <http://karlarao.tiddlyspot.com>
    --
    http://www.freelists.org/webpage/oracle-l
  • Radoulov, Dimitre at Apr 15, 2011 at 2:06 pm
    Thank you very much Matthew,
    I'm forwarding your email to the list.

    Best regards
    Dimitre
    On 15/04/2011 16:00, Zito, Matthew wrote:
    Hey guys,

    I haven't gotten this email address straightened out on Oracle-L yet, but I figured I'd drop you a note, and you could forward it on to the list if you cared to.

    The doc you read is correct, powerpath will cheerfully work with any of the devices you send IOs to, because the kernel driver intercepts requests for all devices and routes them through itself before dishing them down the appropriate path.

    However, setting scandisks to the emcpower has the administrative benefits of making sure the disks don't show up twice. However, even if ASM picks the first of the two disks, it will still be load-balanced successfully.

    Thanks,
    Matt Zito
    (former EMC solutions architect)
    ________________________________________
    From: oracle-l-bounce_at_freelists.org [oracle-l-bounce_at_freelists.org] On Behalf Of Radoulov, Dimitre [cichomitiko_at_gmail.com]
    Sent: Friday, April 15, 2011 9:21 AM
    To: Karl Arao
    Cc: ORACLE-L
    Subject: Re: ASM and EMC PowerPath

    Hi Karl,
    thanks!

    Unfortunately I'm not able to run such a test, at least not for now.

    So please let me/us know, if/when you have updates on this from the EMC support.


    Best regards
    Dimitre


    On 14/04/2011 19:01, Karl Arao wrote:
    Hi Dimitre,

    I'm also curious about that.. especially on this

    "Unlike most multipath drivers, PowerPath will support full multipath capabilites even with the regular names. PowerPath and ASMLib now work together successfully."

    so I asked some friends from EMC. So we are still waiting for the EMC support for their answer.

    But for a really definitive answer.. if you have a test or r&d environment you can unplug the storage cable connection and see if the fail-over works even if there's no scanorder or scanexclude. And if this does not work, it will be just a simple configuration file change and restart of the nodes.


    --
    Karl Arao
    karlarao.wordpress.com<http://karlarao.wordpress.com>
    karlarao.tiddlyspot.com<http://karlarao.tiddlyspot.com>
    --
    http://www.freelists.org/webpage/oracle-l
  • Karl Arao at Apr 20, 2011 at 2:16 pm
    Forwarding to this thread the reply from Matthew Zito.
    On Fri, Apr 15, 2011 at 10:00 PM, Zito, Matthew wrote:


    Hey guys,

    I haven't gotten this email address straightened out on Oracle-L yet, but I
    figured I'd drop you a note, and you could forward it on to the list if you
    cared to.

    The doc you read is correct, powerpath will cheerfully work with any of the
    devices you send IOs to, because the kernel driver intercepts requests for
    all devices and routes them through itself before dishing them down the
    appropriate path.

    However, setting scandisks to the emcpower has the administrative benefits
    of making sure the disks don't show up twice. However, even if ASM picks
    the first of the two disks, it will still be load-balanced successfully.

    Thanks,
    Matt Zito
    (former EMC solutions architect)
    ________________________________________
    From: oracle-l-bounce_at_freelists.org [oracle-l-bounce_at_freelists.org] On
    Behalf Of Radoulov, Dimitre [cichomitiko_at_gmail.com]
    Sent: Friday, April 15, 2011 9:21 AM
    To: Karl Arao
    Cc: ORACLE-L
    Subject: Re: ASM and EMC PowerPath

    Hi Karl,
    thanks!

    Unfortunately I'm not able to run such a test, at least not for now.

    So please let me/us know, if/when you have updates on this from the EMC
    support.


    Best regards
    Dimitre


    On 14/04/2011 19:01, Karl Arao wrote:
    Hi Dimitre,

    I'm also curious about that.. especially on this

    "Unlike most multipath drivers, PowerPath will support full multipath
    capabilites even with the regular names. PowerPath and ASMLib now work
    together successfully."

    so I asked some friends from EMC. So we are still waiting for the EMC
    support for their answer.

    But for a really definitive answer.. if you have a test or r&d environment
    you can unplug the storage cable connection and see if the fail-over works
    even if there's no scanorder or scanexclude. And if this does not work, it
    will be just a simple configuration file change and restart of the nodes.


    --
    Karl Arao
    karlarao.wordpress.com<http://karlarao.wordpress.com>
    karlarao.tiddlyspot.com<http://karlarao.tiddlyspot.com>
    --
    Karl Arao
    karlarao.wordpress.com
    karlarao.tiddlyspot.com

    --
    http://www.freelists.org/webpage/oracle-l

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouporacle-l @
categoriesoracle
postedApr 7, '11 at 2:03p
activeApr 20, '11 at 2:16p
posts34
users9
websiteoracle.com

People

Translate

site design / logo © 2022 Grokbase