FAQ
Hi list,

I'm having one of those 'I'm stupid' -problems with LVM on CentOS 5.2.
I've been working with traditional partitions until now, but I've
finally been sold on the theoretical benefits of using LVM, but for now
I only have a huge pile of broken filesystems to show for my efforts.

My scenario;
I attach a disk, either over USB or iSCSI.
I create a PV on this device, create a VG using the device, and slice
off a tiny LV for tests.

[root@kasse ~]# dmesg
usb 1-3: new high speed USB device using ehci_hcd and address 7
usb 1-3: configuration #1 chosen from 1 choice
scsi27 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 7
usb-storage: waiting for device to settle before scanning
Vendor: ST375064 Model: 0A Rev: 3.AA
Type: Direct-Access ANSI SCSI revision: 00
SCSI device sdg: 1465149168 512-byte hdwr sectors (750156 MB)
sdg: Write Protect is off
sdg: Mode Sense: 10 00 00 00
sdg: assuming drive cache: write through
SCSI device sdg: 1465149168 512-byte hdwr sectors (750156 MB)
sdg: Write Protect is off
sdg: Mode Sense: 10 00 00 00
sdg: assuming drive cache: write through
sdg: sdg1
sd 27:0:0:0: Attached scsi disk sdg
sd 27:0:0:0: Attached scsi generic sg6 type 0
usb-storage: device scan complete
[root@kasse ~]# pvcreate /dev/sdg1
Physical volume "/dev/sdg1" successfully created
[root@kasse ~]# vgcreate testgroup /dev/sdg1
Volume group "testgroup" successfully created
[root@kasse ~]# lvcreate -L 200M testgroup -n testLV
Logical volume "testLV" created


At this point, I have /dev/testgroup/testLV, which I can stick a fs on,
mount, store files on, unmount, remount and recover the files from. Just
as I'd expect.

[root@kasse ~]# lvdisplay
--- Logical volume ---
LV Name /dev/testgroup/testLV
VG Name testgroup
LV UUID 3KjDiZ-gTAD-sC9E-sOSA-772o-41Yk-ZKK7cx
LV Write Access read/write
LV Status available
# open 0
LV Size 200.00 MB
Current LE 50
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:1


I have not explicitly set the vg active, as I understand that it
implicitly will be set active in non-clustered environments like mine.
LVM works as I'd expect it to up until here in any case.

So - with the LV unmounted, I power my USB-device down, and then back up;

[root@kasse ~]# dmesg
usb 1-3: USB disconnect, address 7
usb 1-3: new high speed USB device using ehci_hcd and address 8
usb 1-3: configuration #1 chosen from 1 choice
scsi28 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 8
usb-storage: waiting for device to settle before scanning
Vendor: ST375064 Model: 0A Rev: 3.AA
Type: Direct-Access ANSI SCSI revision: 00
SCSI device sdi: 1465149168 512-byte hdwr sectors (750156 MB)
sdi: Write Protect is off
sdi: Mode Sense: 10 00 00 00
sdi: assuming drive cache: write through
SCSI device sdi: 1465149168 512-byte hdwr sectors (750156 MB)
sdi: Write Protect is off
sdi: Mode Sense: 10 00 00 00
sdi: assuming drive cache: write through
sdi: sdi1
sd 28:0:0:0: Attached scsi disk sdi
sd 28:0:0:0: Attached scsi generic sg6 type 0
usb-storage: device scan complete

(note that my disk is now sdi, not sdg, and connected on scsi28 and not
scsi27)

If I now try to access the device, all falls appart;
[root@kasse ~]# lvdisplay
/dev/testgroup/testLV: read failed after 0 of 4096 at 209649664:
Input/output error
/dev/testgroup/testLV: read failed after 0 of 4096 at 209707008:
Input/output error
/dev/testgroup/testLV: read failed after 0 of 4096 at 0: Input/output
error
/dev/testgroup/testLV: read failed after 0 of 4096 at 4096:
Input/output error
/dev/testgroup/testLV: read failed after 0 of 4096 at 0: Input/output
error
--- Logical volume ---
LV Name /dev/testgroup/testLV
VG Name testgroup
LV UUID 3KjDiZ-gTAD-sC9E-sOSA-772o-41Yk-ZKK7cx
LV Write Access read/write
LV Status available
# open 0
LV Size 200.00 MB
Current LE 50
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:1


Trying to mount or otherwise access the LV also shows the problems;


scsi 27:0:0:0: rejecting I/O to dead device
scsi 27:0:0:0: rejecting I/O to dead device
scsi 27:0:0:0: rejecting I/O to dead device
scsi 27:0:0:0: rejecting I/O to dead device
scsi 27:0:0:0: rejecting I/O to dead device
scsi 27:0:0:0: rejecting I/O to dead device
printk: 25 messages suppressed.
Buffer I/O error on device dm-1, logical block 30
Buffer I/O error on device dm-1, logical block 31
scsi 27:0:0:0: rejecting I/O to dead device
Buffer I/O error on device dm-1, logical block 0
Buffer I/O error on device dm-1, logical block 1
Buffer I/O error on device dm-1, logical block 2
Buffer I/O error on device dm-1, logical block 3
Buffer I/O error on device dm-1, logical block 4
Buffer I/O error on device dm-1, logical block 5
Buffer I/O error on device dm-1, logical block 6
Buffer I/O error on device dm-1, logical block 7
scsi 27:0:0:0: rejecting I/O to dead device
scsi 27:0:0:0: rejecting I/O to dead device
scsi 27:0:0:0: rejecting I/O to dead device
scsi 27:0:0:0: rejecting I/O to dead device
hfs: unable to find HFS+ superblock

So - LVM seems to believe my LV is still attached to scsi27, while it
now actually sits on scsi28.

What am I missing here?

How do I get LVM to use a persistent device instead of a transient path?

Yours,
-S
--
Simen Thoresen, Dolphin ICS
Systems Administration and Wulfkit Support

Search Discussions

  • William L. Maltby at Aug 31, 2008 at 11:14 am

    On Sun, 2008-08-31 at 11:19 +0200, Simen Timian Thoresen wrote:
    Hi list,

    I'm having one of those 'I'm stupid' -problems with LVM on CentOS 5.2.
    I've been working with traditional partitions until now, but I've
    finally been sold on the theoretical benefits of using LVM, but for now
    I only have a huge pile of broken filesystems to show for my efforts.

    My scenario;
    I attach a disk, either over USB or iSCSI.
    I create a PV on this device, create a VG using the device, and slice
    off a tiny LV for tests.
    Uh-oh! I hope these are for temporary usage only!
    [root@kasse ~]# dmesg
    <snip>
    [root@kasse ~]# pvcreate /dev/sdg1
    Physical volume "/dev/sdg1" successfully created
    [root@kasse ~]# vgcreate testgroup /dev/sdg1
    Volume group "testgroup" successfully created
    [root@kasse ~]# lvcreate -L 200M testgroup -n testLV
    Logical volume "testLV" created


    At this point, I have /dev/testgroup/testLV, which I can stick a fs on,
    mount, store files on, unmount, remount and recover the files from. Just
    as I'd expect.

    [root@kasse ~]# lvdisplay
    --- Logical volume ---
    LV Name /dev/testgroup/testLV
    VG Name testgroup
    LV UUID 3KjDiZ-gTAD-sC9E-sOSA-772o-41Yk-ZKK7cx
    LV Write Access read/write
    LV Status available
    # open 0
    LV Size 200.00 MB
    Current LE 50
    Segments 1
    Allocation inherit
    Read ahead sectors auto
    - currently set to 256
    Block device 253:1


    I have not explicitly set the vg active, as I understand that it
    implicitly will be set active in non-clustered environments like mine.
    LVM works as I'd expect it to up until here in any case.

    So - with the LV unmounted, I power my USB-device down, and then back up;

    [root@kasse ~]# dmesg
    <snip>
    (note that my disk is now sdi, not sdg, and connected on scsi28 and not
    scsi27)
    By now you realize that is a big problem?
    If I now try to access the device, all falls appart;
    [root@kasse ~]# lvdisplay
    /dev/testgroup/testLV: read failed after 0 of 4096 at 209649664:
    Input/output error
    <snip>
    --- Logical volume ---
    LV Name /dev/testgroup/testLV
    VG Name testgroup
    LV UUID 3KjDiZ-gTAD-sC9E-sOSA-772o-41Yk-ZKK7cx
    LV Write Access read/write
    LV Status available
    # open 0
    LV Size 200.00 MB
    Current LE 50
    Segments 1
    Allocation inherit
    Read ahead sectors auto
    - currently set to 256
    Block device 253:1


    Trying to mount or otherwise access the LV also shows the problems;


    scsi 27:0:0:0: rejecting I/O to dead device
    <snip>
    Buffer I/O error on device dm-1, logical block 7
    scsi 27:0:0:0: rejecting I/O to dead device
    scsi 27:0:0:0: rejecting I/O to dead device
    scsi 27:0:0:0: rejecting I/O to dead device
    scsi 27:0:0:0: rejecting I/O to dead device
    hfs: unable to find HFS+ superblock

    So - LVM seems to believe my LV is still attached to scsi27, while it
    now actually sits on scsi28.

    What am I missing here?
    Use the pvdisplay command and you'll see.

    # pvdisplay
    --- Physical volume ---
    PV Name /dev/sdb2
    VG Name VolGroup00
    PV Size 148.95 GB / not usable 11.37 MB
    Allocatable yes (but full)
    PE Size (KByte) 32768
    Total PE 4766
    Free PE 0
    Allocated PE 4766
    PV UUID eeqb0A-6EJV-VE3Q-paLi-kTXv-S46k-0xaeTy

    Sit back a moment and think about it. You defined a PV on a specific
    device. That device is no longer available. LVM has to track the
    assignment, allocation and use of PVs. It must have a record of it
    somewhere.

    # ls -l /etc/lvm
    total 44
    drwx------ 2 root root 4096 May 25 00:41 archive
    drwx------ 2 root root 4096 May 25 00:41 backup
    drwx------ 2 root root 4096 Aug 30 08:38 cache
    -rw-r--r-- 1 root root 15911 May 25 00:41 lvm.conf

    Use the man command to read up on more stuff, like "man lvm.conf" and
    "man lvm". Be sure and read some of the other refs at the bottom of the
    man pages.
    How do I get LVM to use a persistent device instead of a transient path?
    Normally you give a persistent device instead of a transient path. You
    didn't do that.

    Generally speaking, LVM is designed to use fixed devices - those that
    always come up on the same address. Neither iscsi nor UHB devices do
    this reliably. However, on a reboot, there is a liklihood that they will
    be the same devices.

    I've not tried to configure it on transient devices. There may be a way,
    but I've not looked into it.

    If you make them their own VG, rather than adding the PV to another VG,
    you can use the export and import LVM commands, but then you really
    accomplish nothing more than what could be done by mounting the
    underlying devices in existing directory structures. No point in LVM
    then unless you anticipate having to grow those mounted structures while
    in use.

    Even so, from my LFS days, I do recall a lot of flexibility in the udev
    process. Maybe there is something there that will allow the assigned
    device addresses to be fixed?

    Do man for udev, udevd, udevinfo, udevmonitor. I'm guessing that you'll
    want to run some utility from udev that will (maybe) look at a a volume
    label (if one if available on the usb device - is there one on the iscsi
    device?) and assign a predetermined name if a certain label is found.

    I've never used iscsi, so I'm even less help on that. Maybe there is
    something in its man pages that allow a persistent device name
    assignment.
    Yours,
    -S
    Sorry I couldn't offer more.

    --
    Bill
  • Nate Amsden at Aug 31, 2008 at 2:27 pm

    Simen Timian Thoresen wrote:

    So - with the LV unmounted, I power my USB-device down, and then back up;

    I believe your problem is you didn't deactivate the logical
    volume, and export the volume group before disconnecting.

    lvchange -a n <path to lvm>
    vgexport -a (don't worry it will only export groups that are
    ready to be exported)

    Now you can safely power the device down(provided the VG was
    successfully exported, all logical volumes must be unmounted
    and deactivated before it will succeed).

    when you power it back up run

    pvscan
    vgimport -a
    lvchange -a y <path to lvm>

    LVM is very useful for volumes that change device names, but you
    must use it properly otherwise bad things will happen as you
    saw.

    If you want to have USB mount automatically it may be better
    to put a label on the file system, at least in Debian(don't
    have a CentOS system with a USB disk handy here) the volume
    is mounted as /media/<disk label> if there is a disk label.
    You can use the e2label command to label the device. I'd
    expect CentOS to likely behave similarly to debian in this
    respect.

    nate
  • Simen Timian Thoresen at Aug 31, 2008 at 3:50 pm

    nate wrote:
    Simen Timian Thoresen wrote:
    So - with the LV unmounted, I power my USB-device down, and then back up;
    Hi Nate,
    I believe your problem is you didn't deactivate the logical
    volume, and export the volume group before disconnecting.
    Yes! Thank you - exporting and then importing again after replugging worked.

    I'll have to read up on what actually happens here .-)

    Still - say that the USB (or iSCSI) PV is not exported before being
    unplugged (cables tripped over, internet connection going down, etc) -
    how do I recover from an unplugged but not exported PV/VG? It seems that
    vg[im/ex]port only fiddle with the off-PV metadata. Is this correct?

    I think I'm looking for a force-import that allows me to bring a VG with
    associated LVs up provided that the PVs are present - affectively a
    'assemble' ala mdadm.

    Thank you for pointing me in the right way ;-)

    -S
    lvchange -a n <path to lvm>
    vgexport -a (don't worry it will only export groups that are
    ready to be exported)

    Now you can safely power the device down(provided the VG was
    successfully exported, all logical volumes must be unmounted
    and deactivated before it will succeed).

    when you power it back up run

    pvscan
    vgimport -a
    lvchange -a y <path to lvm>

    LVM is very useful for volumes that change device names, but you
    must use it properly otherwise bad things will happen as you
    saw.

    If you want to have USB mount automatically it may be better
    to put a label on the file system, at least in Debian(don't
    have a CentOS system with a USB disk handy here) the volume
    is mounted as /media/<disk label> if there is a disk label.
    You can use the e2label command to label the device. I'd
    expect CentOS to likely behave similarly to debian in this
    respect.

    nate


    _______________________________________________
    CentOS mailing list
    CentOS@centos.org
    http://lists.centos.org/mailman/listinfo/centos

    --
    Simen Thoresen, Dolphin ICS
    Systems Administration and Wulfkit Support
  • Simen Timian Thoresen at Aug 31, 2008 at 4:27 pm

    Simen Timian Thoresen wrote:
    nate wrote:
    Simen Timian Thoresen wrote:
    So - with the LV unmounted, I power my USB-device down, and then back
    up;
    Hi Nate,
    I believe your problem is you didn't deactivate the logical
    volume, and export the volume group before disconnecting.
    Yes! Thank you - exporting and then importing again after replugging
    worked.

    I'll have to read up on what actually happens here .-)

    Still - say that the USB (or iSCSI) PV is not exported before being
    unplugged (cables tripped over, internet connection going down, etc) -
    how do I recover from an unplugged but not exported PV/VG? It seems that
    vg[im/ex]port only fiddle with the off-PV metadata. Is this correct?

    I think I'm looking for a force-import that allows me to bring a VG with
    associated LVs up provided that the PVs are present - affectively a
    'assemble' ala mdadm.
    Hah! I got this one on my own.

    After unplugging, I removed the /etc/lvm/archive/ /etc/lvm/backup and
    /dev/mapper/<device> entries, and then replugged;

    [root@kasse ~]# pvscan
    PV /dev/iscsi_01 VG iscsistorage lvm2 [102.79 GB / 102.59 GB free]
    Total: 1 [102.79 GB] / in use: 1 [102.79 GB] / in no VG: 0 [0 ]
    (this I've seen before)

    Thinking that I could import the VG, I found that it's impossible to
    imoport a non-exported VG;

    [root@kasse ~]# vgimport -a
    Volume group "iscsistorage" is not exported

    ...but just setting it active worked;
    [root@kasse ~]# vgchange -a y iscsistorage
    1 logical volume(s) in volume group "iscsistorage" now active
    [root@kasse ~]# lvdisplay
    /dev/mapper/iscsistorage-lv1: open failed: No such file or directory
    /dev/mapper/iscsistorage-lv1: open failed: No such file or directory
    --- Logical volume ---
    LV Name /dev/iscsistorage/lv1
    VG Name iscsistorage
    LV UUID mKP1DG-Pc1q-vFzf-sR9v-RE3e-ChNV-T0dWq1
    (...)

    ...and the LV is not mountable;
    [root@kasse ~]# mount /dev/iscsistorage/lv1 /mnt/
    mount: special device /dev/iscsistorage/lv1 does not exist


    To remove the missing /dev/mapper entries, I exported and then
    reimported the VG;

    [root@kasse ~]# vgexport -a
    Volume group "iscsistorage" successfully exported
    [root@kasse ~]# vgimport -a
    Volume group "iscsistorage" successfully imported
    [root@kasse ~]# lvdisplay
    --- Logical volume ---
    LV Name /dev/iscsistorage/lv1
    VG Name iscsistorage
    LV UUID mKP1DG-Pc1q-vFzf-sR9v-RE3e-ChNV-T0dWq1
    (...)
    [root@kasse ~]# mount /dev/iscsistorage/lv1 /mnt/
    [root@kasse ~]# ls /mnt/
    lost+found
    (...)

    Woila!

    I think I saw someone doing something to /dev/mapper/control as well to
    skip the export-reimport steps.

    I'll see if I can robustify this a little, but if this works, I'll be
    significantly closer to my main goal.

    Thank you again for your help .-)

    Yours,
    -S

    Thank you for pointing me in the right way ;-)

    -S
    lvchange -a n <path to lvm>
    vgexport -a (don't worry it will only export groups that are
    ready to be exported)

    Now you can safely power the device down(provided the VG was
    successfully exported, all logical volumes must be unmounted
    and deactivated before it will succeed).

    when you power it back up run

    pvscan
    vgimport -a
    lvchange -a y <path to lvm>

    LVM is very useful for volumes that change device names, but you
    must use it properly otherwise bad things will happen as you
    saw.

    If you want to have USB mount automatically it may be better
    to put a label on the file system, at least in Debian(don't
    have a CentOS system with a USB disk handy here) the volume
    is mounted as /media/<disk label> if there is a disk label.
    You can use the e2label command to label the device. I'd
    expect CentOS to likely behave similarly to debian in this
    respect.

    nate


    _______________________________________________
    CentOS mailing list
    CentOS@centos.org
    http://lists.centos.org/mailman/listinfo/centos

    --
    Simen Thoresen, Dolphin ICS
    Systems Administration and Wulfkit Support

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcentos @
categoriescentos
postedAug 31, '08 at 9:19a
activeAug 31, '08 at 4:27p
posts5
users3
websitecentos.org
irc#centos

People

Translate

site design / logo © 2021 Grokbase