FAQ
Hi,

I'm setting up xen (on centos 5.3) and I'm facing a weird performance
issue with iSCSI.

I attach the iSCSI targets to dom0 and if I use them from dom0 I obtain
good performance with simple dd tests (~100MB/s both reading and
writing).

I then use the block devices for the domU and if I repeat the same dd
test
from within the domU the write performace is still good (~100MB/s), but
the read performance is cut in half (~55MB/s).

I tried changing several parameters like read_ahead and such, but I can
not obtain a good read performance in the domU.

Any idea?

Thanks, Daniel.

Search Discussions

  • Ben Montanelli at Sep 24, 2009 at 6:45 pm
    Are you using separate physical NICs for dom0 (isolated) and the domUs?

    In a CentOS domU I just, a few hours ago, had same issue on network
    reads, but doing:

    # ethtool -K eth0 tx off

    then rebooted domU and it cleared up and got much closer to max. Yes,
    that is the TX checksum fix, but it seemed to affect my reads (IO in).



    Daniel De Marco wrote:
    Hi,

    I'm setting up xen (on centos 5.3) and I'm facing a weird performance
    issue with iSCSI.

    I attach the iSCSI targets to dom0 and if I use them from dom0 I obtain
    good performance with simple dd tests (~100MB/s both reading and
    writing).

    I then use the block devices for the domU and if I repeat the same dd
    test
    from within the domU the write performace is still good (~100MB/s), but
    the read performance is cut in half (~55MB/s).

    I tried changing several parameters like read_ahead and such, but I can
    not obtain a good read performance in the domU.

    Any idea?

    Thanks, Daniel.

    _______________________________________________
    CentOS-virt mailing list
    CentOS-virt at centos.org
    http://lists.centos.org/mailman/listinfo/centos-virt
  • Daniel De Marco at Sep 24, 2009 at 8:05 pm
    Ben,

    thanks for the suggestion, it didn't work, but it led me to test the
    network of the domu and it turns out that with netperf to a different
    machine I can get ~500Mbit/s from the domU and ~950Mbit/s from the dom0.
    From another domU on a different dom0 (older systems, centos 5.3 with
    kernel 2.6.18-92.1.22.el5xen i686 instead of 2.6.18-164.el5xen x86_64)
    netperf produces 950Mbit/s as well.

    The weird thing is however that it's dom0 that acts as iSCSI initiator,
    so it is dom0 that is using the network and there it works fine.

    I'll go do some more tests.
    Daniel.


    * Ben Montanelli [09/24/2009 14:45]:
    Are you using separate physical NICs for dom0 (isolated) and the domUs?

    In a CentOS domU I just, a few hours ago, had same issue on network
    reads, but doing:

    # ethtool -K eth0 tx off

    then rebooted domU and it cleared up and got much closer to max. Yes,
    that is the TX checksum fix, but it seemed to affect my reads (IO in).



    Daniel De Marco wrote:
    Hi,

    I'm setting up xen (on centos 5.3) and I'm facing a weird performance
    issue with iSCSI.

    I attach the iSCSI targets to dom0 and if I use them from dom0 I obtain
    good performance with simple dd tests (~100MB/s both reading and
    writing).

    I then use the block devices for the domU and if I repeat the same dd
    test
    from within the domU the write performace is still good (~100MB/s), but
    the read performance is cut in half (~55MB/s).

    I tried changing several parameters like read_ahead and such, but I can
    not obtain a good read performance in the domU.

    Any idea?

    Thanks, Daniel.

    _______________________________________________
    CentOS-virt mailing list
    CentOS-virt at centos.org
    http://lists.centos.org/mailman/listinfo/centos-virt
    _______________________________________________
    CentOS-virt mailing list
    CentOS-virt at centos.org
    http://lists.centos.org/mailman/listinfo/centos-virt
  • Ben Montanelli at Sep 24, 2009 at 9:01 pm
    I think that is an interesting coincidence about x86_64 vs. 32bit
    kernel. In this same CentOS pv domU I was talking about I created an
    x86_64 by "accident." I ended up knocking it down and going to i386 tree
    (its a web app) and found it to be snappier.

    Please post your results especially if it is a kernel difference.

    Daniel De Marco wrote:
    Ben,

    thanks for the suggestion, it didn't work, but it led me to test the
    network of the domu and it turns out that with netperf to a different
    machine I can get ~500Mbit/s from the domU and ~950Mbit/s from the dom0.
    From another domU on a different dom0 (older systems, centos 5.3 with
    kernel 2.6.18-92.1.22.el5xen i686 instead of 2.6.18-164.el5xen x86_64)
    netperf produces 950Mbit/s as well.

    The weird thing is however that it's dom0 that acts as iSCSI initiator,
    so it is dom0 that is using the network and there it works fine.

    I'll go do some more tests.
    Daniel.


    * Ben Montanelli [09/24/2009 14:45]:
    Are you using separate physical NICs for dom0 (isolated) and the domUs?

    In a CentOS domU I just, a few hours ago, had same issue on network
    reads, but doing:

    # ethtool -K eth0 tx off

    then rebooted domU and it cleared up and got much closer to max. Yes,
    that is the TX checksum fix, but it seemed to affect my reads (IO in).



    Daniel De Marco wrote:
    Hi,

    I'm setting up xen (on centos 5.3) and I'm facing a weird performance
    issue with iSCSI.

    I attach the iSCSI targets to dom0 and if I use them from dom0 I obtain
    good performance with simple dd tests (~100MB/s both reading and
    writing).

    I then use the block devices for the domU and if I repeat the same dd
    test
    from within the domU the write performace is still good (~100MB/s), but
    the read performance is cut in half (~55MB/s).

    I tried changing several parameters like read_ahead and such, but I can
    not obtain a good read performance in the domU.

    Any idea?

    Thanks, Daniel.

    _______________________________________________
    CentOS-virt mailing list
    CentOS-virt at centos.org
    http://lists.centos.org/mailman/listinfo/centos-virt
    _______________________________________________
    CentOS-virt mailing list
    CentOS-virt at centos.org
    http://lists.centos.org/mailman/listinfo/centos-virt
    _______________________________________________
    CentOS-virt mailing list
    CentOS-virt at centos.org
    http://lists.centos.org/mailman/listinfo/centos-virt
  • Daniel De Marco at Sep 24, 2009 at 9:18 pm

    * Ben Montanelli [09/24/2009 17:06]:
    I think that is an interesting coincidence about x86_64 vs. 32bit
    kernel. In this same CentOS pv domU I was talking about I created an
    x86_64 by "accident." I ended up knocking it down and going to i386 tree
    (its a web app) and found it to be snappier.
    I tested with a i386 domU and the results are the same.

    Daniel.
  • Daniel De Marco at Sep 24, 2009 at 10:01 pm
    Please disregard the network speed discrepancy. I was not comparing the
    same thing. Both dom0 and domU are doing 950Mbit/s, but the iSCSI part
    is still weird.

    Daniel.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcentos-virt @
categoriescentos
postedSep 24, '09 at 6:34p
activeSep 24, '09 at 10:01p
posts6
users2
websitecentos.org
irc#centos

2 users in discussion

Daniel De Marco: 4 posts Ben Montanelli: 2 posts

People

Translate

site design / logo © 2022 Grokbase