FAQ
Hi all,

After finding multiples answers to this question via google, but without
making it work on my servers. Has anybody iSCSI Offload working on a Dell
Server with Broadcom NICs ?

My environment: I'm running CentOS 5.6 CR, on a Dell PowerEdge R710 with
Broadcom Corporation NetXtreme II BCM5709 conecting to an EMC CX4-120 SAN,
via 2x Cisco 2960G-24TC-L switches. It's working without the iSCSI offload.

Thanks in advance,

Antonio.

--
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Antonio S. Martins Jr. - Support Analyst | "Only The Shadow Knows |
Universidade Estadual de Maring? - Brasil| what evil lurks in the |
NPD - N?cleo de Processamento de Dados | Heart of Men!" |
E-Mail: asmartins at uem.br / shadow at uem.br | !!! Linux User: 52392 !!! |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
"Real Programmers don?t need comments ? the code is obvious."

Search Discussions

  • Drew Weaver at Sep 9, 2011 at 8:05 am
    Hi,

    I could be wrong here but don't you go into the Broadcom NIC configuration while the server is booting and add the iSCSI target in there and then it should appear as "Just Another Volume (TM)" to the operating system?

    I've never tried it but I assume thats how the 'offloading' works.

    thanks,
    -Drew


    -----Original Message-----
    From: centos-bounces at centos.org [mailto:centos-bounces at centos.org] On Behalf Of Antonio da Silva Martins Junior
    Sent: Friday, September 09, 2011 8:01 AM
    To: CentOS mailing list
    Subject: [CentOS] CentOS5 with Dell Broadcom iSCSI Offload, does it work ?

    Hi all,

    After finding multiples answers to this question via google, but without
    making it work on my servers. Has anybody iSCSI Offload working on a Dell
    Server with Broadcom NICs ?

    My environment: I'm running CentOS 5.6 CR, on a Dell PowerEdge R710 with
    Broadcom Corporation NetXtreme II BCM5709 conecting to an EMC CX4-120 SAN,
    via 2x Cisco 2960G-24TC-L switches. It's working without the iSCSI offload.

    Thanks in advance,

    Antonio.

    --
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    Antonio S. Martins Jr. - Support Analyst | "Only The Shadow Knows |
    Universidade Estadual de Maring? - Brasil| what evil lurks in the |
    NPD - N?cleo de Processamento de Dados | Heart of Men!" |
    E-Mail: asmartins at uem.br / shadow at uem.br | !!! Linux User: 52392 !!! |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    "Real Programmers don?t need comments ? the code is obvious."
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
  • Steve Walsh at Sep 9, 2011 at 8:17 am

    On 09/09/2011 10:05 PM, Drew Weaver wrote:
    Hi,

    I could be wrong here but don't you go into the Broadcom NIC configuration while the server is booting and add the iSCSI target in there and then it should appear as "Just Another Volume (TM)" to the operating system?

    I've never tried it but I assume thats how the 'offloading' works.
    close, but not. TCP Offloading (obviously) offloads the TCP protocol
    processing from the server CPU onto the server adapter. This preserves
    valuable CPU cycles for applications processing and improves overall
    server performance and network efficiency, as the heavy lifting for
    iSCSI is done on the card, by the card.

    <snip>
    After finding multiples answers to this question via google, but without
    making it work on my servers. Has anybody iSCSI Offload working on a Dell
    Server with Broadcom NICs ?
    Depending on how the Broadcom is in the server (ie - Physical Card vs
    onboard mezzanine card) depends on if you enable it in BIOS, or switch
    into the firmware when prompted at boot time to enable it.
  • Antonio da Silva Martins Junior at Sep 9, 2011 at 3:30 pm

    ----- "Steve Walsh" <steve at nerdvana.net.au> escreveu:

    De: "Steve Walsh" <steve at nerdvana.net.au>
    Para: centos at centos.org
    Enviadas: Sexta-feira, 9 de Setembro de 2011 9:17:50 (GMT-0300) Auto-Detected
    Assunto: Re: [CentOS] CentOS5 with Dell Broadcom iSCSI Offload, does it work ?
    On 09/09/2011 10:05 PM, Drew Weaver wrote:
    Hi,

    I could be wrong here but don't you go into the Broadcom NIC
    configuration while the server is booting and add the iSCSI target in
    there and then it should appear as "Just Another Volume (TM)" to the
    operating system?

    I've never tried it but I assume thats how the 'offloading' works.
    close, but not. TCP Offloading (obviously) offloads the TCP protocol
    processing from the server CPU onto the server adapter. This preserves
    valuable CPU cycles for applications processing and improves overall
    server performance and network efficiency, as the heavy lifting for
    iSCSI is done on the card, by the card.

    <snip>
    After finding multiples answers to this question via google, but
    without making it work on my servers. Has anybody iSCSI Offload
    working on a Dell Server with Broadcom NICs ?
    Depending on how the Broadcom is in the server (ie - Physical Card vs
    onboard mezzanine card) depends on if you enable it in BIOS, or switch
    into the firmware when prompted at boot time to enable it.
    Hi,

    Well... I looked into the NICs config to make sure it's on. On BIOS I can
    see the 4 onboard NICs. On these I can chage from "no boot" / PXE / iSCSI boot. And
    the BIOS "detect" that this NICs are TOE & ISOE "capable". I had PXE on the first one
    and "no boot" on the other 3. From the NICs firmware menu I can see all the 8 NICs
    (including the 4 onboard), I can change the same options, no option to enable/disable iSCSI
    Offload, only iSCSI boot.

    I'm trying the instructions from:

    - http://www.open-iscsi.org/docs/README
    - http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/pdf/Online_Storage_Reconfiguration_Guide/Red_Hat_Enterprise_Linux-5-Online_Storage_Reconfiguration_Guide-en-US.pdf
    - http://webpages.marshall.edu/~wolfe21/papers/rhel5-iscsi-HOWTO.pdf


    All of these say almost the same, and I can configure open-iscsi to access the EMC SAN using the two
    broadcom NICs using dm-multipath. But when I tryed to use the second MAC on the NIC (to activate
    iSCSI Offload) then I can't connect on the SAN. I can discovery it using the second MAC, but can't connect.

    Using "default" iface, it will conect using eth1 or eth2:

    [root at vHst-153-NEAD03 ~]# iscsiadm -m discoverydb -t st -p 10.252.1.1:3260 -I default --discover -P 1
    Target: iqn.1992-04.com.emc:cx.ckm00091500140.a2
    Portal: 10.252.1.1:3260,1
    Iface Name: default
    Target: iqn.1992-04.com.emc:cx.ckm00091500140.b2
    Portal: 10.252.1.2:3260,2
    Iface Name: default
    Target: iqn.1992-04.com.emc:cx.ckm00091500140.a3
    Portal: 10.252.2.1:3260,3
    Iface Name: default
    Target: iqn.1992-04.com.emc:cx.ckm00091500140.b3
    Portal: 10.252.2.2:3260,4
    Iface Name: default

    [root at vHst-153-NEAD03 ~]# iscsiadm -m node -I default -lLogging in to [iface: default, target: iqn.1992-04.com.emc:cx.ckm00091500140.a3, portal: 10.252.2.1,3260]
    Logging in to [iface: default, target: iqn.1992-04.com.emc:cx.ckm00091500140.b2, portal: 10.252.1.2,3260]
    Logging in to [iface: default, target: iqn.1992-04.com.emc:cx.ckm00091500140.b3, portal: 10.252.2.2,3260]
    Logging in to [iface: default, target: iqn.1992-04.com.emc:cx.ckm00091500140.a2, portal: 10.252.1.1,3260]
    Login to [iface: default, target: iqn.1992-04.com.emc:cx.ckm00091500140.a3, portal: 10.252.2.1,3260] successful.
    Login to [iface: default, target: iqn.1992-04.com.emc:cx.ckm00091500140.b2, portal: 10.252.1.2,3260] successful.
    Login to [iface: default, target: iqn.1992-04.com.emc:cx.ckm00091500140.b3, portal: 10.252.2.2,3260] successful.
    Login to [iface: default, target: iqn.1992-04.com.emc:cx.ckm00091500140.a2, portal: 10.252.1.1,3260] successful.

    [root at vHst-153-NEAD03 ~]# iscsiadm -m session
    tcp: [5] 10.252.2.1:3260,3 iqn.1992-04.com.emc:cx.ckm00091500140.a3
    tcp: [6] 10.252.1.2:3260,2 iqn.1992-04.com.emc:cx.ckm00091500140.b2
    tcp: [7] 10.252.2.2:3260,4 iqn.1992-04.com.emc:cx.ckm00091500140.b3
    tcp: [8] 10.252.1.1:3260,1 iqn.1992-04.com.emc:cx.ckm00091500140.a2

    [root at vHst-153-NEAD03 ~]# iscsiadm -m session -P 1
    Target: iqn.1992-04.com.emc:cx.ckm00091500140.a3
    Current Portal: 10.252.2.1:3260,3
    Persistent Portal: 10.252.2.1:3260,3
    **********
    Interface:
    **********
    Iface Name: default
    Iface Transport: tcp
    Iface Initiatorname: iqn.1994-05.com.redhat:vHst-153-NEAD03
    Iface IPaddress: 10.252.2.153
    Iface HWaddress: <empty>
    Iface Netdev: <empty>
    SID: 5
    iSCSI Connection State: LOGGED IN
    iSCSI Session State: LOGGED_IN
    Internal iscsid Session State: NO CHANGE
    <snip>

    But, if I try the same with the eth1 offload, I get:

    [root at vHst-153-NEAD03 ~]# ifconfig eth1
    eth1 Link encap:Ethernet Endere?o de HW BC:30:5B:FD:F7:E7
    inet end.: 10.252.1.153 Bcast:10.252.1.255 Masc:255.255.255.0
    endere?o inet6: fe80::be30:5bff:fefd:f7e7/64 Escopo:Link
    UP BROADCASTRUNNING MULTICAST MTU:9000 M?trica:1
    RX packets:735 errors:0 dropped:0 overruns:0 frame:0
    TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
    colis?es:0 txqueuelen:1000
    RX bytes:47040 (45.9 KiB) TX bytes:620 (620.0 b)
    IRQ:26 Mem?ria:d2000000-d2012800

    [root at vHst-153-NEAD03 ~]# iscsiadm -m iface
    default tcp,<empty>,<empty>,<empty>,<empty>
    iser iser,<empty>,<empty>,<empty>,<empty>
    bnx2i.bc:30:5b:fd:f7:e8 bnx2i,bc:30:5b:fd:f7:e8,10.252.1.53,<empty>,<empty>
    bnx2i.bc:30:5b:fd:f7:ea bnx2i,bc:30:5b:fd:f7:ea,10.252.2.53,<empty>,<empty>

    [root at vHst-153-NEAD03 ~]# iscsiadm -m iface -I bnx2i.bc:30:5b:fd:f7:e8
    # BEGIN RECORD 2.0-872
    iface.iscsi_ifacename = bnx2i.bc:30:5b:fd:f7:e8
    iface.net_ifacename = <empty>
    iface.ipaddress = 10.252.1.53
    iface.hwaddress = bc:30:5b:fd:f7:e8
    iface.transport_name = bnx2i
    iface.initiatorname = <empty>
    # END RECORD

    [root at vHst-153-NEAD03 ~]# iscsiadm -m discoverydb -t st -p 10.252.1.1:3260 -I bnx2i.bc:30:5b:fd:f7:e8 --discover -P 1
    Target: iqn.1992-04.com.emc:cx.ckm00091500140.a2
    Portal: 10.252.1.1:3260,1
    Iface Name: bnx2i.bc:30:5b:fd:f7:e8
    Target: iqn.1992-04.com.emc:cx.ckm00091500140.b2
    Portal: 10.252.1.2:3260,2
    Iface Name: bnx2i.bc:30:5b:fd:f7:e8
    Target: iqn.1992-04.com.emc:cx.ckm00091500140.a3
    Portal: 10.252.2.1:3260,3
    Iface Name: bnx2i.bc:30:5b:fd:f7:e8
    Target: iqn.1992-04.com.emc:cx.ckm00091500140.b3
    Portal: 10.252.2.2:3260,4
    Iface Name: bnx2i.bc:30:5b:fd:f7:e8

    [root at vHst-153-NEAD03 ~]# iscsiadm -m node -I bnx2i.bc:30:5b:fd:f7:e8 -l
    Logging in to [iface: bnx2i.bc:30:5b:fd:f7:e8, target: iqn.1992-04.com.emc:cx.ckm00091500140.a3, portal: 10.252.2.1,3260]
    Logging in to [iface: bnx2i.bc:30:5b:fd:f7:e8, target: iqn.1992-04.com.emc:cx.ckm00091500140.b2, portal: 10.252.1.2,3260]
    Logging in to [iface: bnx2i.bc:30:5b:fd:f7:e8, target: iqn.1992-04.com.emc:cx.ckm00091500140.b3, portal: 10.252.2.2,3260]
    Logging in to [iface: bnx2i.bc:30:5b:fd:f7:e8, target: iqn.1992-04.com.emc:cx.ckm00091500140.a2, portal: 10.252.1.1,3260]
    iscsiadm: Could not login to [iface: bnx2i.bc:30:5b:fd:f7:e8, target: iqn.1992-04.com.emc:cx.ckm00091500140.a3, portal: 10.252.2.1,3260].
    iscsiadm: initiator reported error (8 - connection timed out)
    iscsiadm: Could not login to [iface: bnx2i.bc:30:5b:fd:f7:e8, target: iqn.1992-04.com.emc:cx.ckm00091500140.b2, portal: 10.252.1.2,3260].
    iscsiadm: initiator reported error (8 - connection timed out)
    iscsiadm: Could not login to [iface: bnx2i.bc:30:5b:fd:f7:e8, target: iqn.1992-04.com.emc:cx.ckm00091500140.b3, portal: 10.252.2.2,3260].
    iscsiadm: initiator reported error (8 - connection timed out)
    iscsiadm: Could not login to [iface: bnx2i.bc:30:5b:fd:f7:e8, target: iqn.1992-04.com.emc:cx.ckm00091500140.a2, portal: 10.252.1.1,3260].
    iscsiadm: initiator reported error (8 - connection timed out)
    iscsiadm: Could not log into all portals

    Well, has anybody make this to work ? What I'm missing ?

    Thanks in advance,

    Antonio.

    --
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    Antonio S. Martins Jr. - Support Analyst | "Only The Shadow Knows |
    Universidade Estadual de Maring? - Brasil| what evil lurks in the |
    NPD - N?cleo de Processamento de Dados | Heart of Men!" |
    E-Mail: asmartins at uem.br / shadow at uem.br | !!! Linux User: 52392 !!! |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    "Real Programmers don?t need comments ? the code is obvious."
  • Alexander Dalloz at Sep 9, 2011 at 3:38 pm

    Am 09.09.2011 14:00, schrieb Antonio da Silva Martins Junior:
    Hi all,

    After finding multiples answers to this question via google, but without
    making it work on my servers. Has anybody iSCSI Offload working on a Dell
    Server with Broadcom NICs ?

    My environment: I'm running CentOS 5.6 CR, on a Dell PowerEdge R710 with
    Broadcom Corporation NetXtreme II BCM5709 conecting to an EMC CX4-120 SAN,
    via 2x Cisco 2960G-24TC-L switches. It's working without the iSCSI offload.

    Thanks in advance,

    Antonio.
    http://www.dell.com/us/enterprise/p/broadcom-netxtremeii-5709-iscsi/pd.aspx

    quote "No Linux support for TOE"

    Alexander
  • Alexander Dalloz at Sep 9, 2011 at 4:03 pm

    Am 09.09.2011 21:38, schrieb Alexander Dalloz:
    Am 09.09.2011 14:00, schrieb Antonio da Silva Martins Junior:
    Hi all,

    After finding multiples answers to this question via google, but without
    making it work on my servers. Has anybody iSCSI Offload working on a Dell
    Server with Broadcom NICs ?

    My environment: I'm running CentOS 5.6 CR, on a Dell PowerEdge R710 with
    Broadcom Corporation NetXtreme II BCM5709 conecting to an EMC CX4-120 SAN,
    via 2x Cisco 2960G-24TC-L switches. It's working without the iSCSI offload.

    Thanks in advance,

    Antonio.
    http://www.dell.com/us/enterprise/p/broadcom-netxtremeii-5709-iscsi/pd.aspx

    quote "No Linux support for TOE"

    Alexander
    On the other hand Dell has documentation on how to use your card for
    iSCSI offloading:

    http://support.euro.dell.com/support/edocs/network/BroadCom/R230837/en/linux.htm

    Important part is to load the cnic module together with the bnx2i
    module. You must see a message like

    bnx2i [05:00.00]: ISCSI_INIT passed

    at system bootup.

    Alexander

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcentos @
categoriescentos
postedSep 9, '11 at 8:00a
activeSep 9, '11 at 4:03p
posts6
users4
websitecentos.org
irc#centos

People

Translate

site design / logo © 2022 Grokbase