FAQ
I've been reading that it's possible to set up a system with multiple
NIC to provide redundant internet connectivity such that it will
switch to a secondary connection if the primary ISP fails.

Is it possible in a similar way to setup redundant LAN routing? I read
that it is possible to aggregate/bond multiple NIC to stackable
switches that support link aggregation and redundancy. But if only
simple switches are available, is something like this possible?

e.g.
System A
eth0 -> lan switch/router 1
eth1 -> lan switch/router 2

System B
eth0 -> lan switch 1
eth1 -> lan switch 2

Then somehow specify that, if lan switch 1 fails, the two systems will
switch to using switch 2 so that in case of a switch failure, the
network continues to remain operational.

Search Discussions

  • Boris Epstein at Jul 10, 2010 at 7:01 pm

    On Sat, Jul 10, 2010 at 5:21 PM, Emmanuel Noobadmin wrote:

    I've been reading that it's possible to set up a system with multiple
    NIC to provide redundant internet connectivity such that it will
    switch to a secondary connection if the primary ISP fails.

    Is it possible in a similar way to setup redundant LAN routing? I read
    that it is possible to aggregate/bond multiple NIC to stackable
    switches that support link aggregation and redundancy. But if only
    simple switches are available, is something like this possible?

    e.g.
    System A
    eth0 -> lan switch/router 1
    eth1 -> lan switch/router 2

    System B
    eth0 -> lan switch 1
    eth1 -> lan switch 2

    Then somehow specify that, if lan switch 1 fails, the two systems will
    switch to using switch 2 so that in case of a switch failure, the
    network continues to remain operational.
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
    I'd think for this to be possible you will need a router with multiple
    WAN addresses/interfaces... I am not sure how that pertains to your
    LAN per se.

    Boris.
  • Whit Blauvelt at Jul 10, 2010 at 9:17 pm

    On Sun, Jul 11, 2010 at 05:21:50AM +0800, Emmanuel Noobadmin wrote:

    e.g.
    System A
    eth0 -> lan switch/router 1
    eth1 -> lan switch/router 2

    System B
    eth0 -> lan switch 1
    eth1 -> lan switch 2

    Then somehow specify that, if lan switch 1 fails, the two systems will
    switch to using switch 2 so that in case of a switch failure, the
    network continues to remain operational.
    If you're clever with scripting and iproute2 commands, rules and multiple
    routing tables, and everything's Linux, this is certainly doable. You could
    have your System A ping System B's IP via eth0 every minute, and on failure
    reassign its default route and IP to eth1. Meanwhile you can set up rules and
    routes on System B so that whichever NIC traffic comes in on, the response
    will use the same NIC ... stuff you'll find if you google around for how to
    be dual-homed between ISPs is quite applicable here.

    It's too complex to work it out for you in detail without spending an hour
    on it. But I've done this sort of thing and had it work very well.

    Whit
  • Rajagopal Swaminathan at Jul 10, 2010 at 9:54 pm
    Greetings,
    On 7/11/10, Emmanuel Noobadmin wrote:
    I've been reading that it's possible to set up a system with multiple
    NIC to provide redundant internet connectivity such that it will
    switch to a secondary connection if the primary ISP fails.

    Is it possible in a similar way to setup redundant LAN routing? I read
    that it is possible to aggregate/bond multiple NIC to stackable
    switches that support link aggregation and redundancy. But if only
    simple switches are available, is something like this possible?

    e.g.
    System A
    eth0 -> lan switch/router 1
    eth1 -> lan switch/router 2

    System B
    eth0 -> lan switch 1
    eth1 -> lan switch 2

    Then somehow specify that, if lan switch 1 fails, the two systems will
    switch to using switch 2 so that in case of a switch failure, the
    network continues to remain operational.

    hmm.. lartc.org comes to mind to begin with...

    duh.. that was too primitive. pfSense perhaps...

    But then there is untangle if you want to pay them..

    etc. etc.

    Regards,

    Rajagopal
  • Jerry Franz at Jul 10, 2010 at 10:59 pm

    On 7/10/2010 2:21 PM, Emmanuel Noobadmin wrote:
    I've been reading that it's possible to set up a system with multiple
    NIC to provide redundant internet connectivity such that it will
    switch to a secondary connection if the primary ISP fails.

    Is it possible in a similar way to setup redundant LAN routing? I read
    that it is possible to aggregate/bond multiple NIC to stackable
    switches that support link aggregation and redundancy. But if only
    simple switches are available, is something like this possible?

    e.g.
    System A
    eth0 -> lan switch/router 1
    eth1 -> lan switch/router 2

    System B
    eth0 -> lan switch 1
    eth1 -> lan switch 2

    Then somehow specify that, if lan switch 1 fails, the two systems will
    switch to using switch 2 so that in case of a switch failure, the
    network continues to remain operational.
    Yes. You can do it. I've done it before. All you need is the right
    choice of bonding mode . You set up bond0 for eth0 and eth1 and it 'just
    works'. To make it more robust, cross-connect the two switches as well.

    --
    Benjamin Franz
  • Emmanuel Noobadmin at Jul 11, 2010 at 12:48 am
    Thanks for the suggestion, I'll read up more about them. The
    bond0 and just works sounds simple which is a Good Thing! The problem
    was the last time I tried to cross connect multiple switches,
    everything just died so there must be something a bit more involved?
    :D

    In the mean time since my post, I came across STP (spanning tree
    protocol) that seems to be designed to handle this sort of thing, i.e.
    figure out the shortest path and prevent network shortcircuit like
    what I had experienced with cross connecting multiple switches.

    But it apparently takes 50 seconds to reconfigure anytime sometime in
    the circuit fails. There is supposedly a Rapid STP that only takes 3
    seconds. Several couple-of-years old search results indicate that it
    was tested in 2.4 kernel and will be in 2.6 kernel. However, I cannot
    seem to find anything newer that confirms if such functionality is
    really in the current kernel. Anybody has any idea?


    On 7/11/10, Jerry Franz wrote:
    On 7/10/2010 2:21 PM, Emmanuel Noobadmin wrote:
    I've been reading that it's possible to set up a system with multiple
    NIC to provide redundant internet connectivity such that it will
    switch to a secondary connection if the primary ISP fails.

    Is it possible in a similar way to setup redundant LAN routing? I read
    that it is possible to aggregate/bond multiple NIC to stackable
    switches that support link aggregation and redundancy. But if only
    simple switches are available, is something like this possible?

    e.g.
    System A
    eth0 -> lan switch/router 1
    eth1 -> lan switch/router 2

    System B
    eth0 -> lan switch 1
    eth1 -> lan switch 2

    Then somehow specify that, if lan switch 1 fails, the two systems will
    switch to using switch 2 so that in case of a switch failure, the
    network continues to remain operational.
    Yes. You can do it. I've done it before. All you need is the right
    choice of bonding mode . You set up bond0 for eth0 and eth1 and it 'just
    works'. To make it more robust, cross-connect the two switches as well.

    --
    Benjamin Franz
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
  • Jerry Franz at Jul 11, 2010 at 5:50 am

    On 07/10/2010 09:48 PM, Emmanuel Noobadmin wrote:
    Thanks for the suggestion, I'll read up more about them. The
    bond0 and just works sounds simple which is a Good Thing! The problem
    was the last time I tried to cross connect multiple switches,
    everything just died so there must be something a bit more involved?
    :D
    Not really. You should connect the 'uplink' port to a regular port or
    use a cross-over cable to connect switches (assuming your switches don't
    auto-switch ports) and make only one connection between each switch and
    the next. I've got four switches chained here in my house right now
    without a problem to distribute my internet connection around various
    rooms using cheap retail 5 port d-link switches. Just don't create loops
    or other weird architectures, don't chain too many together, and you
    should be fine.
    In the mean time since my post, I came across STP (spanning tree
    protocol) that seems to be designed to handle this sort of thing, i.e.
    figure out the shortest path and prevent network shortcircuit like
    what I had experienced with cross connecting multiple switches.

    But it apparently takes 50 seconds to reconfigure anytime sometime in
    the circuit fails. There is supposedly a Rapid STP that only takes 3
    seconds. Several couple-of-years old search results indicate that it
    was tested in 2.4 kernel and will be in 2.6 kernel. However, I cannot
    seem to find anything newer that confirms if such functionality is
    really in the current kernel. Anybody has any idea?
    You probably don't need to worry about STP unless you are using
    explicitly bridging the servers' NICs. And hopefully your hardware is
    reliable enough that worrying about a 50 second reconfiguration is
    something that happens once in several years in the first place.

    --
    Jerry Franz
  • Chan Chung Hang Christopher at Jul 11, 2010 at 12:05 pm

    Emmanuel Noobadmin wrote:
    Thanks for the suggestion, I'll read up more about them. The
    bond0 and just works sounds simple which is a Good Thing! The problem
    was the last time I tried to cross connect multiple switches,
    everything just died so there must be something a bit more involved?
    :D

    In the mean time since my post, I came across STP (spanning tree
    protocol) that seems to be designed to handle this sort of thing, i.e.
    figure out the shortest path and prevent network shortcircuit like
    what I had experienced with cross connecting multiple switches.
    You only really need STP when you have switches that are connected
    together in such a way as to have multiple paths. For the setup you
    first posted, you could just have two physically separate networks. That
    does leave the question of what solution to use to get the boxes to use
    the other switch if the primary one goes down. So if you connect both
    networks to make say a big 'circular' network, then you need STP.

    But it apparently takes 50 seconds to reconfigure anytime sometime in
    the circuit fails. There is supposedly a Rapid STP that only takes 3
    seconds. Several couple-of-years old search results indicate that it
    was tested in 2.4 kernel and will be in 2.6 kernel. However, I cannot
    seem to find anything newer that confirms if such functionality is
    really in the current kernel. Anybody has any idea?
    You probably want to inform the switch which ports are link ports and
    which ports are edge ports (that is, only hosts will use the port) to
    reduce the amount of work and therefore time needed.

    On 7/11/10, Jerry Franz wrote:
    On 7/10/2010 2:21 PM, Emmanuel Noobadmin wrote:
    I've been reading that it's possible to set up a system with multiple
    NIC to provide redundant internet connectivity such that it will
    switch to a secondary connection if the primary ISP fails.

    Is it possible in a similar way to setup redundant LAN routing? I read
    that it is possible to aggregate/bond multiple NIC to stackable
    switches that support link aggregation and redundancy. But if only
    simple switches are available, is something like this possible?

    e.g.
    System A
    eth0 -> lan switch/router 1
    eth1 -> lan switch/router 2

    System B
    eth0 -> lan switch 1
    eth1 -> lan switch 2

    Then somehow specify that, if lan switch 1 fails, the two systems will
    switch to using switch 2 so that in case of a switch failure, the
    network continues to remain operational.
    Yes. You can do it. I've done it before. All you need is the right
    choice of bonding mode . You set up bond0 for eth0 and eth1 and it 'just
    works'. To make it more robust, cross-connect the two switches as well.

    --
    Benjamin Franz
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos
  • Emmanuel Noobadmin at Jul 11, 2010 at 1:25 pm

    On 7/12/10, Chan Chung Hang Christopher wrote:
    You only really need STP when you have switches that are connected
    together in such a way as to have multiple paths. For the setup you
    first posted, you could just have two physically separate networks. That
    does leave the question of what solution to use to get the boxes to use
    the other switch if the primary one goes down. So if you connect both
    networks to make say a big 'circular' network, then you need STP.
    I think I must had made a mistake in my setup example because the
    multipath thing seems to be what I had in mind. i.e. all machines have
    at least 2 paths to the others so as long as one switch is alive,
    functionality remains.

    A more complete network diagram

    WAN
    Managed Switch/Router
    -> connects to Internet, for external connectivity

    Internal (for networked storage)
    Switch 1 (primary, 1Gbps dumb switch)
    -> connects to WAN Switch
    -> connects to Machine 1,2,3...X eth0

    Switch 2 (backup, 100Mbps dumb switch)
    -> connects to WAN Switch
    -> connects to Machine 1,2,3...X eth1

    Possibly a Switch 3 in the future if more data nodes are added than
    the primary have ports.

    Idea being that the dumb switches are used solely for local data
    transfer between up to X number of App servers and storage nodes. The
    managed switch then handles only external communications as well as
    any firewalling.

    Would connect bond0 to both switches still work without STP in this
    kind of a setup, or is this when STP comes in? Or is there a better
    network topology, given that I don't have the budget for awsome HP
    ProCurves ;) Reusing existing router/switch (DLink DFL-800) and dumb
    Gb switches.
  • Gordon Messmer at Jul 11, 2010 at 4:34 pm

    On 07/11/2010 10:25 AM, Emmanuel Noobadmin wrote:
    I think I must had made a mistake in my setup example because the
    multipath thing seems to be what I had in mind. i.e. all machines have
    at least 2 paths to the others so as long as one switch is alive,
    functionality remains.
    A critical element of your configuration involves whether or not your
    managed switch links the two unmanaged switches.

    If it does not, then there are two paths from any host to the other
    hosts and to the managed switch/router. One path will be eth0->switch
    1->destination host or router. The other path will be eth1->switch
    2->destination host or router. You'll need to use "active-bonding" mode
    in order for this to work.

    If it does link them and the two are not directly linked, you have
    mostly the same situation. Any switch or ethernet failure will continue
    to allow communication between hosts, using active-bonding.

    If you then link the two unmanaged switches together, there become an
    infinite number of paths between any two hosts. A packet can be sent
    from eth0->switch 1->switch 2->switch 1->...->destination. Broadcast
    packets *will*. That's why your network failed the last time you tried
    cross-connecting the switches. You can't do this with unmanaged
    switches. Not even with STP. STP is used when you have a mesh of
    managed switches to locate the shortest path between hosts and to handle
    the failure of a switch in the mesh.
    Would connect bond0 to both switches still work without STP in this
    kind of a setup, or is this when STP comes in?
    STP comes in when you're linking multiple managed switches in a topology
    other than daisy-chain or star. It has nothing to do with the
    configuration of network nodes which are not switches.
  • Chan Chung Hang Christopher at Jul 11, 2010 at 6:38 pm

    Idea being that the dumb switches are used solely for local data
    transfer between up to X number of App servers and storage nodes. The
    managed switch then handles only external communications as well as
    any firewalling.
    Oh you have dumb switches in the mix? Not going to work as Gordon has said.

    Would connect bond0 to both switches still work without STP in this
    kind of a setup, or is this when STP comes in? Or is there a better
    network topology, given that I don't have the budget for awsome HP
    ProCurves ;) Reusing existing router/switch (DLink DFL-800) and dumb
    Gb switches.
    You don't need HP ProCurves...unless you need good jumboframe support.
    You will need at least D-Link 3100 switches for what you want.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcentos @
categoriescentos
postedJul 10, '10 at 5:21p
activeJul 11, '10 at 6:38p
posts11
users7
websitecentos.org
irc#centos

People

Translate

site design / logo © 2022 Grokbase