FreeBSD Handbook
Prev | Chapter 32 Advanced Networking | Next

# 32.6 Link Aggregation and Failover

Written by Andrew Thompson.

## 32.6.1 Introduction

The lagg(4) interface allows aggregation of multiple network interfaces as one virtual interface for the purpose of providing fault-tolerance and high-speed links.

## 32.6.2 Operating Modes

Failover

  Sends and receives traffic only through the master port. If the master port becomes unavailable, the next active port is used. The first interface added is the master port; any interfaces added after that are used as failover devices. If failover to a non-master port occurs, the original port will become master when it becomes available again.

Cisco® Fast EtherChannel®

  Cisco Fast EtherChannel (FEC), is a static setup and does not negotiate aggregation with the peer or exchange frames to monitor the link. If the switch supports LACP then that should be used instead.

  FEC balances outgoing traffic across the active ports based on hashed protocol header information and accepts incoming traffic from any active port. The hash includes the Ethernet source and destination address, and, if available, the VLAN tag, and the IPv4/IPv6 source and destination address.

LACP

  The IEEE® 802.3ad Link Aggregation Control Protocol (LACP) and the Marker Protocol. LACP will negotiate a set of aggregable links with the peer in to one or more Link Aggregated Groups (LAG). Each LAG is composed of ports of the same speed, set to full-duplex operation. The traffic will be balanced across the ports in the LAG with the greatest total speed, in most cases there will only be one LAG which contains all ports. In the event of changes in physical connectivity, Link Aggregation will quickly converge to a new configuration.

  LACP balances outgoing traffic across the active ports based on hashed protocol header information and accepts incoming traffic from any active port. The hash includes the Ethernet source and destination address, and, if available, the VLAN tag, and the IPv4/IPv6 source and destination address.

Loadbalance

  This is an alias of FEC mode.

Round-robin

  Distributes outgoing traffic using a round-robin scheduler through all active ports and accepts incoming traffic from any active port. This mode violates Ethernet Frame ordering and should be used with caution.

## 32.6.3 Examples

Example 32-1. LACP Aggregation with a Cisco® Switch

This example connects two interfaces on a FreeBSD machine to the switch as a single load balanced and fault tolerant link. More interfaces can be added to increase throughput and fault tolerance. Since frame ordering is mandatory on Ethernet links then any traffic between two stations always flows over the same physical link limiting the maximum speed to that of one interface. The transmit algorithm attempts to use as much information as it can to distinguish different traffic flows and balance across the available interfaces.

On the Cisco switch add the `FastEthernet0/1` and `FastEthernet0/2` interfaces to the channel-group `1`:

    interface FastEthernet0/1
     channel-group 1 mode active
     channel-protocol lacp
    !
    interface FastEthernet0/2
     channel-group 1 mode active
     channel-protocol lacp

Create the lagg(4) interface using `fxp0` and `fxp1`, and bring the interfaces up with the IP Address of `10.0.0.3/24`:

    # ifconfig fxp0 up
    # ifconfig fxp1 up
    # ifconfig lagg0 create
    # ifconfig lagg0 up laggproto lacp laggport fxp0 laggport fxp1 10.0.0.3/24

View the interface status by running:

    # ifconfig lagg0

Ports marked as ACTIVE are part of the active aggregation group that has been negotiated with the remote switch and traffic will be transmitted and received. Use the verbose output of ifconfig(8) to view the LAG identifiers.

    lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
            options=8<VLAN_MTU>
            ether 00:05:5d:71:8d:b8
            media: Ethernet autoselect
            status: active
            laggproto lacp
            laggport: fxp1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
            laggport: fxp0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>

To see the port status on the switch, use `show lacp neighbor`:

    switch# show lacp neighbor
    Flags:  S - Device is requesting Slow LACPDUs
            F - Device is requesting Fast LACPDUs
            A - Device is in Active mode       P - Device is in Passive mode

    Channel group 1 neighbors

    Partner's information:

                      LACP port                        Oper    Port     Port
    Port      Flags   Priority  Dev ID         Age     Key     Number   State
    Fa0/1     SA      32768     0005.5d71.8db8  29s    0x146   0x3      0x3D
    Fa0/2     SA      32768     0005.5d71.8db8  29s    0x146   0x4      0x3D

For more detail use the `show lacp neighbor detail` command.

To retain this configuration across reboots, the following entries can be added to `/etc/rc.conf`:

    ifconfig_fxp0="up"
    ifconfig_fxp1="up"
    cloned_interfaces="lagg0"
    ifconfig_lagg0="laggproto lacp laggport fxp0 laggport fxp1 10.0.0.3/24"

Example 32-2. Failover Mode

Failover mode can be used to switch over to a secondary interface if the link is lost on the master interface. Bring the underlying physical interfaces up. Create the lagg(4) interface, using `fxp0` as the master interface and `fxp1` as the secondary interface and assign an IP Address of `10.0.0.15/24`:

    # ifconfig fxp0 up
    # ifconfig fxp1 up
    # ifconfig lagg0 create
    # ifconfig lagg0 up laggproto failover laggport fxp0 laggport fxp1 10.0.0.15/24

The interface will look something like this, the major differences will be the MAC address and the device names:

    # ifconfig lagg0
    lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
            options=8<VLAN_MTU>
            ether 00:05:5d:71:8d:b8
            inet 10.0.0.15 netmask 0xffffff00 broadcast 10.0.0.255
            media: Ethernet autoselect
            status: active
            laggproto failover
            laggport: fxp1 flags=0<>
            laggport: fxp0 flags=5<MASTER,ACTIVE>

Traffic will be transmitted and received on `fxp0`. If the link is lost on `fxp0` then `fxp1` will become the active link. If the link is restored on the master interface then it will once again become the active link.

To retain this configuration across reboots, the following entries can be added to `/etc/rc.conf`:

    ifconfig_fxp0="up"
    ifconfig_fxp1="up"
    cloned_interfaces="lagg0"
    ifconfig_lagg0="laggproto failover laggport fxp0 laggport fxp1 10.0.0.15/24"

Example 32-3. Failover Mode Between Wired and Wireless Interfaces

For laptop users, it is usually desirable to make wireless as a secondary interface, which is to be used when the wired connection is not available. With lagg(4), it is possible to use one IP address, prefer the wired connection for both performance and security reasons, while maintaining the ability to transfer data over the wireless connection.

In this setup, we will need to override the underlying wireless interface's MAC address to match the lagg(4)'s, which is inherited from the master interface being used, the wired interface.

In this setup, we will treat the wired interface, `bge0`, as the master, and the wireless interface, `wlan0`, as the failover interface. The `wlan0` was created from `iwn0` which we will set up with the wired connection's MAC address. The first step would be to obtain the MAC address from the wired interface:

    # ifconfig bge0
    bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    	options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
    	ether 00:21:70:da:ae:37
    	inet6 fe80::221:70ff:feda:ae37%bge0 prefixlen 64 scopeid 0x2
    	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
    	media: Ethernet autoselect (1000baseT <full-duplex>)
    	status: active

You can replace the `bge0` to match your reality, and will get a different `ether` line which is the MAC address of your wired interface. Now, we change the underlying wireless interface, `iwn0`:

    # ifconfig iwn0 ether 00:21:70:da:ae:37

Bring up the wireless interface but don't set up any IP address on it:

    # ifconfig wlan0 create wlandev iwn0 ssid my_router up

Bring the `bge0` interface up. Create the lagg(4) interface with `bge0` as master, and failover to `wlan0` if necessary:

    # ifconfig bge0 up
    # ifconfig lagg0 create
    # ifconfig lagg0 up laggproto failover laggport bge0 laggport wlan0

The interface will look something like this, the major differences will be the MAC address and the device names:

    # ifconfig lagg0
    lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
            options=8<VLAN_MTU>
            ether 00:21:70:da:ae:37
            media: Ethernet autoselect
            status: active
            laggproto failover
            laggport: wlan0 flags=0<>
            laggport: bge0 flags=5<MASTER,ACTIVE>

Then start the DHCP client to obtain an IP address:

    # dhclient lagg0

To retain this configuration across reboots, the following entries can be added to `/etc/rc.conf`:

    ifconfig_bge0="up"
    ifconfig_iwn0="ether 00:21:70:da:ae:37"
    wlans_iwn0="wlan0"
    ifconfig_wlan0="WPA"
    cloned_interfaces="lagg0"
    ifconfig_lagg0="laggproto failover laggport bge0 laggport wlan0 DHCP"

Prev | Home | Next
Bridging | Up | Diskless Operation