Tag Archives: LACP

Juniper, Networking, Switching, Ubuntu

Configuring a 802.3ad Bonded Interface Ubuntu (NIC Teaming)

5 Comments

Messing about in the lab configuring 802.3ad LACP bundled interfaces between switches and I wanted to see how easy (or hard) it would be to create a bonded interface on a server. I’ve got an Ubuntu 14.04LTS VM and 3 NICs available, so eth1 and eth2 were told they will become one 😀

NOTE
Please make sure you are either doing this via ILO/KVM or have a management interface I like have, as you are making network changes and you could lock yourself out of your server, if it goes horribly wrong!

Let’s get cracking!

Firstly, I configured the switch as 802.3ad LACP aggregated interface and set the interfaces to apart of the aggregated interface:

{master:0}[edit interfaces]
[email protected]# show  
ge-0/0/2 {
    description "km-vm1 1GB";
    enable;
    ether-options {
        802.3ad ae1;
    }
}
ge-0/0/3 {
    description "km-vm1 eth2 1GB";
    enable;
    ether-options {
        802.3ad ae1;
    }
}
ae1 {
    aggregated-ether-options {
        lacp {
            active;                     
            periodic fast;
        }
    }
    unit 0 {
        family ethernet-switching {
            port-mode access;
            vlan {
                members v10;
            }
        }
    }
}

Server wise, check that the NICs can be configured as an 802.3ad bond, as when I’m using LACP method of bonding, you need to ensure that the NICs support ethtool.

By running ethtool {interface} , if a link is detected then you’re good to go:

[email protected]:~$ ethtool eth1
Settings for eth1:
	Supported ports: [ TP ]
	Supported link modes:   1000baseT/Full 
	                        10000baseT/Full 
	Supported pause frame use: No
	Supports auto-negotiation: No
	Advertised link modes:  Not reported
	Advertised pause frame use: No
	Advertised auto-negotiation: No
	Speed: 10000Mb/s
	Duplex: Full
	Port: Twisted Pair
	PHYAD: 0
	Transceiver: internal
	Auto-negotiation: off
	MDI-X: Unknown
Cannot get wake-on-lan settings: Operation not permitted
	Link detected: yes

[email protected]:~$ ethtool eth2
Settings for eth2:
	Supported ports: [ TP ]
	Supported link modes:   1000baseT/Full 
	                        10000baseT/Full 
	Supported pause frame use: No
	Supports auto-negotiation: No
	Advertised link modes:  Not reported
	Advertised pause frame use: No
	Advertised auto-negotiation: No
	Speed: 10000Mb/s
	Duplex: Full
	Port: Twisted Pair
	PHYAD: 0
	Transceiver: internal
	Auto-negotiation: off
	MDI-X: Unknown
Cannot get wake-on-lan settings: Operation not permitted
	Link detected: yes

I needed to install ifenslave package, as this package is used to attach and detach NICs to a bonding interface

sudo apt-get install ifenslave

Once that has been installed, the kernel module file needs to be edited to include bonding before creating a bonded interface:

sudo nano /etc/modules

# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
# Parameters can be specified after the module name.

lp
rtc
bonding

Once that is saved, manually load the module:

sudo modprobe bonding

Next edit the interfaces into a bond sudo nano /etc/network/interfaces

auto eth1
iface eth1 inet manual
    bond-master bond0

auto eth2
iface eth2 inet manual
    bond-master bond0

auto bond0
iface bond0 inet static
    # For jumbo frames, change mtu to 9000
    mtu 1500
    address 192.31.1.2
    netmask 255.255.255.0
    network 192.31.1.0
    broadcast 192.31.1.255
    gateway 192.31.1.1
    bond-miimon 100
    bond-downdelay 200 
    bond-updelay 200 
    bond-mode 4
    bond-slaves none
Bond Configuration Details
Bond-MiimonBond-DowndelayBond-UpdelayBond-ModeBond-Slaves
Specifies the MII link monitoring frequency in milliseconds. This determines how often the link state of each slave is inspected for link failures
Specifies the time, in milliseconds, to wait before disabling a slave after a link failure has been detected.
Specifies the time, in milliseconds, to wait before enabling a slave after a link recovery has been detected.
Specifies what mode of NIC bonding configured. There’s 7 mode:

  • Mode 0 – balance-rr
  • Mode 1 – active-backup
  • Mode 2 – balance-xor
  • Mode 3 – broadcast
  • Mode 4 – 802.3ad
  • Mode 5 – balance-tlb
  • Mode 6 – balance-alb

For more in-depth details on bonding modes and Linux Ethernet Bonding visit Kernel.org white paper documentation

Defines all the interfaces that will be in the bond. My example has none because I had defined them with bond-master

Save and Exit, then you need to do network restart or reboot the server for the change to take effect.

Once the reboot/restart has completed you should be sorted. You can check this by running the commands ifconfig

[email protected]:~$ ifconfig 
bond0     Link encap:Ethernet  HWaddr 00:0c:29:4f:26:c5  
          inet addr:192.31.1.2  Bcast:192.31.1.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe4f:26c5/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:150 errors:0 dropped:5 overruns:0 frame:0
          TX packets:446 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:14381 (14.3 KB)  TX bytes:53888 (53.8 KB)

eth0      Link encap:Ethernet  HWaddr 00:0c:29:4f:26:bb  
          inet addr:10.1.0.137  Bcast:10.1.0.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe4f:26bb/64 Scope:Link
          inet6 addr: 2001:41c1:4:8040:20c:29ff:fe4f:26bb/64 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:304 errors:0 dropped:0 overruns:0 frame:0
          TX packets:127 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:26921 (26.9 KB)  TX bytes:24900 (24.9 KB)

eth1      Link encap:Ethernet  HWaddr 00:0c:29:4f:26:c5  
          inet6 addr: fe80::20c:29ff:fe4f:26c5/64 Scope:Link
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:24 errors:0 dropped:1 overruns:0 frame:0
          TX packets:216 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:4155 (4.1 KB)  TX bytes:26653 (26.6 KB)

eth2      Link encap:Ethernet  HWaddr 00:0c:29:4f:26:c5  
          inet6 addr: fe80::20c:29ff:fe4f:26c5/64 Scope:Link
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:126 errors:0 dropped:4 overruns:0 frame:0
          TX packets:230 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:10226 (10.2 KB)  TX bytes:27235 (27.2 KB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:64 errors:0 dropped:0 overruns:0 frame:0
          TX packets:64 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:5696 (5.6 KB)  TX bytes:5696 (5.6 KB)

or cat /proc/net/bonding/bond0

[email protected]:~$ cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
	Aggregator ID: 1
	Number of ports: 2
	Actor Key: 33
	Partner Key: 2
	Partner Mac Address: cc:e1:7f:2b:82:80

Slave Interface: eth1
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:4f:26:c5
Aggregator ID: 1
Slave queue ID: 0

Slave Interface: eth2
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:4f:26:cf
Aggregator ID: 1
Slave queue ID: 0

By using cat /proc/net/bonding/bond0 you can also check if a link in the bond has failed as the Link Failure Count would increase.

And thats how you can configure 802.3ad Bonded Interface 🙂

Share this:
Share