Configuring a 802.3ad Bonded Interface Ubuntu (NIC Teaming)
Table of Contents
Messing about in the lab configuring 802.3ad LACP bundled interfaces between switches and I wanted to see how easy (or hard) it would be to create a bonded interface on a server. I’ve got an Ubuntu 14.04LTS VM and 3 NICs available, so eth1 and eth2 were told they will become one :D
Please make sure you are either doing this via ILO/KVM or have a management interface I like have, as you are making network changes and you could lock yourself out of your server, if it goes horribly wrong!
Let’s get cracking!
Configuring LACP on Juniper Switch⌗
Firstly, I configured the switch as 802.3ad LACP aggregated interface and set the interfaces to apart of the aggregated interface:
{master:0}[edit interfaces\]
root@EX4200-A# show
ge-0/0/2 {
description "km-vm1 1GB";
enable;
ether-options {
802.3ad ae1;
}
}
ge-0/0/3 {
description "km-vm1 eth2 1GB";
enable;
ether-options {
802.3ad ae1;
}
}
ae1 {
aggregated-ether-options {
lacp {
active;
periodic fast;
}
}
unit 0 {
family ethernet-switching {
port-mode access;
vlan {
members v10;
}
}
}
}
Server side Configuration⌗
Server wise, check that the NICs can be configured as an 802.3ad bond, as when I’m using LACP method of bonding, you need to ensure that the NICs support ethtool.
By running ethtool {interface} , if a link is detected then you’re good to go
marquk01@km-vm1:~$ ethtool eth1
Settings for eth1:
Supported ports: \[ TP \]
Supported link modes: 1000baseT/Full
10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: No
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Speed: 10000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
MDI-X: Unknown
Cannot get wake-on-lan settings: Operation not permitted
**Link detected: yes**
marquk01@km-vm1:~$ ethtool eth2
Settings for eth2:
Supported ports: \[ TP \]
Supported link modes: 1000baseT/Full
10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: No
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Speed: 10000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
MDI-X: Unknown
Cannot get wake-on-lan settings: Operation not permitted
**Link detected: yes**
I needed to install ifenslave package, as this package is used to attach and detach NICs to a bonding interface: sudo apt-get install ifenslave
Once that has been installed, the kernel module file needs to be edited to include bonding before creating a bonded interface: sudo nano /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
# Parameters can be specified after the module name.
lp
rtc
bonding
Once that is saved, manually load the module:
sudo modprobe bonding
Next edit the interfaces into a bond sudo nano /etc/network/interfaces
auto eth1
iface eth1 inet manual
bond-master bond0
auto eth2
iface eth2 inet manual
bond-master bond0
auto bond0
iface bond0 inet static
# For jumbo frames, change mtu to 9000
mtu 1500
address 192.31.1.2
netmask 255.255.255.0
network 192.31.1.0
broadcast 192.31.1.255
gateway 192.31.1.1
bond-miimon 100
bond-downdelay 200
bond-updelay 200
bond-mode 4
bond-slaves none
Bond Configuration Details⌗
Bond-Miimon⌗
Specifies the MII link monitoring frequency in milliseconds. This determines how often the link state of each slave is inspected for link failures
Bond-Downdelay⌗
Specifies the time, in milliseconds, to wait before disabling a slave after a link failure has been detected.
Bond-Updelay⌗
Specifies the time, in milliseconds, to wait before enabling a slave after a link recovery has been detected.
Bond-Mode⌗
Specifies what mode of NIC bonding configured. There’s 7 mode:
Mode 0: balance-rrMode 1: active-backupMode 2: balance-xorMode 3: broadcastMode 4: 802.3adMode 5: balance-tlbMode 6: balance-alb
For more in-depth details on bonding modes and Linux Ethernet Bonding visit Kernel.org white paper documentation
Bond-Slaves⌗
Defines all the interfaces that will be in the bond. My example has none because I had defined them with bond-master
Save and Exit, then you need to do network restart or reboot the server for the change to take effect.
Once the reboot/restart has completed you should be sorted. You can check this by running the commands ifconfig
marquk01@km-vm1:~$ ifconfig
bond0 Link encap:Ethernet HWaddr 00:0c:29:4f:26:c5
inet addr:192.31.1.2 Bcast:192.31.1.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe4f:26c5/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:150 errors:0 dropped:5 overruns:0 frame:0
TX packets:446 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:14381 (14.3 KB) TX bytes:53888 (53.8 KB)
eth0 Link encap:Ethernet HWaddr 00:0c:29:4f:26:bb
inet addr:10.1.0.137 Bcast:10.1.0.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe4f:26bb/64 Scope:Link
inet6 addr: 2001:41c1:4:8040:20c:29ff:fe4f:26bb/64 Scope:Global
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:304 errors:0 dropped:0 overruns:0 frame:0
TX packets:127 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:26921 (26.9 KB) TX bytes:24900 (24.9 KB)
eth1 Link encap:Ethernet HWaddr 00:0c:29:4f:26:c5
inet6 addr: fe80::20c:29ff:fe4f:26c5/64 Scope:Link
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:24 errors:0 dropped:1 overruns:0 frame:0
TX packets:216 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4155 (4.1 KB) TX bytes:26653 (26.6 KB)
eth2 Link encap:Ethernet HWaddr 00:0c:29:4f:26:c5
inet6 addr: fe80::20c:29ff:fe4f:26c5/64 Scope:Link
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:126 errors:0 dropped:4 overruns:0 frame:0
TX packets:230 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:10226 (10.2 KB) TX bytes:27235 (27.2 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:64 errors:0 dropped:0 overruns:0 frame:0
TX packets:64 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:5696 (5.6 KB) TX bytes:5696 (5.6 KB)
or cat /proc/net/bonding/bond0
marquk01@km-vm1:~$ cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0
802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad\_select): stable
Active Aggregator Info:
Aggregator ID: 1
Number of ports: 2
Actor Key: 33
Partner Key: 2
Partner Mac Address: cc:e1:7f:2b:82:80
Slave Interface: eth1
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:4f:26:c5
Aggregator ID: 1
Slave queue ID: 0
Slave Interface: eth2
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:4f:26:cf
Aggregator ID: 1
Slave queue ID: 0
By using cat /proc/net/bonding/bond0 you can also check if a link in the bond has failed as the Link Failure Count would increase.
And thats how you can configure 802.3ad Bonded Interface :)