Messing about in the lab configuring 802.3ad LACP bundled interfaces between switches and I wanted to see how easy (or hard) it would be to create a bonded interface on a server. I’ve got an Ubuntu 14.04LTS VM and 3 NICs available, so eth1 and eth2 were told they will become one 😀
Let’s get cracking!
Firstly, I configured the switch as 802.3ad LACP aggregated interface and set the interfaces to apart of the aggregated interface:
{master:0}[edit interfaces] [email protected]# show ge-0/0/2 { description "km-vm1 1GB"; enable; ether-options { 802.3ad ae1; } } ge-0/0/3 { description "km-vm1 eth2 1GB"; enable; ether-options { 802.3ad ae1; } } ae1 { aggregated-ether-options { lacp { active; periodic fast; } } unit 0 { family ethernet-switching { port-mode access; vlan { members v10; } } } }
Server wise, check that the NICs can be configured as an 802.3ad bond, as when I’m using LACP method of bonding, you need to ensure that the NICs support ethtool.
By running ethtool {interface} , if a link is detected then you’re good to go:
[email protected]:~$ ethtool eth1 Settings for eth1: Supported ports: [ TP ] Supported link modes: 1000baseT/Full 10000baseT/Full Supported pause frame use: No Supports auto-negotiation: No Advertised link modes: Not reported Advertised pause frame use: No Advertised auto-negotiation: No Speed: 10000Mb/s Duplex: Full Port: Twisted Pair PHYAD: 0 Transceiver: internal Auto-negotiation: off MDI-X: Unknown Cannot get wake-on-lan settings: Operation not permitted Link detected: yes [email protected]:~$ ethtool eth2 Settings for eth2: Supported ports: [ TP ] Supported link modes: 1000baseT/Full 10000baseT/Full Supported pause frame use: No Supports auto-negotiation: No Advertised link modes: Not reported Advertised pause frame use: No Advertised auto-negotiation: No Speed: 10000Mb/s Duplex: Full Port: Twisted Pair PHYAD: 0 Transceiver: internal Auto-negotiation: off MDI-X: Unknown Cannot get wake-on-lan settings: Operation not permitted Link detected: yes
I needed to install ifenslave package, as this package is used to attach and detach NICs to a bonding interface
sudo apt-get install ifenslave
Once that has been installed, the kernel module file needs to be edited to include bonding before creating a bonded interface:
sudo nano /etc/modules # /etc/modules: kernel modules to load at boot time. # # This file contains the names of kernel modules that should be loaded # at boot time, one per line. Lines beginning with "#" are ignored. # Parameters can be specified after the module name. lp rtc bonding
Once that is saved, manually load the module:
sudo modprobe bonding
Next edit the interfaces into a bond sudo nano /etc/network/interfaces
auto eth1 iface eth1 inet manual bond-master bond0 auto eth2 iface eth2 inet manual bond-master bond0 auto bond0 iface bond0 inet static # For jumbo frames, change mtu to 9000 mtu 1500 address 192.31.1.2 netmask 255.255.255.0 network 192.31.1.0 broadcast 192.31.1.255 gateway 192.31.1.1 bond-miimon 100 bond-downdelay 200 bond-updelay 200 bond-mode 4 bond-slaves none
- Mode 0 – balance-rr
- Mode 1 – active-backup
- Mode 2 – balance-xor
- Mode 3 – broadcast
- Mode 4 – 802.3ad
- Mode 5 – balance-tlb
- Mode 6 – balance-alb
For more in-depth details on bonding modes and Linux Ethernet Bonding visit Kernel.org white paper documentation
Save and Exit, then you need to do network restart or reboot the server for the change to take effect.
Once the reboot/restart has completed you should be sorted. You can check this by running the commands ifconfig
[email protected]:~$ ifconfig bond0 Link encap:Ethernet HWaddr 00:0c:29:4f:26:c5 inet addr:192.31.1.2 Bcast:192.31.1.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe4f:26c5/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:150 errors:0 dropped:5 overruns:0 frame:0 TX packets:446 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:14381 (14.3 KB) TX bytes:53888 (53.8 KB) eth0 Link encap:Ethernet HWaddr 00:0c:29:4f:26:bb inet addr:10.1.0.137 Bcast:10.1.0.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe4f:26bb/64 Scope:Link inet6 addr: 2001:41c1:4:8040:20c:29ff:fe4f:26bb/64 Scope:Global UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:304 errors:0 dropped:0 overruns:0 frame:0 TX packets:127 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:26921 (26.9 KB) TX bytes:24900 (24.9 KB) eth1 Link encap:Ethernet HWaddr 00:0c:29:4f:26:c5 inet6 addr: fe80::20c:29ff:fe4f:26c5/64 Scope:Link UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:24 errors:0 dropped:1 overruns:0 frame:0 TX packets:216 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4155 (4.1 KB) TX bytes:26653 (26.6 KB) eth2 Link encap:Ethernet HWaddr 00:0c:29:4f:26:c5 inet6 addr: fe80::20c:29ff:fe4f:26c5/64 Scope:Link UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:126 errors:0 dropped:4 overruns:0 frame:0 TX packets:230 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:10226 (10.2 KB) TX bytes:27235 (27.2 KB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:64 errors:0 dropped:0 overruns:0 frame:0 TX packets:64 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:5696 (5.6 KB) TX bytes:5696 (5.6 KB)
or cat /proc/net/bonding/bond0
[email protected]:~$ cat /proc/net/bonding/bond0 Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: IEEE 802.3ad Dynamic link aggregation Transmit Hash Policy: layer2 (0) MII Status: up MII Polling Interval (ms): 0 Up Delay (ms): 0 Down Delay (ms): 0 802.3ad info LACP rate: slow Min links: 0 Aggregator selection policy (ad_select): stable Active Aggregator Info: Aggregator ID: 1 Number of ports: 2 Actor Key: 33 Partner Key: 2 Partner Mac Address: cc:e1:7f:2b:82:80 Slave Interface: eth1 MII Status: up Speed: 10000 Mbps Duplex: full Link Failure Count: 0 Permanent HW addr: 00:0c:29:4f:26:c5 Aggregator ID: 1 Slave queue ID: 0 Slave Interface: eth2 MII Status: up Speed: 10000 Mbps Duplex: full Link Failure Count: 0 Permanent HW addr: 00:0c:29:4f:26:cf Aggregator ID: 1 Slave queue ID: 0
By using cat /proc/net/bonding/bond0 you can also check if a link in the bond has failed as the Link Failure Count would increase.
And thats how you can configure 802.3ad Bonded Interface 🙂