Monthly Archives: August 2015

Juniper EX Virtual Chassis Part 2

I’ve already written a post on how to create a Virtual Chassis by using the 1/10GB uplink modules. If you have a switch in production and want to add another switch for additional ports or redundancy, you can easily create a virtual chassis. This time I’ll be using the dedicated VC ports and cables and adding a new switch to a production switch.

I’ll be using the preprovisioned method, and before I do any virtual chassis configuration I’ll need to add some features to the master member to minimize failover times:

set system commit synchronize
set chassis redundancy graceful-switchover
set routing-options nonstop-routing
set ethernet-switching-options nonstop-bridging

Having added these features, we can now configure preprovisioned virtual chassis onto the master switch, which will become member 0. Because this is only a 2 member VC, I’ve added the no-split-detection command as recommended by Juniper, and to help with the failover times fast-failover on all ports ge/xe that have been enabled.

set virtual-chassis preprovisioned
set virtual-chassis no-split-detection
set virtual-chassis member 0 role routing-engine
set virtual-chassis member 0 serial-number BP0214340104
set virtual-chassis member 1 role routing-engine
set virtual-chassis member 1 serial-number BP0215090120
set virtual-chassis fast-failover ge
set virtual-chassis fast-failover xe

For now, that’s everything on the master member. On the new switch (member 1), you need to clear all config from the switch and set the root password to allow you to commit your changes:

root> edit 
Entering configuration mode
 
{master:0}[edit]
root# delete 
This will delete the entire configuration
Delete everything under this level? [yes,no] (no) yes 
{master:0}[edit]
root# set system root-authentication plain-text-password    
New password:
Retype new password:
root# commit 
configuration check succeeds
commit complete

You need to ensure there are no past virtual chassis configurations, and you can do this by entering the shell cli of the switch and removing anything in the vchassis folder:

root> start shell 
[email protected]:RE:0% rm -rf /config/vchassis/*
[email protected]:RE:0% cd /config/vchassis/
[email protected]:RE:0% ls -la
total 8
drwxr-xr-x  2 root  wheel  512 Sep 13 07:26 .
drwxr-xr-x  5 root  wheel  512 Sep 13 06:57 ..
[email protected]:RE:0% exit
exit

Now you will need to power off the backup member for at least a minute, to ensure that the other switch is elected as master.

After the minute, patch the VC-cable into the dedicated VCP-Ports at the back of the chassis and power on the backup switch. Once member 1 has booted you will be able to verify the new member by running: show virtual-chassis status

[email protected]> show virtual-chassis status     
 
Preprovisioned Virtual Chassis
Virtual Chassis ID: f1a1.ca8e.bbba
Virtual Chassis Mode: Enabled
                                           Mstr           Mixed Neighbor List
Member ID  Status   Serial No    Model     prio  Role      Mode ID  Interface
0 (FPC 0)  Prsnt    BP0214340104 ex4200-48t 129  Master*      N  1  vcp-0      
                                                                 1  vcp-1      
1 (FPC 1)  Prsnt    BP0215090120 ex4200-48t 129  Backup       N  0  vcp-0      
                                                                 0  vcp-1  

And you can verify the health of the VCP ports by running: show virtual-chassis vc-port

[email protected]> show virtual-chassis vc-port    
fpc0:
--------------------------------------------------------------------------
Interface   Type              Trunk  Status       Speed        Neighbor
or                             ID                 (mbps)       ID  Interface
PIC / Port
vcp-0       Dedicated           1    Up           32000        1   vcp-0  
vcp-1       Dedicated           2    Up           32000        1   vcp-1  
 
fpc1:
--------------------------------------------------------------------------
Interface   Type              Trunk  Status       Speed        Neighbor
or                             ID                 (mbps)       ID  Interface
PIC / Port
vcp-0       Dedicated           1    Up           32000        0   vcp-0  
vcp-1       Dedicated           2    Up           32000        0   vcp-1  
Share this:
Share

Configuring a 802.3ad Bonded Interface Ubuntu (NIC Teaming)

Messing about in the lab configuring 802.3ad LACP bundled interfaces between switches and I wanted to see how easy (or hard) it would be to create a bonded interface on a server. I’ve got an Ubuntu 14.04LTS VM and 3 NICs available, so eth1 and eth2 were told they will become one 😀

NOTE
Please make sure you are either doing this via ILO/KVM or have a management interface I like have, as you are making network changes and you could lock yourself out of your server, if it goes horribly wrong!

Let’s get cracking!

Firstly, I configured the switch as 802.3ad LACP aggregated interface and set the interfaces to apart of the aggregated interface:

{master:0}[edit interfaces]
[email protected]# show  
ge-0/0/2 {
    description "km-vm1 1GB";
    enable;
    ether-options {
        802.3ad ae1;
    }
}
ge-0/0/3 {
    description "km-vm1 eth2 1GB";
    enable;
    ether-options {
        802.3ad ae1;
    }
}
ae1 {
    aggregated-ether-options {
        lacp {
            active;                     
            periodic fast;
        }
    }
    unit 0 {
        family ethernet-switching {
            port-mode access;
            vlan {
                members v10;
            }
        }
    }
}

Server wise, check that the NICs can be configured as an 802.3ad bond, as when I’m using LACP method of bonding, you need to ensure that the NICs support ethtool.

By running ethtool {interface} , if a link is detected then you’re good to go:

[email protected]:~$ ethtool eth1
Settings for eth1:
	Supported ports: [ TP ]
	Supported link modes:   1000baseT/Full 
	                        10000baseT/Full 
	Supported pause frame use: No
	Supports auto-negotiation: No
	Advertised link modes:  Not reported
	Advertised pause frame use: No
	Advertised auto-negotiation: No
	Speed: 10000Mb/s
	Duplex: Full
	Port: Twisted Pair
	PHYAD: 0
	Transceiver: internal
	Auto-negotiation: off
	MDI-X: Unknown
Cannot get wake-on-lan settings: Operation not permitted
	Link detected: yes

[email protected]:~$ ethtool eth2
Settings for eth2:
	Supported ports: [ TP ]
	Supported link modes:   1000baseT/Full 
	                        10000baseT/Full 
	Supported pause frame use: No
	Supports auto-negotiation: No
	Advertised link modes:  Not reported
	Advertised pause frame use: No
	Advertised auto-negotiation: No
	Speed: 10000Mb/s
	Duplex: Full
	Port: Twisted Pair
	PHYAD: 0
	Transceiver: internal
	Auto-negotiation: off
	MDI-X: Unknown
Cannot get wake-on-lan settings: Operation not permitted
	Link detected: yes

I needed to install ifenslave package, as this package is used to attach and detach NICs to a bonding interface

sudo apt-get install ifenslave

Once that has been installed, the kernel module file needs to be edited to include bonding before creating a bonded interface:

sudo nano /etc/modules

# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
# Parameters can be specified after the module name.

lp
rtc
bonding

Once that is saved, manually load the module:

sudo modprobe bonding

Next edit the interfaces into a bond sudo nano /etc/network/interfaces

auto eth1
iface eth1 inet manual
    bond-master bond0

auto eth2
iface eth2 inet manual
    bond-master bond0

auto bond0
iface bond0 inet static
    # For jumbo frames, change mtu to 9000
    mtu 1500
    address 192.31.1.2
    netmask 255.255.255.0
    network 192.31.1.0
    broadcast 192.31.1.255
    gateway 192.31.1.1
    bond-miimon 100
    bond-downdelay 200 
    bond-updelay 200 
    bond-mode 4
    bond-slaves none
Bond Configuration Details
Bond-MiimonBond-DowndelayBond-UpdelayBond-ModeBond-Slaves
Specifies the MII link monitoring frequency in milliseconds. This determines how often the link state of each slave is inspected for link failures
Specifies the time, in milliseconds, to wait before disabling a slave after a link failure has been detected.
Specifies the time, in milliseconds, to wait before enabling a slave after a link recovery has been detected.
Specifies what mode of NIC bonding configured. There’s 7 mode:

  • Mode 0 – balance-rr
  • Mode 1 – active-backup
  • Mode 2 – balance-xor
  • Mode 3 – broadcast
  • Mode 4 – 802.3ad
  • Mode 5 – balance-tlb
  • Mode 6 – balance-alb

For more in-depth details on bonding modes and Linux Ethernet Bonding visit Kernel.org white paper documentation

Defines all the interfaces that will be in the bond. My example has none because I had defined them with bond-master

Save and Exit, then you need to do network restart or reboot the server for the change to take effect.

Once the reboot/restart has completed you should be sorted. You can check this by running the commands ifconfig

[email protected]:~$ ifconfig 
bond0     Link encap:Ethernet  HWaddr 00:0c:29:4f:26:c5  
          inet addr:192.31.1.2  Bcast:192.31.1.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe4f:26c5/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:150 errors:0 dropped:5 overruns:0 frame:0
          TX packets:446 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:14381 (14.3 KB)  TX bytes:53888 (53.8 KB)

eth0      Link encap:Ethernet  HWaddr 00:0c:29:4f:26:bb  
          inet addr:10.1.0.137  Bcast:10.1.0.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe4f:26bb/64 Scope:Link
          inet6 addr: 2001:41c1:4:8040:20c:29ff:fe4f:26bb/64 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:304 errors:0 dropped:0 overruns:0 frame:0
          TX packets:127 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:26921 (26.9 KB)  TX bytes:24900 (24.9 KB)

eth1      Link encap:Ethernet  HWaddr 00:0c:29:4f:26:c5  
          inet6 addr: fe80::20c:29ff:fe4f:26c5/64 Scope:Link
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:24 errors:0 dropped:1 overruns:0 frame:0
          TX packets:216 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:4155 (4.1 KB)  TX bytes:26653 (26.6 KB)

eth2      Link encap:Ethernet  HWaddr 00:0c:29:4f:26:c5  
          inet6 addr: fe80::20c:29ff:fe4f:26c5/64 Scope:Link
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:126 errors:0 dropped:4 overruns:0 frame:0
          TX packets:230 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:10226 (10.2 KB)  TX bytes:27235 (27.2 KB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:64 errors:0 dropped:0 overruns:0 frame:0
          TX packets:64 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:5696 (5.6 KB)  TX bytes:5696 (5.6 KB)

or cat /proc/net/bonding/bond0

[email protected]:~$ cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
	Aggregator ID: 1
	Number of ports: 2
	Actor Key: 33
	Partner Key: 2
	Partner Mac Address: cc:e1:7f:2b:82:80

Slave Interface: eth1
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:4f:26:c5
Aggregator ID: 1
Slave queue ID: 0

Slave Interface: eth2
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:4f:26:cf
Aggregator ID: 1
Slave queue ID: 0

By using cat /proc/net/bonding/bond0 you can also check if a link in the bond has failed as the Link Failure Count would increase.

And thats how you can configure 802.3ad Bonded Interface 🙂

Share this:
Share

Adding a Default & Static Route Ubuntu

I’m doing some testing using some Ubuntu server and I wasn’t able to ping across from other server I’ve got. I checked the network and it was fine, so I can checked the server and saw that the routing on the server was acting unexpcted :/

Checked the network file /etc/network/interfaces and that was fine

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp

auto eth1
iface eth1 inet static
address 192.31.3.2
netmask 255.255.255.0
gateway 192.31.3.1

However when I checked the routing table the default gateway was going via the management subnet

[email protected]:~$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.1.0.1        0.0.0.0         UG    0      0        0 eth0
10.1.0.0        0.0.0.0         255.255.255.0   U     0      0        0 eth0
192.31.3.0      0.0.0.0         255.255.255.0   U     0      0        0 eth1

[email protected]:~$ ip route 
default via 10.1.0.1 dev eth0 
10.1.0.0/24 dev eth0  proto kernel  scope link  src 10.1.0.140 
192.31.3.0/24 dev eth1  proto kernel  scope link  src 192.31.3.2

Needed to change this, to have the default gateway as 192.31.3.1. Firstly, I had to remove the current default gateway:

[email protected]:~$ sudo route del default

Verified that the route was removed, using ip route

[email protected]:~$ ip route
10.1.0.0/24 dev eth0  proto kernel  scope link  src 10.1.0.140 
192.31.3.0/24 via 192.31.3.1 dev eth1 
192.31.3.0/24 dev eth1  proto kernel  scope link  src 192.31.3.2

To add the new default route, I had to run the command route add default gw {IP address}

[email protected]:~$ sudo route add default gw 192.31.3.1

Then verified with ip route again

[email protected]:~$ ip route 
default via 192.31.3.1 dev eth1 
10.1.0.0/24 dev eth0  proto kernel  scope link  src 10.1.0.140 
192.31.3.0/24 via 192.31.3.1 dev eth1 
192.31.3.0/24 dev eth1  proto kernel  scope link  src 192.31.3.2

Now when I did a mtr to my other test server, routing is working as expected 🙂

                            My traceroute  [v0.85]
km-vm3 (0.0.0.0)                                       Mon Aug  3 10:37:15 2015
Resolver: Received error response 2. (server failure)er of fields   quit
                                       Packets               Pings
 Host                                Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 192.31.3.1                        0.0%     9    1.0   0.9   0.8   1.0   0.0
 2. 172.31.1.2                        0.0%     9    0.9   1.0   0.9   1.1   0.0
 3. 192.31.1.2                        0.0%     9    0.7   0.5   0.4   0.7   0.0

Throughout the troubleshooting (as I’m not a server man!) I did also figure out how to make add a static route as well! This is done by use the command: route add -net [subnet/mask] gw [IP address] dev [interface]

[email protected]:~$ sudo route add -net 192.31.3.0/24 gw 192.31.3.1 dev eth1

As you see the static route was added.

[email protected]:~$ ip route
default via 10.1.0.1 dev eth0 
10.1.0.0/24 dev eth0  proto kernel  scope link  src 10.1.0.140 
192.31.3.0/24 via 192.31.3.1 dev eth1 
192.31.3.0/24 dev eth1  proto kernel  scope link  src 192.31.3.2

Although this didn’t fix my issue, it was a useful thing to come across as it could be helpful in the future!!

Share this:
Share