Border Gateway Protocol (BGP)

BGP Basics

What is BGP

Border Gateway Protocol (BGP) is regards as the most influential network protocols as it is backbone of the internet today. BGP is a Path Vector Routing Protocol, that unlike other routing protocols uses TCP (port 179, as its transport layer) to establish connectivity before exchanging routing information with another BGP speaker (peer). BGP communication can be done between same and/or different networks, these networks are known as Autonomous Systems (AS) with an  AS being a set of Routers that are managed by single entity, business and/or company. BGP uses routing information to maintain a BGP Routing Information Base (RIB) of Network Layer Reachability Information (NLRI) which it will exchange with other BGP peer or Peer ASs. BGP is a classless protocol, it can support any IP prefix regardless of class, this is both for IPv4 and IPv6. It is important to note that requires TCP connection first before building BGP connection, without that first established session a BGP peering never happen, however once that session is connected it will not have to made again unless a change is made. BGP uses Keepalive messages to ensure reliability of the session as it does not use any transport protocol-based keep-alive mechanism to determine if peers are reachable.

BGP Usage

BGP is largely (but not exclusively) used in large enterprises and data centre hosting environments where the need for single or multihomed to multiple Internet Service Providers (ISPs) connections are needed, this is known as Exterior BGP (eBGP). BGP is extensively used with Service Provider environments. BGP allows a large range of the policy based controls for an AS to influence and/or manipulate routed inbound and outbound traffic to help optimise the movement of traffic for their own needs. Additionally BGP can be used between BGP routers within the same AS to advertise internal routes with the same level of control as eBGP, with some small however important difference, this is known as Interior BGP (iBGP).

eBGP vs iBGP

There are some Key Differences between eBGP and iBGP that are important to note:

eBGP

  • eBGP session is between BGP peers with different AS numbers
  • Inter-AS communication is by via eBGP
  • eBGP respects the AS_Path Path Attribute
  • Routes learnt via eBGP will be advertised to other eBGP and iBGP peers

iBGP

  • iBGP session is between BGP peers with the same AS number
  • Intra-AS communication can be by via iBGP
  • iBGP commonly uses an IGP for network reachability and to establish BGP TCP session via Loopback address
  • Routes learnt via iBGP will not be advertised to other iBGP peers however will advertise routes to an eBGP peer

The above isn’t the full differences but just some of the main difference that need to remember. Additionally there are situations where some of these rules may need to be manipulated and can be done in design and/or configuration however that is for later

BGP Peering States

When establishing a BGP session there are 6 states that need to be completed before peering session comes up. The first 3 states are to ensure the TCP transport layer connectivity is there, once this has been completed then BGP connectivity is established with the final 3 states:

BGP State Connect Description
Idle TCP This is when all BGP connections will be refused. An Idle state occurs when the BGP session hasn’t been configured on the other BGP peer or BGP has isn’t enabled at all. Commonly, a start event is required from the other peer to prepare the TCP connectivity.
Connect TCP The router listening for TCP connections and is waiting for the TCP 3-way handshake to be completed:

  • If this is completed, then an Open message is sent and is transitioned into the OpenSent State.
  • If TCP connections fails, the BGP peer restarts the ConnectRetryTimer and waits for the remote peer to initiate a TCP connection and transitions into an Active State.
Active TCP This is when the BGP peer is trying to establish TCP connection.
OpenSent BGP When in the OpenSent, an open message has been sent by the BGP peer however has not received by the local peer:

  • Once the message has been received, checked and has no errors, the local peer will send a Keepalive message.
  • If a message is received, checked and an error is found then state is transitioned back to an Idle state.
OpenConfirm BGP When in the OpenConfirm, the BGP is waiting on a Keepalive or Notification message:

  • Once the peer receives a Keepalive message, it will move into the Established state
  • If the local peer does not receive a Keepalive within the negotiated session hold timer, it will send a Notification message and transition back to Idle state. The same will occur if the local peer sends out a Notification message.
Established BGP Having received the Keepalive message, the BGP session is fully Established. The peers are now able to exchange Update, Notification and Keepalive messages

BGP Message Types

As shown above, there are a number of different messages sent between BGP peers to Establish a session and when even the peering has been established, messages are used to ensure that both peers have synchronized routing information. BGP can only process a message after the entire message has been received, the maximum message size is 4096 bytes with 19 bytes being the smallest message size, this would be just be a header with no data. Each message type uses a fixed header size of 19 bytes with BGP Keepalives not include any data after the header, so they will always use the minimum size.

Each Message would be include the following:

Message Type Description
Open Once TCP connection has been completed both peers will send out an Open Message. This message starts the peering session, it provides details about the remote peer, in addition to details about supported and optional options.

These details are included:

  1. BGP version (normally version 4)
  2. AS number
  3. Hold Time
  4. Router ID
  5. Par-Len
    • If this set, it informs the peer that optional parameters should be expected
  6. Optional Parameters
    • This is where negotiable parameters are indicated, these would be authentication and capability extension such as Multiprotocol Extensions and route refresh.
Update An Update Message sends a list of new, withdrawn or types of routes from the remote peer. Depending on the routing policy of remote peer these may or not be entered into the Routing Table.

These details are included:

  1. Unfeasible Route Length
    • If this is set, it will tell the peer, the length of withdraw routes
  2. Withdrawn Routes
    • Lists IP prefixes that have been removed as they are no longer deemed as reachable
  3. Path Attribute Length
    • This indicates the total length of Path Attributes field. Its value allows the length of the Network Layer Reachability field to be determined. A value of 0 determines that neither Path Attribute and NLRI is present in the update.
  4. Path Attribute
    • The following properties for a route is included:
      1. Origin
      2. AS Path
      3. Next-Hop
      4. Multi-Exit Discriminator (MED)
      5. Local Preference
  5. Network Length Reachability Information (NLRI)
    • Lists IP prefixes that will be advertised as reachable via the AS
Keepalive It is important to always remember that Keepalive Messages are not used to ensure the TCP connection between peers is kept. They are used to ensure that BGP Hold Timers do not expire keeping alive the route exchange.
Notification Notification Message is used to inform a peer that there is an error with the BGP session.

There are 6 Error code numbers:

  1. Message Header Error
  2. Open Message Error
  3. Update Message Error
  4. Hold Timer Expired
  5. Finite State Machine Error
  6. Cease

In addition to 17 Sub Error codes (6 Open Message Errors and 11 Update Message Errors). These can found in RFC4271

Refresh Normally BGP can not readvertise routes that have already been acknowledged by a peer, if the BGP peer has been configured to soft clear of BGP sessions then peers will be able to exchange Refresh Messages. Some vendors you have to explicitly configure this, in Cisco you need to configure soft-reconfiguration whereas with Juniper it is set by default within JunOS.

BGP Attributes

Unlike other Routing Protocols, BGP primary function is to find the best path to a destination and not the shortest path. BGP uses a number of attributes to calculate the best path for any given destination prefix. These attributes can be broken down into 4 types:

Well Known Attribute Types
Well known Mandatory These attributes must be known and understood by all BGP speakers. Additionally must exist within the BGP update messages.

Attributes classed as Well Known Attributes:

  • Origin
  • AS Path
  • Next-Hop
Well known Optional These attributes must be known and understood by all BGP speakers. However they don’t have to exist within a BGP update message.

Attributes classed as Well Known Optional Attributes:

  • Local Preference
  • Atomic Aggregator
Optional BGP Attribute Types
Optional Transitive Attributes don’t need to be understood by a BGP speaker however the set flag(s) will need to be passed onto other neighbours.

Attributes classed as Optional Transitive:

  • Aggregator
  • Community
  • Extended Community
Optional Non-Transitive These attributes don’t need to be understood by a BGP speaker and the set flag(s) will not be passed onto other neighbours.

  • Multi Exit Discriminator
  • Originator ID
  • Cluster List
  • Multiprotocol Reachable NLRI
  • Multiprotocol Unreachable NLRI

A BGP Update message could include some, if not all, of the following attributes:

 
Message Information
 Origin (Attribute Code 1) The Origin Attribute confirms the source of the route aka where the route was learnt from. The Origin of a route can either be:

  1. I: Internal (0) The Route is learnt from IGP

  2. E: External (1) The Route is learnt from EGP

  3. ?: Incomplete (2) The Route is learnt by something that isn’t by Internal or External methods

The rule used for Origin is that: Internal is better than External which is better than Incomplete

 AS Path (Attribute Code 2) AS Path is a list of AS numbers that are between the source AS router to the our own AS. The AS Path is primary usages are to prevent Routing Loops, assist in the Path Selection and Policy Based Routing (PBR). BGP router will drop any routes received where it can see its own AS number within the AS Path this is how Routing Loops are prevented. The path enables the router to make policy decisions based on the presence of certain AS’s within the path. Additionally routes with a shorter AS Path are preferred over routes with longer AS Path
Next-Hop (Attribute Code 3) This Attribute contains the IP address of the BGP peer that advertises the route. The Next-Hop is used for reachability and reliable of for the BGP session. For eBGP it is usually the peering address associated with the physical link with another AS. iBGP works differently as you can have situations where due to rules with iBGP the next-hop address isn’t reachable due to learning the route from another iBGP peer, in this situation the Next-Hop can be changed by policy.
Multi Exit Discriminator (Attribute Code 4) Multi Exit Discriminator (MED) is used when there are more than one route to the same upstream AS. The route with the lowest MED value is always preferred by default.
Local Preference (Attribute Code 5) Local Preference is an important attribute as it is the first attribute evaluated in the Path Selection Process. Local Preference is used for Infra-AS traffic communications for BGP session. As the name, suggests is only used to influence traffic within an AS. Oddly BGP prefers routes with the Highest Local Preference.
Atomic Aggregator (Attribute Code 6) Atomic Aggregator attribute is a notification that tells other BGP speakers within the AS-Path that some information has been lost and/or changed due to route aggregation. This may affect the best path selection because a less specific route was selected over more specific route.  
Aggregator (Attribute Code 7) Aggregator attribute is set when an advertised route has been aggregated. This attribute contains the AS number and Router-ID of the Router that has performed the aggregation
Communities (Attribute Code 8) Community attribute is tag that is use to modify, filter and/or influence a common group of IP Prefix(es) to act in a user defined way. Communities uses 4-octets of space to represent its value. Communities are used in conjunction with PBR. A community is 32-bit value, that is common defined as AS/IP-address:User-defined ie 100:1 or 192.168.100.1:1. 100 would be the AS or 192.168.100.1 being the device loopback address with 1 being a value significant within AS100.
Originator ID (Attribute Code 9) Originator attribute is a loop prevention mechanism used within iBGP network using a Route Reflector. The Route Reflector attaches if own Router-ID to routes, so if it receives a route with its own Router-ID it will ignore the route.
Cluster List (Attribute Code 10) Cluster List similar to the Originator ID attribute is a loop prevention mechanism however if an iBGP network is used clustered set of Route Reflectors then routes have the Route Reflectors Cluster ID attached to the advertised routes.
Multi-Protocol Reachable NLRI (Attribute Code 14) Multi-Protocol Reachable NLRI has two main functions as defined in RFC 4760:

  1. Negotiates what non IPv4 unicast families will be announced between two BGP peers.

  2. Defines the Network Layer Address of the router that should be the next-hop of the destination families. Ie if you have advertised l2vpn bgp family the next-hop for this bgp family will be defined within this attribute.

When this attribute is used in a BGP Update message, the Origin and AS Path attributes have to be included. Local Preference attribute is additionally added to Update messages for iBGP peering sessions.

Multi-Protocol Unreachable NLRI (Attribute Code 15) Multi-Protocol Unreachable NLRI attribute is used to withdraw any BGP families that are no longer being advertised between BGP peers.
Extended Communities (Attribute Code 16) Extended Communities are the same as Community attribute however it has 8 octets of space to represent the community compared to 4 octets with normal communities. This allows 64-bit value, it can be represented as Type:Global-Administrator:Local-Administrator. It is important to note that you have set amount of bits you can use. You will have 16 bits for the Type, 16 bits, for the Global-Administrator (commonly the ASN/IP address) and 32 bits, for the Local-Administrator (commonly user defined).   

BGP Path Selection

When a destination prefix reached by multiple routes via BGP by default only one path will be advertised into the Routing Table. With this in mind BGP has used its Route Selection Algorithm to determine what path will be installed into the Routing Table. The algorithm uses the following steps:

  • Prefer the highest Local Preference Value
  • Checks what path has shortest AS Path
  • The Route with the Lowest Origin Value
  • If the route has a Lower MED
  • If the Prefix is learnt via eBGP is preferred over being learnt via iBGP
  • The path with the better exit out of the local AS. This means that the underlying IGP metric cost is taken into consideration, the path with the lowest IGP is preferred
  • The eBGP route that has the longest uptime or prefer the routes from the peer with lowest Router ID
  • Prefer routes with the shortest Cluster List Length. This is when you use a Route Reflector within your iBGP peering session
  • Prefer routes from a peer with the lowest IP Address

Some vendors have their own vendor specific additions to the path selection algorithm. Cisco use Weight before checking Local Preference and Juniper verify that the Next-Hop is reachable before checking Local Preference. With JunOS, if the Next-Hop isn’t verified then the route is set as a Hidden route and will need investigating.

Resetting Admin Password & Factory Reset a Nokia IP390 via CLI

So this is going to be quick post, as recently i had to decommission a Nokia IP390 Checkpoint firewall, where i had no details for the device 😀

Fortunately I was on site and had my trusty console cable! So this is going to be a reminder to myself on: How to do a password reset and factory reset of a Nokia IP390 via CLI

Let’s get cracking 🙂

Password Reset

Firstly, you need to have console access and the ability to reboot the device.

You will need to enter single user mode. Reboot the device and as it is going through it boot process, you will need to look for Type any character to enter command mode, once you see that hit enter and type boot -s to enter single user mode.

1,072,300,032 bytes of system memory tested OK
Starting bootmgr
Loading boot manager..
Boot manager loaded.
diskless platform
Entering autoboot mode.
Type any character to enter command mode.
BOOTMGR[1]> boot -s

You will be prompted with Enter pathname of shell or RETURN for sh: Just hit enter.

Enter pathname of shell or RETURN for sh:

With that you will be in Single-User Mode now, as you will have # prompt.

To reset the admin password, you need to run the overpw script: /etc/overpw

# /etc/overpw
    This program is used to set a temporary admin password when you have 
    lost the configured password.  You must have booted the machine into 
    single user mode to run it.  The configured password will be changed.
    Please change the temporary password as soon as you log on to your
    system through voyager.

Please enter password for user admin: 
Please re-enter password for confirmation: 
Continue? [n] y

You will be prompted with:

Admin password changed.  You may enter ^D to continue booting.  
    THIS IS A TEMPORARY PASSWORD CHANGE.
    PLEASE USE VOYAGER TO CREATE A PERMENANT PASSWORD FOR THE USER ADMIN.

As instructed hit ctrl + D and the booting process will continue. With that the admin password will be reset

Factory Reset

Now we have got admin access to the device, to do a factory reset is pretty noddy, to be honest. The IP390 is UNIX based cli, as shown by the single-user mode, firstly su into root:

Nokia[admin]# su root

Next, we need to change directory into the config folder:

Nokia[admin]# ls
bin     cdrom   dev     image   proc    tmp     var
bootmgr config  etc     opt     sbin    usr     web
Nokia[admin]# cd config

Delete all files the active files:

Nokia[admin]# ls
active  db
Nokia[admin]# rm active
Nokia[admin]# ls
db

Once all the files have been deleted, all that is needed is to reboot the device for the change to take affect:

Nokia[admin]# reboot

With that Bob’s your uncle, Sally’s your aunty and you have a decommissioned Nokia IP390 😀

What is BGP FlowSpec?

I recently messed about with some Junos Automate Scripts that one of my colleagues had previously been working on, that could be used to add static routes to enable Remote Triggered Blackhole (RTBH) Filtering (which can be found here), and I found it was a bit rough around the edges (for people who aren’t cli junkies). As I do, I started looking into RTBH and saw that it’s a heavy-handed solution in trying to combat DDoS attacks against a network. RTBH technology has been around for a number of years now and has been defined in RFC 3882 and RFC 5635. In its most basic of terms, you can either blackhole all traffic from a source address and/or to a destination address by injecting the attacking/attacked prefix into BGP with a community that will rewrite the next-hop to a pre-configured discard route on edge routers. If you have massive DDoS trying to block every source address, it would be like going fishing with a shotgun. By blocking the destination address the attacker will have got their desired outcome. With that in mind, using RTBH is ideally a last resort solution. There is an alternative more subtle way of blocking unwanted attack traffic from our network. This alternative method is known as BGP FlowSpec.

What is BGP FlowSpec

BGP FlowSpec is defined in RFC 5575. RFC 5575 defines a new Multi-Protocol BGP Extension MP-BGP, in addition, with new Network Layer Reachability Information NLRI. The new NLRI collects 12 types of Layer 3 and Layer 4 details that are used to define a Flow Specification then actions are assigned to these routes dependant on the user’s needs. If you wanted to look at FlowSpec in a simple form, it is a firewall filter that is injected into BGP to filter out specific port(s) and protocol(s) just as a normal ACL would do. BGP uses NLRI to exchange routing details between BGP speakers, each of the MP-BGP Extensions have their own NLRI details that are identified by their Address Family Indicator AFI and Subsequent Address Family Indicator AFI. Usually IPv4 unicast routes (also known as BGP families) are the default for BGP peers, if non IPv4 unicast routes need to be exchanged ie IPv6, EVPN, L2VPN, FlowSpec routes, then MP-BGP defines the relevant NLRI of the router that should have the next-hop of the destination families. This had been defined in RFC 2858 and RFC 4760. As stated above, as of writing, there has been 12 NLRI types defined for BGP FlowSpec, these fields will be added to NLRI field within the BGP Update Message and advertised to peers. In addition, FlowSpec does not support IPv6 yet.

FlowSpec NLRI Types

These are the 12 FlowSpec NLRI types:

Type NLRI Component
1 Destination Prefix
Defines the destination prefix to match
2 Source Prefix
Defines the source prefix
3 IP Protocol
Contains a set of {operator, value} pairs that are used to match the IP protocol value byte in IP packets.
4 Port
This is defines whether TCP, UDP or both will be packets will be influenced
5 Destination Port
Defines the destination port that will be influenced by FlowSpec
6 Source Port
Defines the source port that will be influenced by FlowSpec
7 ICMP Type
8 ICMP Code
9 TCP flags
10 Packet Length
Match on the total IP packet length (excluding Layer 2 but including IP header)
11 DSCP
Match on the Class Of Service flag
12 Fragment Encoding

NOTE: Not all 12 types have to be defined for FlowSpec to be enabled

FlowSpec Actions

RFC 5575 has defined 4 minimum Actions that routes matching FlowSpec NRLI types can take. These actions are carried as BGP extended communities added to the FlowSpec route. These actions are:

Traffic-Rate Community

The Traffic-Rate community is non-transitive, that tells the receiving BGP peer, what to rate limit matching traffic to. If the traffic needs to be discarded or dropped, this will be limit of 0 should be used.

Traffic-Action Community

The Traffic-Action community is used to sample defined traffic. This allows sampling and logging metrics to be collected from the FlowSpec route, that could be used to get a better understand of the attack traffic.

Redirect Community

The Redirect community allows the FlowSpec traffic to be redirected into a Virtual Routing and Forward Instance VRF. As the same Route-Targets and Route-Distinguisher can be used, you are able to import routes into a dedicated blackhole VPN or any other VPNv4.

Traffic-Marking Community

The Traffic-Marking community is used to modify the Differentiated Service Code Point DSCP bits of a transiting IP packet to the defined value. This could be used to set to FlowSpec routes to highest discard probability, allowing traffic not to dropped/discarded until co

FlowSpec Rule Ordering

It is important to note, that unlike normal firewall filters, FlowSpec routes use a different method of ordering rules. Most firewall filters and/or ACLs use the top-down approach, where in, once the filter has a match any other rules afterward are not inspected. With FlowSpec a deterministic algorithm to order the rules is used. By comparing the left component of each FlowSpec NLRI, the algorithm will use the following details to order FlowSpec Routes:

    1. If the types differ, the lowest type is used. If the types are the same then component values within that component are compared
    2. For IP values, the lowest IP prefix is chosen. If the IP addresses are the same then most specific prefix is used
    3. For all other types, the binary string of the contents is compared to determine the order

Validation Checks

Validate checks within FlowSpec are important, because you could get into a situation where, if no validation checks are done, FlowSpec route(s) could be injected by an attacker that doesn’t own a set of prefix(es) that could blackhole traffic. Like any other unicast BGP route, the next-hop address must resolve for the route to be usable, as per the normal BGP path selection process. In addition, to a valid next-hop, RFC 5775 has defined the follow must be valid of a Flow Specification:

    1. The originator of the flow specification matches the originator of the best-match unicast route for the destination prefix embedded in the flow specification.
    2. There are no more specific unicast routes, when compared with the flow destination prefix, that have been received from a different neighbouring AS than the best-match unicast route, which has been determined in step 1

The overall goal is to confirm that the originator of the FlowSpec route is the same as the originator of the BGP unicast route, this is done by either using BGP’s AS Path attribute or if that isn’t present (in iBGP situation) then the Peering IP address is used.

FlowSpec and Junos

Configuring FlowSpec on a JunOS device is actually quite straightforward. I’m being naughty and I don’t actually have a topology set up to show the full verification ‘show command’ outputs on the cli, but when I get the time to set something up, I’ll be back to edit this post. With all that said, Let’s getting cracking :p

The scenario is that we have an attack from 172.90.87.15 on TCP port 80 to the web-server 8.9.0.1. First we will inject a FlowSpec route to discard all TCP port 80 traffic to 8.9.0.1 when the source is from 172.90.87.15. We will need to make sure that we can order the terms as per the RFC requirement, this is done under the show routing-options flow stanza:

marquk01@TestMX480# show routing-options flow                       
term-order standard;

Then enable MP-BGP family flow to BGP group

marquk01@TestMX480# show protocols bgp group test 
type internal;
family inet {
    unicast;
    flow

Next configure the FlowSpec Route under routing-options flow route stanza:

[edit routing-options flow route test]
marquk01@TestMX480# show 
match {
    destination 8.9.0.1/32;
    source 172.90.87.15/32;
    protocol tcp;
    port 80;
}
then discard;

With these are the options available under match and then flags. You will note that they are largely the same flags that were stated in the RFC

Match FlagsThen Flags
[edit routing-options flow]
marquk01@TestMX480# set route test match ?  
Possible completions:
+ apply-groups         Groups from which to inherit configuration data
+ apply-groups-except  Don't inherit configuration data from these groups
  destination          Destination prefix for this traffic flow
+ destination-port     Destination TCP/UDP port
+ dscp                 Differentiated Services (DiffServ) code point (DSCP) (0-63)
+ fragment             
+ icmp-code            ICMP message code
+ icmp-type            ICMP message type
+ packet-length        Packet length (0-65535)
+ port                 Source or destination TCP/UDP port
+ protocol             IP protocol value
  source               Source prefix for this traffic flow
+ source-port          Source TCP/UDP port
+ tcp-flags            TCP flags
[edit routing-options flow]
marquk01@TestMX480# set route test then ?                          
Possible completions:
  accept               Allow traffic through
+ apply-groups         Groups from which to inherit configuration data
+ apply-groups-except  Don't inherit configuration data from these groups
  community            Name of BGP community
  discard              Discard all traffic for this flow
  next-term            Continue the filter evaluation after matching this flow
  rate-limit           Rate in bits/sec to limit the flow traffic (9600..1000000000000)
  routing-instance     Redirect to instance identified via Route Target community
  sample               Sample traffic that matches this flow

Once committed you will be able to verify Flowspec routes because they are installed into their own routing table inetflow.0 and if dedicated, VRF for FlowSpec routes and the table will be under routing-instance-name.inetflow.0. You can also check FlowSpec firewall filter by running the command show firewall filter __flowspec_default_inet__

FlowSpec TableFlowSpec Firewall Filter
marquk01@TestMX480> show route table inetflow.0 extensive 

inetflow.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
8.9.0.1,172.90.87.15,proto=6,port=80/term:3 (1 entry, 1 announced)
TSI:
KRT in dfwd;
Action(s): discard,count
        *Flow   Preference: 5
                Next hop type: Fictitious
                Address: 0x94359c4
                Next-hop reference count: 6
                State: 
                Local AS: 65123 
                Age: 4:10 
                Validation State: unverified 
                Task: RT Flow
                Announcement bits (1): 0-Flow 
                AS path: I
                Communities: traffic-rate:0:0
marquk01@TestMX480> show firewall filter __flowspec_default_inet__    

Filter: __flowspec_default_inet__                              
Counters:
Name                                                Bytes              Packets
8.9.0.1,172.90.87.15,proto=6,port=80                    0                    0

Layer-2 VPNs on Junos

It has been a busy few weeks trying to stay ahead of all the new work that has been coming towards myself and the team, due to the in sourcing of the core network! Lucky enough for my team, we have finally got our hands onto full end-to-end connectivity! Fun times 😀

With that being said, I’ve been given a wee project to provision a circuit for a business customer between two sites for a Proof Of Concept. As this circuit is being using as a POC (for now), it was agreed that a Layer 2 VPN (L2VPN/pseudowire) will be best suited, because a simple point-to-point connection was needed between two PEs. As we have a MPLS enabled network, it was decided that would be the easiest way to get their POC up and running quickly, as we were under a bit of a hard deadline!

For me, it was good little project, even though I know what L2VPNs were and how they work, I had never configured one myself. You see where I’m going with this now?

This post will over note how to configure L2VPN with Junos 😀

L2VPN, also known as a pseudowire, is defined in RFC4665, where they are called Virtual Private Wire Service (VPWS):

The PE devices provide a logical interconnect such that a pair of CE devices appears to be connected by a single logical Layer 2 circuit. PE devices act as Layer 2 circuit switches. Layer 2 circuits are then mapped onto tunnels in the SP network. These tunnels can either be specific to a particular VPWS, or be shared among several services. VPWS applies for all services, including Ethernet, ATM, Frame Relay, etc. Each PE device is responsible for allocating customer Layer 2 frames to the appropriate VPWS and for proper forwarding to the intended destinations.

In essence, L2VPNs are virtual point-to-point circuit that use the underlying Transport Labels (LDP/RSVP) or a statically defined MPLS path to go between two PE’s, that allows the extension of a layer 2 broadcast domain. If you need multiple sites on the same layer 2 broadcast you will need to consider Virtual Private Lan Service (VPLS) or Ethernet VPN (EVPN).

Within Junos there are 3 ways of configuring L2VPNs, two are regarded as modern way and has been rectified with RFC’s with an additional legacy method. Kompella and Martini are regarded as the industry standard, with Circuit Cross-Connect (CCC) seen as legacy:

  • Circuit Cross-Connect: The Circuit Cross-Connect style of L2VPN uses a single Outer Label, also known as the Tunnel/Transport Label, to transport L2 payload from PE to PE. CCC can ONLY use RSVP as MPLS transport, in addition each CCC connection has its own dedicated RSVP-signalled LSP associated, the transport label cannot be shared between multiple connections. LSPs are manually created on each PE to determines which circuit the frame belongs to on the other end.
  • Martini: The Martini style of L2VPN has a pair of labels before the L2 frame. The Outer label is the transport mechanism that allows the frame from egress interface from the sending PE to ingress interface of the receiving PE. The Inner label, known as the VC Label, is the label that informs the receiving PE, where the L2VPN payload should go. It is important to note that if you are using the Martini style, although either LDP or RVSP can be used MPLS transport, that LDP is used for the signalling of the VC label. So if the RSVP is used as the MPLS transport, LDP will need to be enabled on the loopback address of both PE routers. A minimum of 2 LSPs will need to be set, as MPLS LSPs are unidirectional.
  • Kompella: The Kompella style of L2VPN is similar to Martini style as both use stacked labels before the Layer 2 payload and both can use LDP, RSVP or both as Transport Label. There difference comes in that unlike Martini, Kompella uses BGP signalling as its VC Label. This means you will need to have BGP enabled network, in addition, it’s not compulsory to send static LSPs as BGP provides a mechanism for autodiscovery of new point-to-point links similar to a VPLS. Although Kompella has a more complex configuration, because of its usage of BGP signalling it is regarded as the best option for large scale deployments as it will in-conjunction with other BGP families. RFC6624 has more details on L2VPN using BGP for Auto-Discovery and Signaling

In our network, we use the Kompella style of L2VPNs. The bulk and most depth of my testing was with that method… Although I was able to get a wee bit of naughty time after to configure the other methods 🙂

The topology I’ll be working with is a simple one. I’ve a got a single MX480 broken up into 3 Logical Systems.

L2VPN Topology


The underlying IGP is IS-IS with RSVP, LDP and BGP enabled. This is a mirror, of what we have in production. With all the L2VPNs the customer facing physical interface has to be set to the correct encapsulation. For my testing, as I wont be using VLANs, Bridging or Setting a VPLS. I used ethernet-ccc and had set the logical interface to family ccc, you can find out more about the different physical encapsulations here

Interface ConfigRSVPMPLSBGPIS-ISLDP
set interfaces xe-0/1/0 enable
set interfaces xe-0/1/0 encapsulation ethernet-ccc
set interfaces xe-0/1/0 unit 0 family ccc
set protocols rsvp interface xe-1/0/0.0
set protocols rsvp interface xe-1/0/2.0
set protocols mpls explicit-null
set protocols mpls ipv6-tunneling
set protocols mpls no-decrement-ttl
set protocols mpls interface xe-1/0/0.0
set protocols mpls interface xe-1/0/2.0
set protocols bgp group Master type internal
set protocols bgp group Master local-address 192.168.2.1
set protocols bgp group Master family inet unicast
set protocols bgp group Master family inet6 unicast
set protocols bgp group Master local-as 100
set protocols bgp group Master neighbor 192.168.2.2 
set protocols bgp group Master neighbor 192.168.2.3
set protocols isis reference-bandwidth 1000g
set protocols isis level 1 disable
set protocols isis level 2 wide-metrics-only
set protocols isis interface xe-1/0/0.0 ldp-synchronization
set protocols isis interface xe-1/0/0.0 point-to-point
set protocols isis interface xe-1/0/0.0 link-protection
set protocols isis interface xe-1/0/2.0 ldp-synchronization
set protocols isis interface xe-1/0/2.0 point-to-point
set protocols isis interface xe-1/0/2.0 link-protection
set protocols isis interface xe-1/0/3.0 ldp-synchronization
set protocols isis interface xe-1/0/3.0 point-to-point
set protocols isis interface xe-1/0/3.0 link-protection
set protocols isis interface lo0.0
sset protocols ldp track-igp-metric
set protocols ldp explicit-null
set protocols ldp transport-address router-id
set protocols ldp interface xe-1/0/0.0
set protocols ldp interface xe-1/0/2.0
set protocols ldp interface lo0.0

All configurations will be done on the Master and SiteA, and for my examples I will show work done on the Master Instance. With all that out of the way… Let’s get cracking 😀

Kompella

As stated before, BGP is used as the VPN signalling method, with that in mind, we will need to enable layer-2 signalling within MP-BGP. This is simply done by adding the command family l2vpn signaling with the BGP stanza. This can be added globally within BGP or under the specific neighbour.

set protocols bgp group Master family l2vpn signaling

With the signalling sorted we can go straight into the configuration of the L2VPN. Just like L3VPNs, L2VPNs configuration is done within the routing-instance stanza and uses the same parameters as L3VPN by having Route Distinguisher (RD) and Route-Target/vrf-target (RT). The RD has to be unique per device with RT matching on all devices within the L2VPN, this is important, so that traffic can be routed accordingly per site. In addition, routing-instance has to be set to l2vpn and the interface(s) have to be defined within the routing-instance as well.

set routing-instances Master instance-type l2vpn
set routing-instances Master interface xe-0/1/0.0
set routing-instances Master route-distinguisher 100:0001
set routing-instances Master vrf-target target:100:0000

Next the properties for that site within the L2VPN will need to configured under protocol l2vpn within the routing-instance. The encapsulation has to match all site that want to participate within the VPN. The Site identifier must be unique to the entire site within the L2VPN as the site ID is used to compute label values for site-to-site communications. The interface(s) have to be defined within l2vpn and l2vpn site stanzas.

set routing-instances Master protocols l2vpn encapsulation-type ethernet
set routing-instances Master protocols l2vpn interface xe-0/1/0.0
set routing-instances Master protocols l2vpn site Master site-identifier 1
set routing-instances Master protocols l2vpn site Master interface xe-0/1/0.0
Full Kompella Configuration

Verification

The primary command that will be used to check the status of a pseudowire would be show l2vpn connections. As Komplella signalling uses BGP, we will be able to do a show bgp summary and see a route being advertised within the l2vpn and routing instance tables show route table Master.l2vpn.0 or show route table bgp.l2vpn.0 respectfully. Additionally we will be able to mpls.0 table to confirm that the L2VPN incoming label and interface(s) for the pseudowire have made the routing table, by using show route table mpls.0.

Show l2vpn Connectionsshow bgp summaryshow route table Master.l2vpn.0show route table mpls.0
marquk01@TestMX480> show l2vpn connections    
Layer-2 VPN connections:

Legend for connection status (St)   
EI -- encapsulation invalid      NC -- interface encapsulation not CCC/TCC/VPLS
EM -- encapsulation mismatch     WE -- interface and instance encaps not same
VC-Dn -- Virtual circuit down    NP -- interface hardware not present 
CM -- control-word mismatch      -> -- only outbound connection is up
CN -- circuit not provisioned    <- -- only inbound connection is up
OR -- out of range               Up -- operational
OL -- no outgoing label          Dn -- down                      
LD -- local site signaled down   CF -- call admission control failure      
RD -- remote site signaled down  SC -- local and remote site ID collision
LN -- local site not designated  LM -- local site ID not minimum designated
RN -- remote site not designated RM -- remote site ID not minimum designated
XX -- unknown connection status  IL -- no incoming label
MM -- MTU mismatch               MI -- Mesh-Group ID not available
BK -- Backup connection	         ST -- Standby connection
PF -- Profile parse failure      PB -- Profile busy
RS -- remote site standby	 SN -- Static Neighbor
LB -- Local site not best-site   RB -- Remote site not best-site
VM -- VLAN ID mismatch

Legend for interface status 
Up -- operational           
Dn -- down

Instance: Master
  Local site: Master (1)
    connection-site           Type  St     Time last up          # Up trans
    2                         rmt   Up     Jun  4 12:36:46 2016           2
      Remote PE: 192.168.2.2, Negotiated control-word: Yes (Null)
      Incoming label: 800001, Outgoing label: 800000
      Local interface: xe-0/1/0.0, Status: Up, Encapsulation: ETHERNET

marquk01@TestMX480> show bgp summary 
Groups: 1 Peers: 2 Down peers: 0
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
inet.0               
                       0          0          0          0          0          0
inet6.0              
                       0          0          0          0          0          0
bgp.l2vpn.0          
                       1          1          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
192.168.2.2             100       3234       3229       0       1  1d 0:19:17 Establ
  inet.0: 0/0/0/0
  inet6.0: 0/0/0/0
  Master.l2vpn.0: 1/1/1/0
  bgp.l2vpn.0: 1/1/1/0
192.168.2.3             100       5735       5724       0       1 1d 19:06:59 Establ
  inet.0: 0/0/0/0
  inet6.0: 0/0/0/0
  Master.l2vpn.0: 0/0/0/0
  bgp.l2vpn.0: 0/0/0/0/

marquk01@TestMX480> show route table Master.l2vpn.0 

Master.l2vpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

100:1:1:1/96                
                   *[L2VPN/170/-101] 1d 20:37:12, metric2 1
                      Indirect
100:2:2:1/96                
                   *[BGP/170] 00:01:02, localpref 100, from 192.168.2.2
                      AS path: I, validation-state: unverified
                    > to 192.168.1.14 via xe-1/0/0.0, Push 0
                      to 192.168.1.6 via xe-1/0/2.0, Push 300000

marquk01@TestMX480> show route table mpls.0 protocol l2vpn    

mpls.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

800001             *[L2VPN/7] 23:30:09
                    > via xe-0/1/0.0, Pop       Offset: 4
xe-0/1/0.0         *[L2VPN/7] 00:06:20, metric2 100
                    > to 192.168.1.14 via xe-1/0/0.0, Push 800000 Offset: 252
                      to 192.168.1.6 via xe-1/0/2.0, Push 800000, Push 300000(top) Offset: 252

From the end host point of view, we have end-to-end connectivity 😀

marquk01@km-vm2:~$ ping -c 2 -q 192.168.137.3
PING 192.168.137.3 (192.168.137.3) 56(84) bytes of data.

--- 192.168.137.3 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.431/0.637/0.843/0.206 ms
Note
The route given from the show route table Master.l2vpn.0 is the Route Distinguisher of the other end of the pseudowire

Martini

Martini signalling uses LDP, as stated before, and with LDP enabled already, I will focus on the actual configuration, which is done within the protocol l2circuit stanza. Compared to Kompella, the configuration for Martini style of L2VPNs is much simpler. All that is needed is for:

  • The remote neighbour to be defined. In my example I will be using the loopback address SiteA as the remote neighbour
  • The customer facing interface connecting into the VPN
  • Set a circuit ID, that must match on both sides

All this can be done in one line!

set protocols l2circuit neighbor 192.168.2.2 interface xe-0/1/0.0 virtual-circuit-id 1

With that we have Martini style L2VPN configured 🙂

Verifications

To check the status of Martini style L2VPN, you will use show l2circuit connections, the output is near enough the same as show l2vpn connections. Martini, as discussed above, uses LDP for the signalling, we will be able to use show ldp neighbor to check that the neighbour relationship with the remote side has been successful and we will be able to check the LDP database by using show ldp database to verify that new labels associated with the pseudowire (L2CKT) has been installed into the database. Additionally you can check the inet.3 and mpls.0 routing tables, by using show route table inet.3 & show route table mpls.0

Show l2circuit Connectionsshow ldp neighborshow ldp databaseshow route table inet.3show route table mpls.0
marquk01@TestMX480> show l2circuit connections 
Layer-2 Circuit Connections:

Legend for connection status (St)   
EI -- encapsulation invalid      NP -- interface h/w not present   
MM -- mtu mismatch               Dn -- down                       
EM -- encapsulation mismatch     VC-Dn -- Virtual circuit Down    
CM -- control-word mismatch      Up -- operational                
VM -- vlan id mismatch		 CF -- Call admission control failure
OL -- no outgoing label          IB -- TDM incompatible bitrate 
NC -- intf encaps not CCC/TCC    TM -- TDM misconfiguration 
BK -- Backup Connection          ST -- Standby Connection
CB -- rcvd cell-bundle size bad  SP -- Static Pseudowire
LD -- local site signaled down   RS -- remote site standby
RD -- remote site signaled down  HS -- Hot-standby Connection
XX -- unknown

Legend for interface status  
Up -- operational            
Dn -- down                   
Neighbor: 192.168.2.2 
    Interface                 Type  St     Time last up          # Up trans
    xe-0/1/0.0(vc 1)          rmt   Up     Jun  5 14:03:37 2016           1
      Remote PE: 192.168.2.2, Negotiated control-word: Yes (Null)
      Incoming label: 300000, Outgoing label: 300016
      Negotiated PW status TLV: No
      Local interface: xe-0/1/0.0, Status: Up, Encapsulation: ETHERNET
      Flow Label Transmit: No, Flow Label Receive: No

marquk01@TestMX480> show ldp neighbor      
Address            Interface          Label space ID         Hold time
192.168.2.2        lo0.0              192.168.2.2:0            43
192.168.1.6        xe-1/0/2.0         192.168.2.3:0            14
192.168.1.14       xe-1/0/0.0         192.168.2.2:0            13

marquk01@TestMX480> show ldp database                             
Input label database, 192.168.2.1:0--192.168.2.2:0
  Label     Prefix
 299984      192.168.2.1/32
      0      192.168.2.2/32
 300000      192.168.2.3/32
 300016      L2CKT CtrlWord ETHERNET VC 1

Output label database, 192.168.2.1:0--192.168.2.2:0
  Label     Prefix
      0      192.168.2.1/32
 299968      192.168.2.2/32
 299984      192.168.2.3/32
 300000      L2CKT CtrlWord ETHERNET VC 1

Input label database, 192.168.2.1:0--192.168.2.3:0
  Label     Prefix
 300016      192.168.2.1/32
 300000      192.168.2.2/32
      0      192.168.2.3/32

Output label database, 192.168.2.1:0--192.168.2.3:0
  Label     Prefix
      0      192.168.2.1/32
 299968      192.168.2.2/32
 299984      192.168.2.3/32

marquk01@TestMX480> show route table inet.3 192.168.2.2 

inet.3: 3 destinations, 4 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.2.2/32     *[LDP/9] 1d 21:16:06, metric 100
                    > to 192.168.1.14 via xe-1/0/0.0, Push 0
                      to 192.168.1.6 via xe-1/0/2.0, Push 300000
                    [RSVP/10/1] 1d 01:14:11, metric 100
                    > to 192.168.1.6 via xe-1/0/2.0, label-switched-path to-siteA

marquk01@TestMX480> show route table mpls.0 protocol l2circuit 

mpls.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

300000             *[L2CKT/7] 00:05:13
                    > via xe-0/1/0.0, Pop       Offset: 4
xe-0/1/0.0         *[L2CKT/7] 00:05:13, metric2 100
                    > to 192.168.1.14 via xe-1/0/0.0, Push 300016 Offset: 252
                      to 192.168.1.6 via xe-1/0/2.0, Push 300016, Push 300000(top) Offset: 252

From the end host point of view, connectivity between the two is there 🙂

marquk01@km-vm2:~$ ping -c 2 -q 192.168.137.3
PING 192.168.137.3 (192.168.137.3) 56(84) bytes of data.

--- 192.168.137.3 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.358/0.532/0.707/0.176 ms

Circuit Cross-Connect

As CCC doesn’t support stacked labels unlike Kompella and Martini, we will need to configure 2 static LSPs between the PE routers. CCC needs to have a LSP for to transmit and another to receive traffic. So firstly, we will need to get the LSPs configured. The received LSP will be configured on the remote PE, so under protocols mpls label-switched-path stanza, this is where we will define the LSP. I've used the loopback address of the remote end with the underlying IGP working out the best path.

set protocols mpls label-switched-path to-siteA to 192.168.2.2
set protocols mpls label-switched-path to-siteA no-cspf

With the LSPs configured, we will need to go under the protocol connections stanza. We need to define the customer facing interface(s) that will be connecting into the VPN, then set the transmit LSP and receive LSP, this will be the name of the LSP set on the remote end.

set protocols connections remote-interface-switch siteA interface xe-0/1/0.0
set protocols connections remote-interface-switch siteA transmit-lsp to-siteA
set protocols connections remote-interface-switch siteA receive-lsp to-Master

With that we are sorted!

Verifications

In regards with CCC there's less show commands, from what I’ve found (let me know if there's more please), but we can check the pseudowire's status by using show connections. We can confirm the Transmit (Ingress) and Receive (Egress) LSP using show mpls lsp and finally, we will be able to mpls.0 table to confirm that the L2VPN incoming label and interface(s) for the pseudowire have made the routing table, by using show route table mpls.0.

Show Connectionsshow mpls lspshow route table mpls.0
marquk01@TestMX480> show connections 
CCC and TCC connections [Link Monitoring On]
Legend for status (St):             Legend for connection types:
 UN -- uninitialized                 if-sw:  interface switching
 NP -- not present                   rmt-if: remote interface switching
 WE -- wrong encapsulation           lsp-sw: LSP switching
 DS -- disabled                      tx-p2mp-sw: transmit P2MP switching
 Dn -- down                          rx-p2mp-sw: receive P2MP switching
 -> -- only outbound conn is up     Legend for circuit types:
 <- -- only inbound  conn is up      intf -- interface
 Up -- operational                   oif  -- outgoing interface
 RmtDn -- remote CCC down            tlsp -- transmit LSP
 Restart -- restarting               rlsp -- receive LSP


Connection/Circuit                Type        St      Time last up     # Up trans
siteA                             rmt-if      Up      Jun  3 12:42:55           1
  xe-0/1/0.0                        intf  Up
  to-siteA                          tlsp  Up
  to-Master                         rlsp  Up

marquk01@TestMX480> show mpls lsp                           
Ingress LSP: 1 sessions
To              From            State Rt P     ActivePath       LSPname
192.168.2.2     192.168.2.1     Up     0 *     to-siteA         to-siteA
Total 1 displayed, Up 1, Down 0

Egress LSP: 1 sessions
To              From            State   Rt Style Labelin Labelout LSPname 
192.168.2.1     192.168.2.2     Up       0  1 FF  300080        - to-Master
Total 1 displayed, Up 1, Down 0

Transit LSP: 0 sessions
Total 0 displayed, Up 0, Down 0

marquk01@TestMX480> show route table mpls.0 protocol ccc    

mpls.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

300080             *[CCC/7] 00:00:04
                    > via xe-0/1/0.0, Pop      
xe-0/1/0.0         *[CCC/10/1] 00:00:04, metric 100
                    > to 192.168.1.14 via xe-1/0/0.0, label-switched-path to-siteA

Finally to confirm end-to-end reachability between the end hosts

marquk01@km-vm2:~$ ping -c 2 -q 192.168.137.3
PING 192.168.137.3 (192.168.137.3) 56(84) bytes of data.

--- 192.168.137.3 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.358/0.532/0.707/0.176 ms

I had planned to have a wee bit more to this post, with what I was actually testing ,however, this is getting a bit longer than I expected, so I'll make this into a two-part 😉

My next post will detail, how you can use traffic engineering to manipulate a L2VPN path between 2 PE routers! Hope to see you there 😀

References

Darren's Blog L2VPN in Junos
RFC4665
MPLS l2VPN
RFC6624
RFC6074
Vlan based CCC L2vpn

Configuring TACACS+ Server on Ubuntu 14.04LTS

It’s all change in the office so far this year, which is quite good as I’m involved in more projects, and who doesn’t enjoy a few projects 😉

The latest thing I was asked to look into was to create a new TACACS+ server as our current server on a HP Proliant BL460c G1 Blade is going to be decommissioned so we need to give it a new home! It was decided that it should be virtualized as there isn’t a need to have a physical server for something that can be slimmed down dramatically. With that being said this post will go over how to configure a TACACS+ server and configure TACACS+ authentication on a Juniper device.

TACACS+ is an improvement on its first version TACACS, as TACACS+ is an entirely new protocol and is not compatible with its predecessors, TACACS and XTACACS. TACACS+ uses TCP. Since TACACS+ uses the authentication, authorisation, and accounting (AAA) architecture, these separate components of the protocol can be segregated and handled on separate servers. TACACS+ allows you to set granular access policies for users and groups, commands, location, subnet, or even device type. The TACACS+ protocol also provides detailed logging of users and what commands have been run on specific devices. In addition, the protocol can run on either Windows or UNIX/Linux.

Although TACACS+ was developed by Cisco Systems, it is actually an open standard as defined by RFC1482 and has been incorporated into a number of different vendors including Alcatel/Lucent, Arbor, Brocade/Foundry, Cisco/Linksys, Extreme, HP/3Com, Huawei, IBM, Juniper/Netscreen, Netgear and any others.

The setup I had for testing was a simple one; I had 2 EXSi Ubuntu 14.04LTS hosts, one as the TACACS+ server with the second being used as Jump-box to access a Juniper SRX220 that will be configured for TACACS authentication.

With all that talk out of the way, let’s get cracking 🙂

You will run sudo/root privileges

Server Configuration

Fortunately, with the newer version of Ubuntu, from apt-get repository you can easily download the tacacs+ package it will also install libtacacs+1

marquk01@km-vm4:~$ sudo apt-get install tacacs+
[sudo] password for marquk01: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  libtacacs+1
The following NEW packages will be installed
  libtacacs+1 tacacs+

Having installed the package now we can run the command ps -ef | grep tac_plus and it will show us the location of the configuration file and if the process is running:

marquk01@km-vm4:~$ ps -ef | grep tac_plus
root      1220     1  0 11:37 ?        00:00:00 /usr/sbin/tac_plus -C /etc/tacacs+/tac_plus.conf
marquk01 22730  2682  0 13:55 pts/0    00:00:00 grep --color=auto tac_plus

As the process is running there’s a few useful binary files that are important to know, these can be seen when you type tac and hit TAB.

marquk01@km-vm4:~$ tac
tac  tac_plus  tac_pwd

The important files are tac_plus and tac_pwd:

  • tac_plus is the TACACS+ daemon. You can run daemon via the cli
  • tac_pwd is used to generate a Data Encryption Standard (DES) or Message-Digest 5 (MD5) hash from clear text. DES is the defualt, to generate a MD5 hash you need to add -m flag.

We will need to configure the tac_plus.conf file, but firstly we will need to back-up the original file to refer back to if there is any issues

marquk01@km-vm4:~$ sudo cp /etc/tacacs+/tac_plus.conf /etc/tacacs+/tac_plus.conf.old

I’ll explain from top-down of what my file looks like. The default file has more parameters than I used, as my file doesn’t need too much complexity. My example will also show you how to configure the basis Accounting, Secret Key, Users and Groups. Logically when I look at the layout of the file as I have, it doesn’t make sense… However, all the information is there soooooo it doesn’t matter :p lol

Accounting

Firstly we’ll need to set the file that the accounting information will be written to. By default this is /var/log/tac_plus.acct, however you can have this file where you like if you don’t want you use the default file and path.

You have to create this file yourself. This can be done by running the command sudo touch /var/log/tac_plus.acct

# Created by Henry-Nicolas Tourneur(henry.nicolas@tourneur.be)
# See man(5) tac_plus.conf for more details

# Define where to log accounting data, this is the default.

accounting file = /var/log/tac_plus.acct

Secret Key

The Server and Client need to have a matching key so the AAA packets can be encrypted. This key can be anything you wish however, if you’re going to have a key with white-space, key-words, or special characters, you’ll need to use quotation marks

# This is the key that clients have to use to access Tacacs+

key = testing123

Users

You’ll need to define the users that will have access to the device. Each user needs to be associated to a group and have their password defined. The password has to be set as either a MD5 or DES hash. By using tac_pwd use can get your hashed output:

marquk01@km-vm4:~$ tac_pwd
Password to be encrypted: lab123
kBeC6JDjU8icY

There is an additional stanza service = junos-exec that defines an additional group. This is Juniper specific and I’ll explain this later. I created two users kmarquis; will have permission to do anything and second usertest; that will only have Read-Only access. Both have the same password. Usernames ARE case sensitive.

# We also can define local users and specify a file where data is stored.
# That file may be filled using tac_pwd
user = kmarquis {
    name = "Keeran Marquis"
    member = admin
    login = des kBeC6JDjU8icY
		service = junos-exec {
			local-user-name = remote-admin
	}
}

user = test {
    name = "Test User"
    member = read-only
    login =  des kBeC6JDjU8icY
        service = junos-exec {
            local-user-name = remote-read-only
               }
}

Groups

As you can guess, groups are where you define the level of access and what commands will be used by the group. The commands, for my example, are used to define actions that are largely accepted by most vendors with the expectation of Juniper (from my knowledge but correct me if I’m wrong), although I wont be confirming the configuration works in this post. I have checked with a Cisco device and they worked as expected.

We have a few parameters that are important remember:

  • default service: defines the default permission that the user will have. By default, if this statement isn’t used or left blank, it’s denied. Meaning that each permitted command users of this group will have to be listed. If you want the default permission to allow, then the statement permit is needed
  • service: define services which the group is authorised to execute, these could be commands that the group is authorised to execute. Authorisation must be configured on both the client and the daemon to operate correctly.
  • cmd: This is where you list a command and set an action, it will be either be a permit or deny. Additionally by having the .* this means that any command after the first word is affected. i.e my example below, all show commands will be permitted

In my example I have two groups, admin and read-only, the admin group will have full access permitted and the read-only group, as the name suggests, will have read-only access and will be denied from any configuration, clear or restart commands.

# We can also specify rules valid per group of users.
group = admin {
	default service = permit
	service = exec {
		priv-lvl = 15
		}
	}

group = read-only {
	service = exec {
		priv-lvl = 15
		}
	cmd = show {
		permit .*
		}
	cmd = write {
		permit term
		}
	cmd = dir {
		permit .*
		}
	cmd = admin {
		permit .*
		}
	cmd = terminal {
		permit .*
		}
	cmd = more {
		permit .*
		}
	cmd = exit {
		permit .*
		}
	cmd = logout {
		permit .*
		}
}

My completed tac_plus file can be seen here.

Note
For more in-depth detail and additional parameters that can be configured in this file, you can find them via the man pages using the command man tac_plus or online Ubuntu tac_plus Manual Documentation

Once you’re happy with everything you can run service tacacs_plus check to make sure the syntax is correct and if you get any errors you will need to restart the daemon using service tacacs_plus restart

TACACS+ Daemon Commands

With that we have a TACACS+ server configured 🙂

Before getting into the configuration of the SRX, I stated earlier that there’s a Juniper Specific stanza in tac_plus.conf file. When authenticating users against a TACACS+ server on juniper devices and you’ll need to apply Juniper Networks Vendor-Specific TACACS+ Attributes.

These attributes can be either:

  1. Specified in the tac_plus.conf file by using regular expressions to list all the commands that the user has permitted or denied. A user will need to be created on the device with that user being referred under the local-user-name statement. The stanza would look something:
    service = junos-exec {
    	local-user-name = xxx
    	allow-commands =  .*
    	allow-configurations = .*
    	deny-commands = 
    	deny-configuration = 
    	user-permissions = 
    	}
  2. Configure a class that has states all the permitted or denied permissions, this class will be linked to a user. Both need to be configured on the device. Once this has been created you’ll need to refer, said user, under the local-user-name

The Junos OS retrieves these attributes through an authorization request of the TACACS+ server after authenticating a user. For my example, I went with the latter. Now we’ll jump onto the SRX220 and get that sorted with TACACS+ AAA configuration.

Juniper Configuration

Firstly, you will have to set the TACACS+ server with its secret key. For standard practice and force of habit, I have set the single connection and forced the source-address of the SRX. By using the single connection statement, this means that instead of multiple TCP sessions connecting to the device from a server, a single session is maintained between them. In addition, for best practice an authentication order should be set so that if there was an issue or loss of connectivity to the TACACS+ server, you’ll be able to fall back to locally defined users.

authentication-order [ tacplus password ];
tacplus-server {
    10.1.0.148 {
        secret "$9$SszyMXVb2aGiYgi.fzCAIEcyvWX7-w24"; ## SECRET-DATA
        single-connection;
        source-address 10.1.0.158;
    }
}

With the TACACS+ server we’re able log different events that take place on the device and get those commands sent to the server. From my experience the accounting events that you would most want logged are logins, configuration changes and interactive commands. This is set under system accounting stanza

accounting {
    events [ login change-log interactive-commands ];
    destination {
        tacplus;
    }
}

Next, under the system login stanza, you need to create a class that has a list of permission available to the user(s) that are going to be associated to it. The user(s) are what are used in the tac_plus.conf file. In my example I created two classes, one with all permission super-user-local and the other user with read-only and basic troubleshooting options (ie ping, traceroute, telnet etc) read-only-user-local. These associated this classes with 2 users remote-admin and remote-read-only

login {
    class read-only-user-local {
        permissions [ network view view-configuration ];
    }
    class super-user-local {            
        permissions all;
    }
    user remote {
        full-name "TACACS User";
        uid 2001;
        class super-user-local;
    }
    user remote-read-only {
        full-name "TACACS read-only user";
        uid 2002;
        class read-only-user-local;
    }
}
NOTE
You can learn more about the different permissions flags available here on Juniper TechLibrary

Verifications

To confirm the configuration is working as expected, I will ssh onto the SRX220 with both the admin user kmarquis and the read-only user test. With both users, I will log in and try to configure the description This is a test on a random port. As you can see below I had no problem with user kmarquis. However, when I logged in with the test user I wasn’t able to enter the configuration mode as the permission wasn’t granted, and for that user the command isn’t even recognized. I ran a show command and you will see that none of the passwords are shown. Again this is due to the permission level granted.

Admin AccessRead Only Access
marquk01@km-vm1:~$ ssh 10.1.0.158 -l kmarquis
Password: 
--- JUNOS 12.1X47-D30.4 built 2015-11-13 14:16:02 UTC
kmarquis@v6-testing> configure 
Entering configuration mode
[edit]
kmarquis@v6-testing# set interfaces ge-0/0/5 description "This is a test" 

[edit]
kmarquis@v6-testing# commit and-quit 

kmarquis@v6-testing>
marquk01@km-vm1:~$ ssh 10.1.0.158 -l test
Password: 
--- JUNOS 12.1X47-D30.4 built 2015-11-13 14:16:02 UTC
test@v6-testing> configure
                 ^
unknown command.

test@v6-testing> show configuration 
## Last commit: 2016-02-01 12:56:23 UTC by kmarquis
version 12.1X47-D30.4;
system {
    host-name v6-testing;
    authentication-order [ tacplus password ];
    root-authentication {
        encrypted-password /* SECRET-DATA */; ## SECRET-DATA
    }

If we check the /var/log/tac_plus.acct file we’ll be able to see all the permitted commands by each user. This is additional confirmation that the users have successfully authenticated against the TACACS+ server and their related permissions authorised to the device.

Feb  1 12:55:38 10.1.0.158      kmarquis        ttyp0   10.1.0.137      start   task_id=1       service=shell   process*mgd[38808]      cmd=login
Feb  1 12:55:41 10.1.0.158      kmarquis        ttyp0   10.1.0.137      stop    task_id=2       service=shell   process*mgd[38808]      cmd=show configuration 
Feb  1 12:55:44 10.1.0.158      kmarquis        ttyp0   10.1.0.137      stop    task_id=3       service=shell   process*mgd[38808]      cmd=edit 
Feb  1 12:56:01 10.1.0.158      kmarquis        ttyp0   10.1.0.137      stop    task_id=4       service=shell   process*mgd[38808]      cmd=set: [interfaces ge-0/0/5 de$
Feb  1 12:56:01 10.1.0.158      kmarquis        ttyp0   10.1.0.137      stop    task_id=5       service=shell   process*mgd[38808]      cmd=set interfaces ge-0/0/5 desc$
Feb  1 12:56:05 10.1.0.158      kmarquis        ttyp0   10.1.0.137      stop    task_id=6       service=shell   process*mgd[38808]      cmd=commit and-quit 
Feb  1 12:56:27 10.1.0.158      kmarquis        ttyp0   10.1.0.137      stop    task_id=7       service=shell   process*mgd[38808]      cmd=exit 
Feb  1 12:56:27 10.1.0.158      kmarquis        ttyp0   10.1.0.137      stop    task_id=1       service=shell   elapsed_time=49 process*mgd[38808]      cmd=logout
Feb  1 12:56:34 10.1.0.158      test    ttyp0   10.1.0.137      start   task_id=1       service=shell   process*mgd[38845]      cmd=login
Feb  1 12:56:44 10.1.0.158      test    ttyp0   10.1.0.137      stop    task_id=2       service=shell   process*mgd[38845]      cmd=show configuration 
Feb  1 12:56:53 10.1.0.158      test    ttyp0   10.1.0.137      stop    task_id=3       service=shell   process*mgd[38845]      cmd=show system uptime 
Feb  1 12:56:56 10.1.0.158      test    ttyp0   10.1.0.137      stop    task_id=4       service=shell   process*mgd[38845]      cmd=exit 
Feb  1 12:56:56 10.1.0.158      test    ttyp0   10.1.0.137      stop    task_id=1       service=shell   elapsed_time=22 process*mgd[38845]      cmd=logout

And with that all, we have a fully configured and working AAA TACACS+ server 🙂

Extra Treat 🙂
Extra Extra Treat 😀

Reference

Configure TACACS+ Ubuntu 14.04LTS
TACACS+ Accounting
TACACS+ Authenication
TACACS+ Advantages