This will be sort of the beginning of the end of the this MPLS series I am going through, Layer 3 VPN is really the last piece of “Cisco Magic” happening on the Provider Edge routers at work, segregating customer traffic on the same device using VRF instances along with MBGP to forwarding them across the MPLS Network.
A quick review of our time honored predecessors to MPLS
And honestly, with SD-WAN building steam quickly, MPLS may make this old timers list in the not too distant future, however here are a few oldies but goodies:
- Leased Line (80’s) – I am not sure if encryption even existed this far back, if someone picked up the Telephone at your house the line would drop, this was the days of internet connectivity we would all like to forget!
- ATM / Frame-Relay / GRE / IPSec (90’s) – I’d argue ATM and Frame-Relay really had relevance up until the early 2000’s as this essentially was enterprise customers dedicated lines when I worked for a Service Provider myself, and into 2000-current I still see site-to-site IPSec VPN tunnels being setup between small business between sites, whereas DMVPN (GRE) is still widely in use between medium sized businesses but these things can be a real nightmare to troubleshoot (especially when you spend 10+ hours troubleshooting an IOS bug).
- SD-WAN is going to be the future in a big way – The tide has turned for SD-WAN in Cisco Lands when it acquired Viptela, which has basically everything from security to QoS to IP SLA to solving the worlds problems (in future releases) built into it, the one thing I cannot get over is Cisco “Don’t touch the CLI!” for Management and Control Plane functions, even though it IS possible they recommend using the GUI because it is just so easy to manage from there!
Back from Network Nostalgia, back to Layer 3 VPN, a look at the “Control Plane”
This means connectivity across the entire Topology from Customer Site A, to the Provider Edge / into the MPLS Network / Exiting the MPLS Network to the Customer site B.
The CE devices must “Peer” with the PE routers via an IGP (OSPF, EIGRP, BGP, RIP, etc) to share routes from that customer site to to the PE, at which the point the PE Router will divide up this traffic to send over the MPLS Network in what is referred to as a Layer 3 VPN or a VPNv4 Route as I have seen it named.
This is done using both separate VRF Instances and mBGP to help form these Layer 3 VPN’s between Provider Edge devices, which those remote PE’s should also have the Customer Sites networks via an IGP between the CE and PE to deliver the traffic.
One more important note on the Control Plane traffic for Layer 3 VPNs – Provider Edge networks will assign a label to traffic cross this L3 MPLS VPN with an MPLS Label that the remote PE is aware of, so when it receives that payload it can identify that this traffic is from Customer A’s site and needs to be routed to its final destination of Customer B.
Configuring mBGP with some necessary mentions of VRF
First off, the PE’s in my Topology (not sure if they are generally) iBGP Peers!
With mBGP we are taking a step further than going into the BGP Router Process, by actually going up a level into the “Address-Family” configuration, so that we can specify “Route-Targets” which actually are tied to VRF as seen shortly within the explanation of how this all works.
Its very confusing as to how this all works, so I will try my best to explain the process:
- VRF exports traffic to be routed to mBGP for transport with a “Route Target”
- A “Route Target” is what defines where the traffic should be delivered once mBGP unpacks the traffic it has transported at its end point PE
- Once the end point PE unpacks the Layer 3 VPN, it sees the route target #:#, and forwards it into the correct Customer Site B according to its own VRF configs.
This will become more clear once I review VRF below, so hang in there!
R1-PE and R5-PE configurations for BGP only:
R1-PE
router bgp 65536
bgp log-neighbor-changes
neighbor 5.5.5.5 remote-as 65536
neighbor 5.5.5.5 update-source Loopback0
!
address-family vpnv4
neighbor 5.5.5.5 activate
neighbor 5.5.5.5 send-community extended
exit-address-family
R5-PE
router bgp 65536
bgp log-neighbor-changes
neighbor 1.1.1.1 remote-as 65536
neighbor 1.1.1.1 update-source Loopback0
!
address-family vpnv4
neighbor 1.1.1.1 activate
neighbor 1.1.1.1 send-community extended
neighbor 1.1.1.1 next-hop-self
exit-address-family
This shows we are configuring our VPNv4 (or IPv4 VPN) via Address family, activating that sucker, and I am struggling to find an absolutely clear explanation of what a BGP send-community is outside of that it assist in the transit of specified traffic to remote BGP Peers in how to handle the traffic being sent (in this case sent via VPNv4).
To verify that this configuration alone (With no VRF’s at all yet as they are used to segregate customer traffic), is shown here:
I am really getting into screen snips instead of battling table formatting on this page 🙂
Note that without VRF at all, BGP neighbors are ready to send and receive VPNv4 traffic!
Keith Barger @ CBT Nuggets (highly recommend his courses) also showed a great new command for IOS devices that is similar to “netstat” on a windows PC, shown below:
R1-PE#sh control-plane host open-ports
^
% Invalid input detected at ‘^’ marker.
R1-PE#sh control?
controllers
R1-PE#sh control-
So that is one limitation I found on my current 7200 series image, which doesn’t have many, so I will actually steal a screen shot from Keith here:
(So please go buy his course so he doesn’t sue me for Copyright Infringement!)
However this really demonstrates the “Local Address” and “Foreign Address” for Layer 3 / Layer 4 Session information between BGP in this example, as it has a Server / Client mode that is very similar to LDP’s Active / Passive mode in the way only one side uses the well known TCP Port 179 to communicate with its BGP neighbor on!
VRF (Virutal Route Forwarding) and how it helps mBGP make L3 VPNs
First I’d like to throw up a flow chart of how this L3 VPN will work with VRF:
When Looped 1 sends traffic to R1-PE’s connected interface, it is met with a VRF instance that handles the traffic, rather than using the global IP routing table:
interface FastEthernet1/0
description connection to Looped site 1
mac-address 0000.1111.1111
vrf forwarding 101:Looped
ip address 172.16.101.1 255.255.255.252
duplex full
So when it enters the interface, it will see this configuration, and the router is immediately going to jump to this segment of information which was reviewed a couple posts back just as a high level contemplative review of the configs:
vrf definition 101:Looped
rd 1.1.1.1:1 <— Router Distinguisher for Looped1
!
address-family ipv4
route-target export 1.1.1.1:101 <— Local Route Target for Looped1
route-target import 5.5.5.5:101 <— Remote Route Target for Looped1
exit-address-family
What we have here is a “Route Distinguisher” of 1.1.1.1:1 which will help identify it within the mBGP traffic / IPv4 VPN within mBGP from other traffic, as this also has to transport Back 1 across the VPN as well:
interface FastEthernet2/0
description Connection to Back site 1
mac-address 0000.1111.1111
vrf forwarding 102:Back
ip address 172.16.101.1 255.255.255.252
duplex full
vrf definition 102:Back
rd 1.1.1.1:2 <— Route Distringuisher for Back1
!
address-family ipv4
route-target export 1.1.1.1:102 <— Local Route Target for Back1
route-target import 5.5.5.5:102 <— Remote Route Target for Back1
exit-address-family
I tried to match it up colorwise for both of those, however they are going to see that “vrf forwarding 102:Back” configuration on the interface traffic arrives at, and the router is going to refer to the “vrf definition 101:Looped” or “vrf definition 102:Back” when traffic comes in to figure out how to route this appropriately.
It depends on these unique RD (Route Distinguisher) IP Addresses to defined them, as can be seen above, both of these interfaces conveniently have the EXACT SAME IP Address on their ingress interface on R1-PE!
On top of all this, MPLS actually slaps a label on these routes as well, as just because it is doing some fancy mBGP / VRF powered Layer 3 VPN across the MPLS network it will still be Label Switched along the way!
VRF / mBGP Route Export vs Route Import
I’m finding this to be the brain twister of a subject with MPLS, however I will do my best to explain it as I understand, as the way that I understand this is that VRF both exports and imports routes to and from mBGP (and mBGP does the same) for transport and delivery of the specified routes.
To throw all this chaos into a graphic for explanation:
The addresses “Exported” from VRF into mBGP will include the “Original Network info, VPN Label #, RD #, and RT #, which is then “Imported” from mBGP into the VRF on the remote / receiving side, and if it has corresponding Route Target information it knows where the IP traffic destination and can then deliver it there!
A few things regarding these configurations:
- The VRF name is locally significant only, but is case sensitive, so if the router is referring back to that VRF Instance be sure to keep everything case sensitive
- Route Distinguishers AND Route Targets can either use an AS (Autonomous System) # or an IP Addresses, however the import / export values do need to match between local and remote routers
- Route Distinguishers can be the exact same on each side when referring to the same customer, however RD’s must be separate for different customers to ID their traffic as unique, however common customers RD’s can match on both sides of a L3 VPN.
There is a lot to unpack with all that explanation, which I attempted to do via wireshark, however Looped1 cannot ping across to Looped2, so we have an issue somewhere in the network, therefor I am calling an audible here to troubleshoot my topology to see where things are missing there connections!
Next post will be configuring and troubleshooting the CE to PE connections
As demonstrated multiple times our Provider network has no issues Transporting R1 to R5 traffic, so time to get some proverbial dirt under the finger nails and figure out why our customer sites are not pinging!
The DEVNET GRIND! 