After giving this some thought, I really can’t in good conscience cap off an MPLS Fundamentals sessions without dipping our toes into the intermediate waters of “Route Leaking” with MPLS, as a Network Engineer you will some day run into two companies under the same Umbrella with two different MPLS rings that need to leak routes between customers whether its to gather SNMP info, an Authentications server, etc.
There are a couple different ways of going about doing this or scenarios in which this may need to be done, so hang with me here through this, and we’ll nail this down quick!
Provider Network router needs to reach a Customer site IP Address
I am not sure what scenario would cause an actual Provider router to need to reach a specific IP on a site, but lets check it out using this topology:
In this scenario for some reason R2-P needs access specifically to the freshly created Lo20 IP Address of 20.20.20.20/24 which is NOT part of the IGP RIP being distributed and VPN’d and all that good stuff – Looped 1 doesn’t need to know about it, just our Provider Router R2-P for whatever reason.
This is actually the easy method, creating a static route, then doing a simple Redistribution into the Provider Network IGP of OSPF, which of course will advertise the route to all 5 Provider routers in the network so some Access Control may be necessary.
So to configure this one up:
Looped2(config)#int lo20
*Nov 29 14:22:13.078: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback20, changed state to up
Looped2(config-if)#ip add 20.20.20.20 255.255.255.0
Looped2(config-if)#
We have our destination IP up and ready to receive traffic!
R5-PE(config)#ip route 20.20.20.0 255.255.255.0 fa1/0 192.168.202.2
R5-PE(config)#
The Static Route must be made in the “Global Route Table” because we don’t want to add this route to our VRF and share it across the VPN, we only want to share it within our Provider Network Routers.
Now pointing a route out an interface is normally a big NO NO, because of the rapid fire ARP requests sent out of it due to not having a static IP entry in the ARP table, however because both sites on both sides have exactly the same IP Address we needed to define an interface or it wouldn’t know WHICH interface to route it out of.
Also adding the IP Address after the interface will allow the ARP table to grab that IP to map to that Interface / Layer 2 Address, thus eliminating the rapid fire ARP’s out of the route in an attempt to get its L3 to L2 mapping in the ARP table.
Finally the last trick in this type of Route Leaking – The Redistribution:
R5-PE(config)#router ospf 1
R5-PE(config-router)#redistribute static subnets
R5-PE(config-router)#
Now this “Leak” only allows for one way communication, so I’ll set a Conditional Debug to show the traffic makes it one way (to Looped2) but not back to R3-P router:
Looped2(config)#access-list 100 permit icmp any any
Looped2(config)#exit
Looped2#debug ip packet 100
IP packet debugging is on for access list 100
Looped2#
Ping from R3-P failing (as they are one way):
R3-P#ping 20.20.20.20
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 20.20.20.20, timeout is 2 seconds:
…..
Success rate is 0 percent (0/5)
R3-P#
Looped2#
*Nov 29 14:45:43.610: IP: s=10.34.0.3 (FastEthernet1/0), d=20.20.20.20, len 100, input feature, MCI Check(92), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
*Nov 29 14:45:43.614: IP: s=10.34.0.3 (FastEthernet1/0), d=20.20.20.20, len 100, rcvd 2
*Nov 29 14:45:43.618: IP: s=10.34.0.3 (FastEthernet1/0), d=20.20.20.20, len 100, stop process pak for forus packet
*Nov 29 14:45:43.622: IP: s=20.20.20.20 (local), d=10.34.0.3, len 100, unroutable
Looped2#
*Nov 29 14:45:45.554: IP: s=10.34.0.3 (FastEthernet1/0), d=20.20.20.20, len 100, input feature, MCI Check(92), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
*Nov 29 14:45:45.558: IP: s=10.34.0.3 (FastEthernet1/0), d=20.20.20.20, len 100, rcvd 2
*Nov 29 14:45:45.562: IP: s=10.34.0.3 (FastEthernet1/0), d=20.20.20.20, len 100, stop process pak for forus packet
*Nov 29 14:45:45.566: IP: s=20.20.20.20 (local), d=10.34.0.3, len 100, unroutable
Looped2#
That is output for 2 of the 5 pings it received, but cannot reply to, as it has no return path.
So, lets make a return path for it!
(The lab goes a bit off the rails here, however I received help from Cisco Learning Network forum to get this back on the rails, so bare with me!)
This is a bit more tricky, as with a Global IP Route into an IGP is not a difficult task, but now we must plant a route into this sites VRF instance, and then redistribute that static route back into its IGP (RIP) so that it can return ping responses to Provider routers!
First we create the static route in the 101:Looped VRF Instance:
R5-PE(config)#ip route vrf 101:Looped 10.34.0.0 255.255.255.0 fa1/0 10.45.0.4
This would be much easier to just make a route from the customer site, but I assume this meant to all be done from the Service Provider side away from customer eyes, so I believe this is the correct way to configure the static route (we’ll find out).
Next is Redistribution not only into RIP, but into the VRF instance inside RIP for 101:Looped on CE Router Looped2:
R5-PE(config)#do sh ip int brief
Interface IP-Address OK? Method Status Protocol
FastEthernet0/0 10.45.0.5 YES NVRAM up up
FastEthernet1/0 192.168.202.1 YES NVRAM up up
FastEthernet2/0 192.168.202.1 YES NVRAM up up
Loopback0 5.5.5.5 YES NVRAM up up
R5-PE(config)#ip route vrf 101:Looped 10.34.0.0 255.255.255.0 10.45.0.4
R5-PE(config)#router rip
R5-PE(config-router)#address-family ipv4 vrf 101:Looped
R5-PE(config-router-af)#redistribute static metric 2
R5-PE(config-router-af)#
I will save that output, but again the ping fails from R3-P, this is because Looped2 is not receiving the route given this method redistributing in the Address-Family VRF in RIP.
So now its time to freestyle a bit, as I will get this to work 🙂
I tried a different approach, redistributing the VRF route via RIP without Address-Family VRF mode and got the following:
R5-PE(config-router)#redistribute vrf 101:Looped static metric 2
% Inter-VRF redistribution is not supported by ‘rip’
R5-PE(config-router)#
However now I am thinking to myself, if it worked with a Global Static Route to Redistributed into an IGP (Provider Network), why not do the same thing going into RIP?
R5-PE(config)#no ip route vrf 101:Looped 10.34.0.0 255.255.255.0 10.45.0.4
R5-PE(config)#ip route 10.34.0.0 255.255.255.0 10.45.0.4
R5-PE(config)#router rip
R5-PE(config-router)#redistribute static metric 2
My thinking is that the 20.20.20.0/24 network with a static route on R5-PE is Directly Connected on Looped2 so the redistribution will have no effect on that, and it will only inject the new 10.34.0.0/24 address into RIP learned routes.
Looped2#sh ip route 10.34.0.0
% Network not in table
Looped2#
By creating a static route in the global routing table for a route already shared by the IGP OSPF Process 1 for the Provider network, I actually broke all routing within it:
R5-PE#sh ip route
(Codes Redacted)
Gateway of last resort is not set
1.0.0.0/32 is subnetted, 1 subnets
O 1.1.1.1 [110/5] via 10.45.0.4, 05:50:49, FastEthernet0/0
2.0.0.0/32 is subnetted, 1 subnets
O 2.2.2.2 [110/4] via 10.45.0.4, 05:50:49, FastEthernet0/0
3.0.0.0/32 is subnetted, 1 subnets
O 3.3.3.3 [110/3] via 10.45.0.4, 05:50:49, FastEthernet0/0
4.0.0.0/32 is subnetted, 1 subnets
O 4.4.4.4 [110/2] via 10.45.0.4, 05:50:49, FastEthernet0/0
5.0.0.0/32 is subnetted, 1 subnets
C 5.5.5.5 is directly connected, Loopback0
10.0.0.0/8 is variably subnetted, 5 subnets, 2 masks
O 10.12.0.0/24 [110/4] via 10.45.0.4, 05:50:49, FastEthernet0/0
O 10.23.0.0/24 [110/3] via 10.45.0.4, 05:50:49, FastEthernet0/0
S 10.34.0.0/24 [1/0] via 10.45.0.4
C 10.45.0.0/24 is directly connected, FastEthernet0/0
L 10.45.0.5/32 is directly connected, FastEthernet0/0
20.0.0.0/24 is subnetted, 1 subnets
S 20.20.20.0 [1/0] via 192.168.202.2, FastEthernet1/0
R5-PE#
So that isn’t going to work, and was removed. Its becoming clear that the static route back to R3-P must be associated with the VRF 101:Looped, it is just a matter of how!
So next I removed the straight up IP route, and added an interface, and can now at least see it in the VRF Routing Table for VRF 101:Looped :
R5-PE#conf t
Enter configuration commands, one per line. End with CNTL/Z.
R5-PE(config)#no ip route vrf 101:Looped 10.34.0.0 255.255.255.0 10.45.0.4
R5-PE(config)#ip route vrf 101:Looped 10.34.0.0 255.255.255.0 fa0/0 10.45.0.4
R5-PE(config)#do sh ip route vrf 101:Looped
Routing Table: 101:Looped
(Codes redacted)
Gateway of last resort is not set
10.0.0.0/24 is subnetted, 1 subnets
S 10.34.0.0 [1/0] via 10.45.0.4, FastEthernet0/0
172.16.0.0/30 is subnetted, 1 subnets
B 172.16.101.0 [200/0] via 1.1.1.1, 00:34:59
192.168.202.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.202.0/30 is directly connected, FastEthernet1/0
L 192.168.202.1/32 is directly connected, FastEthernet1/0
R5-PE(config)#
Finally we are getting somewhere I believe, now to get this VRF Static Route Redistributed into RIP up to Looped2 and we are in business!
So after consulting some friends on Cisco Learning Network forums (great place for expert help), someone pointed out that the RIP VRF 101:Looped was missing network 192.168.202.2 which is a total mystery to me as the previous post I created a telnet session between Looped1 and Looped2 to add Lo9 to show off how cool my network is!
Hats off the “jh” on CLN, great eye, and really appreciate the help catching that!
So of course, the following config was added:
R5-PE(config)#router rip
R5-PE(config-router)#address-family ipv4 vrf 101:Looped
R5-PE(config-router-af)#network 192.168.202.0
R5-PE(config-router-af)#do sh ip proto | b RIP
Interface Send Recv Triggered RIP Key-chain
FastEthernet0/0 2 2
Automatic network summarization is not in effect
Maximum path: 4
Routing for Networks:
10.0.0.0
Routing Information Sources:
Gateway Distance Last Update
Distance: (default is 120)
And of course:
R3-P#ping 20.20.20.20
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 20.20.20.20, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 44/52/72 ms
R3-P#
Not that it was a waste of time to try troubleshooting a config that just somehow went missing (probably from a VRF command that wiped the routing slate clean), however its a great lesson to stay humble and don’t be afraid or embarrassed to ask for a second set of eyes on an issue – We IT Professionals are our own best resources for information!
“Extranet” – Route Leaking between customer VPNs using Route Targets
What I will be going for here is to allow “Looped2” customer location 20.20.20.0/24 subnet to communicate with “Back1” customer site:
This works by first “Exporting” the Prefix to be leaked on one side, in this case that will be R5-PE as the Prefix will be flowing from Looped2 to Back1, so R5-PE would “Export” the Prefix then on the other side on R1-PE we will need to “Import” the route.
On the “Export” side where the route is coming from, we will add this Export-Map to the VRF that the leaked route resides in, in this case it will be 101:Looped. On the opposing side of R1-PE, we will need to “Import” the route to “102:Back” VRF instance.
Lets give this a shot and see if I can get this working 🙂
Prefix-List of Routes to be Leaked
R5-PE#conf t
Enter configuration commands, one per line. End with CNTL/Z.
R5-PE(config)#ip prefix-list EXPORT_ROUTES permit 20.20.20.0/24
R5-PE(config)#
Route-Map creation using Prefix-List
R5-PE(config)#route-map EXPORT_MAP permit 10
R5-PE(config-route-map)#match ip add prefix-list EXPORT_ROUTES
R5-PE(config-route-map)#set ?
as-path Prepend string for a BGP AS-path attribute
automatic-tag Automatically compute TAG value
clns OSI summary address
comm-list set BGP community list (for deletion)
community BGP community attribute
dampening Set BGP route flap dampening parameters
default Set default information
extcomm-list Set BGP/VPN extended community list (for deletion)
extcommunity BGP extended community attribute
global Set to global routing table
interface Output interface
ip IP specific information
ipv6 IPv6 specific information
level Where to import route
local-preference BGP local preference path attribute
metric Metric value for destination routing protocol
metric-type Type of metric for destination routing protocol
mpls-label Set MPLS label for prefix
origin BGP origin code
tag Tag value for destination routing protocol
traffic-index BGP traffic classification number for accounting
vrf Define VRF name
R5-PE(config-route-map)#set extcommunity ?
cost Cost extended community
rt Route Target extended community
soo Site-of-Origin extended community
R5-PE(config-route-map)#set extcommunity rt ?
ASN:nn or IP-address:nn VPN extended community
R5-PE(config-route-map)#set extcommunity rt 20.20.20.20:1 ?
ASN:nn or IP-address:nn VPN extended community
additive Add to the existing extcommunity
<cr>
R5-PE(config-route-map)#set extcommunity rt 20.20.20.20:1 additive ?
<cr>
R5-PE(config-route-map)#set extcommunity rt 20.20.20.20:1 additive
Adding the Route-Map to 101:Looped VRF Instance
R5-PE(config)#vrf definition 101:Looped
R5-PE(config-vrf)#address-family ipv4
R5-PE(config-vrf-af)#export ?
ipv4 Address family based VRF export
map Route-map based VRF export
R5-PE(config-vrf-af)#export map ?
WORD VRF export route-map name
R5-PE(config-vrf-af)#export map EXPORT_MAP ?
<cr>
R5-PE(config-vrf-af)#export map EXPORT_MAP
R5-PE(config-vrf-af)#^Z
R5-PE#
And finally a quick clear of BGP to reset the VPN on R5-PE
R5-PE#clear ip bgp *
R5-PE#
*Dec 2 18:22:16.883: %BGP-5-ADJCHANGE: neighbor 1.1.1.1 Down User reset
*Dec 2 18:22:16.883: %BGP_SESSION-5-ADJCHANGE: neighbor 1.1.1.1 VPNv4 Unicast topology base removed from session User reset
*Dec 2 18:22:16.887: %BGP_SESSION-5-ADJCHANGE: neighbor 1.1.1.1 IPv4 Unicast topology base removed from session User reset
*Dec 2 18:22:16.891: %BGP-5-ADJCHANGE: neighbor 192.168.202.2 vpn vrf 102:Back Down User reset
*Dec 2 18:22:16.895: %BGP_SESSION-5-ADJCHANGE: neighbor 192.168.202.2 IPv4 Unicast vpn vrf 102:Back topology base removed from session User reset
*Dec 2 18:22:17.711: %BGP-5-ADJCHANGE: neighbor 192.168.202.2 vpn vrf 102:Back Up
R5-PE#
*Dec 2 18:22:18.067: %BGP-5-ADJCHANGE: neighbor 1.1.1.1 Up
R5-PE#
That was by far the heavy lifting portion of the Route-Leak, as now on the R1-PE side we only have to import that Route-Target on the VRF 102:Back instance and reset BGP:
R1-PE#conf t
Enter configuration commands, one per line. End with CNTL/Z.
R1-PE(config)#vrf definition 102:Back
R1-PE(config-vrf)#address-family ipv4
R1-PE(config-vrf-af)#route-target import 20.20.20.20:1
R1-PE(config-vrf-af)#^Z
R1-PE#
*Dec 2 18:30:15.571: %SYS-5-CONFIG_I: Configured from console by console
R1-PE#clear ip bgp *
R1-PE#
I’ll spare the BGP reset output, but it does reset, and now to verify if Back1 has the route 20.20.20.0/24 in its Global IP Route table (drum roll please):
Back1#sh ip route
Codes: L – local, C – connected, S – static, R – RIP, M – mobile, B – BGP
D – EIGRP, EX – EIGRP external, O – OSPF, IA – OSPF inter area
N1 – OSPF NSSA external type 1, N2 – OSPF NSSA external type 2
E1 – OSPF external type 1, E2 – OSPF external type 2
i – IS-IS, su – IS-IS summary, L1 – IS-IS level-1, L2 – IS-IS level-2
ia – IS-IS inter area, * – candidate default, U – per-user static route
o – ODR, P – periodic downloaded static route, H – NHRP, l – LISP
+ – replicated route, % – next hop override
Gateway of last resort is not set
20.0.0.0/24 is subnetted, 1 subnets
D EX 20.20.20.0
[170/2560002816] via 172.16.101.1, 00:01:44, FastEthernet2/0
172.16.0.0/16 is variably subnetted, 3 subnets, 2 masks
C 172.16.101.0/30 is directly connected, FastEthernet2/0
L 172.16.101.2/32 is directly connected, FastEthernet2/0
C 172.16.222.2/32 is directly connected, Loopback0
192.168.202.0/30 is subnetted, 1 subnets
D EX 192.168.202.0
[170/2560002816] via 172.16.101.1, 00:01:44, FastEthernet2/0
192.168.222.0/32 is subnetted, 1 subnets
D EX 192.168.222.2
[170/2560002816] via 172.16.101.1, 00:01:44, FastEthernet2/0
Back1#
And to really push my luck with a ping test:
Back1#ping 20.20.20.20
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 20.20.20.20, timeout is 2 seconds:
…..
Success rate is 0 percent (0/5)
Back1#
EHHHHH, WRONG ANSWER!
We have only performed the Export / Import one way, so it will “Leak” the route, however it will not just start pinging willy nilly because of that!
For that we now need to go the OTHER WAY around, which we will do so now!
Creating Loopback30 on “Back1” and adding route to EIGRP
Back1#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Back1(config)#int lo30
*Dec 2 18:22:53.431: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback30, changed state to up
Back1(config-if)#ip add 30.30.30.30 255.255.255.0
Back1(config-if)#router eigrp 102
Back1(config-router)#network 30.30.30.0 0.0.0.255
Back1(config-router)#^Z
Back1#wr
Building configuration…
[OK]
Back1#
Prefix-List of Routes to be Leaked
R1-PE#conf t
Enter configuration commands, one per line. End with CNTL/Z.
R1-PE(config)#ip prefix-list EXPORT_ROUTES permit 30.30.30.0/24
Route-Map creation and adding it to VRF 102:Back
R1-PE(config)#route-map EXPORT_MAP permit 10
R1-PE(config-route-map)#match ip add prefix-list EXPORT_ROUTES
R1-PE(config-route-map)#set extcommunity rt 30.30.30.30:1 additive
R1-PE(config-route-map)#exit
R1-PE(config)#vrf definition 102:Back
R1-PE(config-vrf)#address-family ipv4
R1-PE(config-vrf-af)#export map EXPORT_MAP
R1-PE(config-vrf-af)#^Z
R1-PE#clear ip bgp *
R1-PE#
Now to Import on R5-PE
R5-PE#conf t
Enter configuration commands, one per line. End with CNTL/Z.
R5-PE(config)#vrf definition 101:Looped
R5-PE(config-vrf)#address-family ipv4
R5-PE(config-vrf-af)#route-target import 30.30.30.30:1
R5-PE(config-vrf-af)#^Z
*Dec 2 18:50:30.611: %SYS-5-CONFIG_I: Configured from console by console
R5-PE#clear ip bgp *
And now for the second drum roll pleeeeease:
Looped2#sh ip route
Codes: L – local, C – connected, S – static, R – RIP, M – mobile, B – BGP
D – EIGRP, EX – EIGRP external, O – OSPF, IA – OSPF inter area
N1 – OSPF NSSA external type 1, N2 – OSPF NSSA external type 2
E1 – OSPF external type 1, E2 – OSPF external type 2
i – IS-IS, su – IS-IS summary, L1 – IS-IS level-1, L2 – IS-IS level-2
ia – IS-IS inter area, * – candidate default, U – per-user static route
o – ODR, P – periodic downloaded static route, H – NHRP, l – LISP
+ – replicated route, % – next hop override
Gateway of last resort is not set
8.0.0.0/32 is subnetted, 1 subnets
C 8.8.8.8 is directly connected, Loopback8
9.0.0.0/32 is subnetted, 1 subnets
C 9.9.9.9 is directly connected, Loopback9
20.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C 20.20.20.0/24 is directly connected, Loopback20
L 20.20.20.20/32 is directly connected, Loopback20
30.0.0.0/24 is subnetted, 1 subnets
R 30.30.30.0 [120/5] via 192.168.202.1, 00:00:15, FastEthernet1/0
172.16.0.0/30 is subnetted, 1 subnets
R 172.16.101.0 [120/5] via 192.168.202.1, 00:00:15, FastEthernet1/0
192.168.111.0/32 is subnetted, 1 subnets
C 192.168.111.2 is directly connected, Loopback0
192.168.202.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.202.0/30 is directly connected, FastEthernet1/0
L 192.168.202.2/32 is directly connected, FastEthernet1/0
Looped2#
Success!!!!
But can we ping between these routers now?
Looped2#ping 30.30.30.30
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 30.30.30.30, timeout is 2 seconds:
…..
Success rate is 0 percent (0/5)
Looped2#
Still no! But kind of yes too!
We are not connecting the CE Routers Back1 and Looped2 entirely, but leaking those two ROUTES to talk to each other, so if there were devices behind those routers on those subnets they could communicate but any other subnet could not.
To visually demonstrate this:
Looped2#ping 30.30.30.30 source 20.20.20.20
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 30.30.30.30, timeout is 2 seconds:
Packet sent with a source address of 20.20.20.20
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 72/84/96 ms
Looped2#
Back1#ping 20.20.20.20 source 30.30.30.30
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 20.20.20.20, timeout is 2 seconds:
Packet sent with a source address of 30.30.30.30
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 68/81/92 ms
Back1#
That is the magic of Route Leaking with MPLS Route Targets!
And with that, I am actually done with MPLS now!
Time to move on to greener pastures, those pastures being DMVPN, see you there!
The DEVNET GRIND! 