MPLS – Re-Engineering a 1:1 Customer MPLS L3 VPN Tunnel to serve entire company, tons of ENARSI Topics included in lab, including gotchas for exam day! :)

No PE to CE Adjcancencies, we are getting this entire lab rocking, tonight! 🙂

I don’t usually blow away labs as I learn along the way, as I essentially turned all customer into 1:1 mappings across the MPLS, and its just not worth your time reading (you can refer to here for 1:1 Customer mappings).

I am doing a “we er” / “delete vlan.dat” on both PE Routers to completely start over, and demonstrate how to get Customer Domains propagating across the MPLS rather than just 1:1 mappings without a bunch of labbing mistakes between (though may be some).

I wanted to make this clear at the top, I did successfully re-engineer this 1:1 MPLS Cloud to 1:Many with no instruction or training materials, just looking at logic shown below and applyig it to my lab as shown step by step!

That said lets get right to into configurations and explanations of why the work!

As mentioned several times throughout my MPLS posts, the magic happens on PE Routers, and instead of turning to instructors or training material I worked my butt off for the last couple days to figure out how to make this properly – Enjoy!

To begin a couple of quirks I want to note up front I’ve run into along my labbing

• EIGRP, RIP, and BGP do not have VRF modes you can jump directly into from the “router (protocol) # vrf (VRF)” like OSPF, you must first enter router protocol configuration, then define “address-family ipv4 vrf (VRF)”
• You cannot use the same EIGRP AS # in your “Classic” configuration as in a VRF Address-Family instance, AF AS # takes precedence for EIGRP Routes it contains
• When configuring a “router ospf # vrf (VRF)” it will not work unless you have a) that VRF defined on the Router and b) a Physical Up/Up interface in “vrf forwarding” for that VRF Instance
• VRF’s will fail for ALL of this, unless you type “address-family ipv4” within the VRF configuration mode, as this tells it to work with IPv4 Protocols – OSPF Will kick out an error like “Unable to find Unique RID and process is not able to start” – No matter if you enter a RID manually nothing will work, your VRF needs to be configured onto a Physical Interface that is Up/Up and you will see no problems!

These are just some quirks, you will need to lab to see the full scopy of quirks!

Now back to the matter at hand – Getting our Customer talking across the MPLS!

It begins with the VRF settings on each router, however it begins with an even simpler initial configuration on the Router needed for mBGP to Route traffic across the L3 VPN:

R1-PE#sh run int lo0
Building configuration…

Current configuration : 81 bytes
!
interface Loopback0
ip address 1.1.1.1 255.255.255.255
ip ospf 1 area 0    <—– MPLS Network IP Connecitivty
end

R1-PE#

Consider this with mBGP and how it creates the L3 VPN to R5-PE:

router bgp 65536
bgp log-neighbor-changes
neighbor 5.5.5.5 remote-as 65536
neighbor 5.5.5.5 update-source Loopback0

Totally normal iBGP Neighbor statement, to reach it you might have a static route pointed directly at it, however the below configuration takes care of this:

address-family vpnv4 <—- This defines L3 VPN, Peer info configured below
neighbor 5.5.5.5 activate <—- Send traffic to this neighbor no matter what
neighbor 5.5.5.5 send-community extended <— Send all BGP Traffic to Neighbor
exit-address-family

Address-Family VPNv4 is specific only to BGP that I am aware of, this is the L3 VPN configuration, the remote peer loopedback “Active” tells BGP to attempt to send traffic to that neighbor regardless of connectivity.

“send-community” is used to send BGP Communities information, “extended” means transport all networks BGP learns about via Redistribution across the L3 VPN.

Back to VRFs with this information

This is for the Loopy / OSPF Network:

!
vrf definition 101:Looped <— VRF # : Description
rd 1.1.1.1:1 <— Unique IP sent over VPN to distinguish traffic
!
address-family ipv4  <— Tells VRF to work with IPv4 Protocols
route-target export 1.1.1.1:101 <— Exports VRF 101 Traffic for VPN Transport
route-target import 5.5.5.5:101 <— Imports VPN info to local VRFs for IP Routing
exit-address-family

This is for the Back / EIGRP Network:

vrf definition 102:Back
rd 1.1.1.1:2
!
address-family ipv4
route-target export 1.1.1.1:102
route-target import 5.5.5.5:102
exit-address-family
!

For two customers, no matter the amount of sites, this is the Import / Export configuration that is needed at a basic level, to get traffic from Point A to B.

Note that “Import” refers to the mBGP VPN Peers Address from its AF:

neighbor 5.5.5.5 send-community extended

That is because traffic from that Peer coming to this Peer has a destination off of the local PE, so it does an “Route-Target” Import on the traffic / Route Updates.

R5-PE has a Mirrored configuration, anything it receives from 1.1.1.1 is Imported to local VRF’s, for delivery to customers connected to its Provider Edge Devices.

With that lets get one more look of what this network was, and what its about to be!

What this network currently is, two customers that can reach only each other:

What this network is about to be, where all sites can reach each other via MPLS:

First things first, remove all old configs from the original lab, not shown here.

Actually, we can reconfigure the existing configs to make things easier:

---

!
router eigrp 1
!
address-family ipv4 vrf 102:Back autonomous-system 102
redistribute bgp 65536 metric 1 1 1 1 1
network 172.16.0.0
exit-address-family
!
router ospf 101 vrf 101:Looped
redistribute bgp 65536 subnets
network 172.16.0.0 0.0.15.255 area 101
!

---

 

Lets get dirty:

R1-PE(config-router)#address-family ipv4 vrf 102:Back autonomous-system 100
Changing from AS(102) to AS(100) is not allowed

I forgot IP Addressing can mess with this whole process a lot, so I’m going to jam through R1-PE IP Address / VRF Forwarding quick. all the while with errors scrolling:

Before so actually, I am going to remove BGP configs that may be contributing to the non-stop scrolling of console messages driving me insane:

R1-PE(config-if)#
*Dec 15 06:20:52.911: %OSPF-4-ERRRCV: Received invalid packet: mismatched area ID, from backbone area must be virtual-lin k but not found from 10.1.1.2, FastEthernet1/0
R1-PE(config-if)#
*Dec 15 06:21:01.539: %OSPF-4-ERRRCV: Received invalid packet: mismatched area ID, from backbone area must be virtual-lin k but not found from 10.1.1.2, FastEthernet1/0

Non-Stop.

router bgp 65536
no address-family ipv4 vrf 101:Looped
no address-family ipv4 vrf 102:Back

I found a command, but this gave me a good laugh as I forgot the that ONE command:

R1-PE(config)#no logging concole
Translating “concole”…domain server (255.255.255.255)

Not only does it not understand the command, but its looking up the IP Domain, FACEPALM.

OK AFTER ALL THESE HI-JINX, TIME TO START OVER WITH NEW NODES!

Enough of old configurations mucking up my new configuration, lets gets to work now!!!

I am actually glad I was able to re-cable the Topology to make more sense to me, however, both PE’s have the base config of VRF’s / MPLS / Some BGP – Lets do this.

Blue =  101:Looped – OSPF Routing Domain

Orange = 102:Back – EIGRP Routing Domain

R1-PE Interface Config (no console output)

R1-PE(config)#int fa1/0
R1-PE(config-if)#vrf forwarding 101:Looped
R1-PE(config-if)#ip add 10.1.100.1 255.255.255.0
R1-PE(config-if)#int fa2/0
R1-PE(config-if)#vrf forwarding 101:Looped
R1-PE(config-if)#ip add 10.1.2.1 255.255.255.0
R1-PE(config-if)#int fa3/0
R1-PE(config-if)#vrf forwarding 101:Looped
R1-PE(config-if)#ip add 10.1.3.1 255.255.255.0
R1-PE(config-if)#int fa4/0
R1-PE(config-if)#vrf forwarding 102:Back
R1-PE(config-if)#ip add 192.168.2.1 255.255.255.0

R5-PE Interface Config (no console output)

R5-PE(config)#int fa1/0
R5-PE(config-if)#vrf forwarding 101:Looped
R5-PE(config-if)#ip add 10.1.200.1 255.255.255.0
R5-PE(config-if)#int fa2/0
R5-PE(config-if)#vrf forwarding 101:Looped
R5-PE(config-if)#ip add 10.1.4.1 255.255.255.0
R5-PE(config-if)#int fa3/0
R5-PE(config-if)#vrf forwarding 101:Looped
R5-PE(config-if)#ip add 10.1.3.1 255.255.255.0
R5-PE(config-if)#int fa4/0
R5-PE(config-if)#vrf forwarding 102:Back
R5-PE(config-if)#ip add 192.168.1.1 255.255.255.0

Now that the Physical Interfaces are IP’d and Up, they should allow for specialized VRF Routing Instances to be created without issue, which we then go into BGP Address-Family configuration to Redistribute to the IGP’s / VRF’s over the Layer 3 VPN Tunnel!

So lets get IGP’s setup for Customer sites first (hopefully with no issues!)

R1-PE OSPF / EIGRP VRF’s

R1-PE(config)#router ospf 10 vrf 101:Looped
R1-PE(config-router)#redistribute bgp 65536 subnets
R1-PE(config-router)#network 10.1.100.0 0.0.0.255 area 0
R1-PE(config-router)#network 10.1.1.0 0.0.0.255 area 0
R1-PE(config-router)#network 10.1.2.0 0.0.0.255 area 0
R1-PE(config-router)#exit
R1-PE(config)#router eigrp 10
R1-PE(config-router)#address-family ipv4 vrf 102:Back autonomous-system 100
R1-PE(config-router-af)#redistribute bgp 65536 metric 1 1 1 1 1
R1-PE(config-router-af)#network 192.168.2.0
R1-PE(config-router-af)#^Z
R1-PE#
*Dec 15 07:43:49.059: %SYS-5-CONFIG_I: Configured from console by console
R1-PE#

Almost there – BGP Address-Family setup for OSPF and EIGRP for L3 VPN!

R1-PE(config-router)#router bgp 65536
R1-PE(config-router)#address-family ipv4 vrf 101:Looped
R1-PE(config-router-af)#redistribute ospf 10
R1-PE(config-router-af)#address-family ipv4 vrf 102:Back
R1-PE(config-router-af)#redistribute eigrp 100
R1-PE(config-router-af)#^Z
R1-PE#wr

Ok save that config on R1-PE, save it, repeat on R5-PE and watch it come to life!

OSPF / EIGRP VRFs on R5-PE

R5-PE(config)#router ospf 10 vrf 101:Looped
R5-PE(config-router)#redistribute bgp 65536 subnets
R5-PE(config-router)#network 10.1.200.0 0.0.0.255 area 0
R5-PE(config-router)#network 10.1.3.0 0.0.0.255 area 0
R5-PE(config-router)#network 10.1.4.0 0.0.0.255 area 0
R5-PE(config-router)#exit
R5-PE(config)#router eigrp 10
R5-PE(config-router)#address-family ipv4 vrf 102:Back autonomous-system 100
R5-PE(config-router-af)#redistribute bgp 65536
R5-PE(config-router-af)#network 192.168.1.0
R5-PE(config-router-af)#^Z
R5-PE#

Take cover – Powering on all Nodes in the Lab – Let this configuration work!!!

This is very weird, as usual, the configs half way didn’t save and there are some other random configs on the routers that I need to run through quick here (famous last words).

For panic configuring the following items before stuff started breaking on R5:

• All IP Address (including to MPLS)
• VRF IGP configs
• BGP Address Families

That went amazingly smooth… so I wonder what else broke 🙂

Can we finally see our “Looped” Company OSPF Subnets over the dang MPLS?

Oh boy… all non-core routers have lost all IP / Routing configuration…. I can press on just a bit longer to see if we have full connectivity here.

Looped-ABR1(config-if)#router ospf 1
Looped-ABR1(config-router)#network 10.1.100.0 0.0.0.255 area 0
Looped-ABR1(config-router)#network 10.1.101.0 0.0.0.255 area 11
Looped-ABR1(config-router)#
*Dec 15 08:25:55.795: %OSPF-5-ADJCHG: Process 1, Nbr 10.1.100.1 on FastEthernet1/0 from LOADING to FULL, Loading Done
Looped-ABR1(config-router)#

I’m going to configure Area 11 to see if I can see it from the other side of the MPLS at Looped-Site3 router, that will be the test for tonight before I call it as I am tired enough that I am just making stupid syntax errors.

Looped-Site1(config-router)#network 10.1.101.0 0.0.0.255 area 11
*Dec 15 08:29:54.783: %OSPF-5-ADJCHG: Process 1, Nbr 10.1.101.1 on FastEthernet5/0 from LOADING to FULL, Loading Done
Looped-Site1(config-router)#network 10.1.2.0 0.0.0.255 area 0
Looped-Site1(config-router)#
*Dec 15 08:33:27.327: %OSPF-5-ADJCHG: Process 1, Nbr 10.1.100.1 on FastEthernet2/0 from LOADING to FULL, Loading Done
Looped-Site1(config-router)#

BEFORE I CHECK LOOPED-SITE4 FOR THIS NEW SUBNET AMONG OTHERS

I at least got two OSPF Adjacencies Up between Looped-ABR1 and Looped-Site1 🙂

Honestly though, its been fun just configuring and troubleshooting based on logic and CCNP R/S skills, and even if this fails I will get up tomorrow and try it again and again, until it finally works as expected and I know not just the solution but the problem!

***************DRUM ROLL PLEASE***************

CRAP – I AM SO TIRED I FORGOT ROUTERS LOST ALL THEIR CONFIGS ALREADY!

I’ll throw a quick IP / OSPF Process on routers connected to R5-PE, and then I shall know, one sec here!

So the sites did not come up Fully Connected over the MPLS Network, and I am so tired I am barely able to easy commands like IP Addresses straight – HOWEVER.

However – I did get this little piece of gold of the R5-PE before ending for tonight:

R1-PE#ping 5.5.5.5 source 1.1.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 5.5.5.5, timeout is 2 seconds:
Packet sent with a source address of 1.1.1.1
…..
Success rate is 0 percent (0/5)
R1-PE#

SCRATCH LEAVING THAT BE FOR THE NIGHT, THAT IS SOMETHING SIMPLE I CAN RESOLVE RIGHT NOW, AND SLEEP GOOD TONIGHT!

After just 60 seconds of troubleshooting, there it is:

R1-PE#sh ip int brief
Interface IP-Address OK? Method Status Protocol
FastEthernet0/0 10.12.0.1 YES NVRAM administratively down down
FastEthernet1/0 10.1.100.1 YES NVRAM up up
FastEthernet2/0 10.1.2.1 YES NVRAM up up
FastEthernet3/0 10.1.3.1 YES NVRAM up up
FastEthernet4/0 192.168.2.1 YES NVRAM up up
FastEthernet5/0 unassigned YES NVRAM administratively down down
FastEthernet6/0 unassigned YES NVRAM administratively down down
Loopback0 1.1.1.1 YES NVRAM up up

WHY IS MY MPLS FACING FASTETHERNET INTERFACE ADMIN DOWN!?

R1-PE#conf t
Enter configuration commands, one per line. End with CNTL/Z.
R1-PE(config)#int fa0/0
R1-PE(config-if)#no shut
R1-PE(config-if)#
*Dec 15 09:04:58.767: %LINK-3-UPDOWN: Interface FastEthernet0/0, changed state to up
*Dec 15 09:04:59.767: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state to up
R1-PE(config-if)#
*Dec 15 09:05:08.695: %OSPF-5-ADJCHG: Process 1, Nbr 2.2.2.2 on FastEthernet0/0 from LOADING to FULL, Loading Done
R1-PE(config-if)#
*Dec 15 09:05:23.011: %LDP-5-INFO: FastEthernet0/0: LDP started
*Dec 15 09:05:23.703: %LDP-5-NBRCHG: LDP Neighbor 2.2.2.2:0 (1) is UP
R1-PE(config-if)#
*Dec 15 09:05:33.803: %BGP-5-ADJCHANGE: neighbor 5.5.5.5 Up
R1-PE(config-if)#

The Link State comes up, OSPF Adjacency, LDP / MPLS is UP an working now, BGP is Up and now running my Layer 3 VPN across the MPLS Network!!!!!!!!!

Guess what Loopy-Site4 says now:

Looped-Site4#sh ip route ospf

Gateway of last resort is not set

10.0.0.0/8 is variably subnetted, 8 subnets, 2 masks
O IA 10.1.2.0/24 [110/2] via 10.1.4.1, 00:10:34, FastEthernet2/0
O IA 10.1.100.0/24 [110/2] via 10.1.4.1, 00:10:34, FastEthernet2/0
O IA 10.1.101.0/24 [110/3] via 10.1.4.1, 00:10:34, FastEthernet2/0
O 10.1.200.0/24 [110/2] via 10.1.4.1, 00:23:01, FastEthernet2/0
Looped-Site4#

What a way to end that lab, an Administratively Down interface, thank you CCNP R/S TSHOOT exam skills, I am going to take a well deserved break and come back to hit some ENARSI 300-410 Topics specifically on the Blueprint next round 🙂

See you there fellow geeks!