First things first is getting DMVPN rocking on this Topology, and from in depth reading into different deployments of DMVPN, it seems the preferred method is to actually have two different DMVPNs running on BR1 and BR2, then configure spokes to be able to reach both (and each other).
Its been a lot of head scratching, so I’ve decided to just try labbing it to see if anything becomes any more clear while doing so, so lets get started.
This post contains excellent config setup I encourage you to read, however to keep forward movement in my studies, I will be demonstrating Front Door VRF on my original DMVPN Lab and Performance Routing will come at another time!
First the standard IPSec Profile is added to BR1 (Hub1) and BR2 (Hub2)
conf t
crypto isakmp policy 1
encr aes
auth pre-share
group 14
exit
crypto isakmp key LoopedBack!!! address 0.0.0.0
crypto ipsec transform-set CCIE esp-aes esp-sha-hmac
mode transport
exit
crypto ipsec profile ProtectTheTunnel
set transform-set CCIE
exit
Then Hub1 is configured with an extra command it kicks out an error for
MPLS-BR1(config)#int tu0
MPLS-BR1(config-if)#ip add 172.16.100.10 255.255.255.0
MPLS-BR1(config-if)#ip nhrp auth DmVpN1
MPLS-BR1(config-if)#ip nhrp map multicast dynamic
MPLS-BR1(config-if)#ip nhrp network-id 100
MPLS-BR1(config-if)#ip nhrp redirect
% NHRP-WARNING: ‘ip nhrp redirect’ failed to initialise
MPLS-BR1(config-if)#tunnel source fa1/0
MPLS-BR1(config-if)#tunnel mode gre multipoint
MPLS-BR1(config-if)#ip mtu 1400
MPLS-BR1(config-if)#tunnel protection ipsec profile ProtectTheTunnel
MPLS-BR1(config-if)#
*Dec 10 22:08:19.083: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up
MPLS-BR1(config-if)#
*Dec 10 22:08:20.655: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON
MPLS-BR1(config-if)#
The “ip nhrp redirect” command should allow the Hub to tell the Spoke routers they can communicate directly with each other, however from more looking into this, I am not sure if this is an IOS / EVE-NG bug or if I am just not fully configured yet – So I will keep on moving and hopefully once all said and done it will be working!
Hub2 configuration, same issue with “ip nhrp redirect” not initialising
MPLS-BR2(config)#int tu0
MPLS-BR2(config-if)#ip add 172.16.200.10 255.255.255.0
MPLS-BR2(config-if)#ip nhrp auth DmVpN1
MPLS-BR2(config-if)#ip nhrp map multicast dynamic
MPLS-BR2(config-if)#ip nhrp network-id 200
MPLS-BR2(config-if)#ip nhrp redirect
% NHRP-WARNING: ‘ip nhrp redirect’ failed to initialise
MPLS-BR2(config-if)#tunnel source fa2/0
MPLS-BR2(config-if)#tunnel mode gre multipoint
MPLS-BR2(config-if)#ip mtu 1400
MPLS-BR2(config-if)#tunnel protection ipsec profile ProtectTheTunnel
MPLS-BR2(config-if)#
*Dec 10 22:08:19.083: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up
MPLS-BR2(config-if)#
*Dec 10 22:08:20.655: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON
MPLS-BR2(config-if)#
Given that this looks like a very well known bug to the IOS / Router Series I am emulating in EVE-NG, I will continue on to see if I can simply get Phase 2 running for DMVPN, and see about adding in Phase 3 once Phase 2 is established.
Configuration of Houston BR1 and BR2 routers for DMVPN
Here we will configure both Spoke DMVPN Routers to point back to BOTH DMVPN Hub Routers at the Minneapolis location, I am not entirely sure the effect this will have on the spokes other than providing them with a secondary NHS for Spoke-to-Spoke communication if necessary (though I believe its also part of Phase 3 setup).
Houston BR1
HST-BR1(config)#int tu0
HST-BR1(config-if)#ip add 172.16.100.20 255.255.255.0
HST-BR1(config-if)#ip nhrp auth DmVpN1
HST-BR1(config-if)#ip nhrp network-id 100
HST-BR1(config-if)#ip nhrp map multicast dynamic
HST-BR1(config-if)#tunnel source fa1/0
HST-BR1(config-if)#ip mtu 1400
HST-BR1(config-if)#tunnel mode gre multipoint
HST-BR1(config-if)#ip nhrp nhs 172.16.100.10
HST-BR1(config-if)#ip nhrp nhs 172.16.200.10
HST-BR1(config-if)#ip nhrp map 172.16.100.10 101.101.101.10
HST-BR1(config-if)#ip nhrp map 172.16.200.10 201.201.201.10
HST-BR1(config-if)#ip nhrp map multicast 172.16.100.10
HST-BR1(config-if)#ip nhrp map multicast 172.16.200.10
HST-BR1(config-if)#tunnel protection ipsec profile ProtectTheTunnel shared
HST-BR1(config-if)#
*Dec 10 22:37:35.203: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up
HST-BR1(config-if)#
*Dec 10 22:37:36.431: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON
HST-BR1(config-if)#
Houston BR2
HST-BR2(config)#int tu0
HST-BR2(config-if)#ip add 172.16.200.20 255.255.255.0
HST-BR2(config-if)#ip nhrp auth DmVpN1
HST-BR2(config-if)#ip nhrp network-id 200
HST-BR2(config-if)#ip nhrp map multicast dynamic
HST-BR2(config-if)#tunnel source fa2/0
HST-BR2(config-if)#ip mtu 1400
HST-BR2(config-if)#tunnel mode gre multipoint
HST-BR2(config-if)#ip nhrp nhs 172.16.100.10
HST-BR2(config-if)#ip nhrp nhs 172.16.200.10
HST-BR2(config-if)#ip nhrp map 172.16.100.10 101.101.101.10
HST-BR2(config-if)#ip nhrp map 172.16.200.10 201.201.201.10
HST-BR2(config-if)#ip nhrp map multicast 172.16.100.10
HST-BR2(config-if)#ip nhrp map multicast 172.16.200.10
HST-BR2(config-if)#tunnel protection ipsec profile ProtectTheTunnel shared
*Dec 10 22:37:46.687: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up
HST-BR2(config-if)#
*Dec 10 22:37:48.763: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON
HST-BR2(config-if)#
Notice the key word “shared” at the end of the ipsec profile command, that kicked an error I didn’t really show until down in the post, it pops up for two different reasons.
Configuration of Chicago BR1 and BR2 for DMVPN
Chicago BR1
CHI-BR1(config)#int tu0
CHI-BR1(config-if)#ip add 172.16.100.30 255.255.255.0
CHI-BR1(config-if)#ip nhrp auth DmVpN1
CHI-BR1(config-if)#ip nhrp network-id 100
CHI-BR1(config-if)#ip nhrp map multicast dynamic
CHI-BR1(config-if)#tunnel source fa1/0
CHI-BR1(config-if)#ip mtu 1400
CHI-BR1(config-if)#tunnel mode gre multipoint
CHI-BR1(config-if)#ip nhrp nhs 172.16.100.10
CHI-BR1(config-if)#ip nhrp nhs 172.16.200.10
CHI-BR1(config-if)#ip nhrp map 172.16.100.10 101.101.101.10
CHI-BR1(config-if)#ip nhrp map 172.16.200.10 201.201.201.10
CHI-BR1(config-if)#ip nhrp map multicast 172.16.100.10
CHI-BR1(config-if)#ip nhrp map multicast 172.16.200.10
*Dec 10 22:50:26.903: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up
CHI-BR1(config-if)#tunnel protection ipsec profile ProtectTheTunnel shared
CHI-BR1(config-if)#
*Dec 10 22:50:28.707: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON
CHI-BR1(config-if)#
Chicago BR2
CHI-BR2(config)#int tu0
CHI-BR2(config-if)#ip add 172.16.200.30 255.255.255.0
CHI-BR2(config-if)#ip nhrp auth DmVpN1
CHI-BR2(config-if)#ip nhrp network-id 200
CHI-BR2(config-if)#ip nhrp map multicast dynamic
CHI-BR2(config-if)#tunnel source fa2/0
CHI-BR2(config-if)#ip mtu 1400
CHI-BR2(config-if)#tunnel mode gre multipoint
CHI-BR2(config-if)#ip nhrp nhs 172.16.100.10
CHI-BR2(config-if)#ip nhrp nhs 172.16.200.10
CHI-BR2(config-if)#ip nhrp map 172.16.100.10 101.101.101.10
CHI-BR2(config-if)#ip nhrp map 172.16.200.10 201.201.201.10
CHI-BR2(config-if)#ip nhrp map multicast 172.16.100.10
CHI-BR2(config-if)#ip nhrp map multicast 172.16.200.10
CHI-BR2(config-if)#tunnel protection ipsec profile ProtectTheTunnel shared
CHI-BR2(config-if)#
*Dec 10 22:43:08.603: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up
CHI-BR2(config-if)#
*Dec 10 22:43:10.219: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON
CHI-BR2(config-if)#
Great! So we now have all sites configured with IPSec protect DMVPN on both BR1 and BR2 routers at each Branch, so lets see if we can get it all working!
Now to take a look around the network to see what IS and IS NOT working!
MPLS – BR1 – DMVPN
MPLS-BR1(config-if)#do sh dmvpn
Legend: Attrb –> S – Static, D – Dynamic, I – Incomplete
N – NATed, L – Local, X – No Socket
# Ent –> Number of NHRP entries with same NBMA peer
NHS Status: E –> Expecting Replies, R –> Responding, W –> Waiting
UpDn Time –> Up or Down Time for a Tunnel
==========================================================================
Interface: Tunnel0, IPv4 NHRP Details
Type:Hub, NHRP Peers:1,
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
—– ————— ————— —– ——– —–
1 203.203.203.30 172.16.200.30 UP 00:28:04 D
MPLS-BR1(config-if)#
Not terrible, though I think I know what is needed to bring up both tunnels here.
MPLS – BR2 – DMVPN
MPLS-BR2#sh dmvpn
Legend: Attrb –> S – Static, D – Dynamic, I – Incomplete
N – NATed, L – Local, X – No Socket
# Ent –> Number of NHRP entries with same NBMA peer
NHS Status: E –> Expecting Replies, R –> Responding, W –> Waiting
UpDn Time –> Up or Down Time for a Tunnel
==========================================================================
Interface: Tunnel0, IPv4 NHRP Details
Type:Hub, NHRP Peers:2,
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
—– ————— ————— —– ——– —–
1 102.102.102.20 172.16.100.20 UP 00:15:29 D
1 203.203.203.30 172.16.200.30 UP 00:16:05 D
MPLS-BR2#
UP and UP!!! That is awesome to see, although we haven’t visited Spokes / Branch locations yet, so the celebration may be cut short going right on down to MPLS!
HST – BR1 – DMVPN
HST-BR1#sh dmvpn
Legend: Attrb –> S – Static, D – Dynamic, I – Incomplete
N – NATed, L – Local, X – No Socket
# Ent –> Number of NHRP entries with same NBMA peer
NHS Status: E –> Expecting Replies, R –> Responding, W –> Waiting
UpDn Time –> Up or Down Time for a Tunnel
==========================================================================
Interface: Tunnel0, IPv4 NHRP Details
Type:Spoke, NHRP Peers:2,
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
—– ————— ————— —– ——– —–
1 101.101.101.10 172.16.100.10 IKE 00:17:24 S
1 201.201.201.10 172.16.200.10 UP 00:15:17 S
HST-BR1#
So I am starting to see a pattern here, DMVPN Tunnel is Up and rocking to BR2 in MPLS location, however BR1 needs an extra kick in its configuration. How about BR2?
HST – BR2 – DMVPN
HST-BR2#sh dmvpn
Legend: Attrb –> S – Static, D – Dynamic, I – Incomplete
N – NATed, L – Local, X – No Socket
# Ent –> Number of NHRP entries with same NBMA peer
NHS Status: E –> Expecting Replies, R –> Responding, W –> Waiting
UpDn Time –> Up or Down Time for a Tunnel
==========================================================================
Interface: Tunnel0, IPv4 NHRP Details
Type:Spoke, NHRP Peers:2,
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
—– ————— ————— —– ——– —–
1 101.101.101.10 172.16.100.10 IKE 00:17:33 S
1 201.201.201.10 172.16.200.10 NHRP 00:17:33 S
HST-BR2#
So I think we can all agree that things went slightly uphill there, then kind of plunged into an oh no this is looking worse and worse scenario, and I am going to try to attempt to fix this situation using some Front Door VRF’s and a Secondary Tunnel between BR1 and BR2 at the Minneapolis location (as though it were a client).
We need a Tunnel Interface between our two NHS Servers!
I think.
From the hours of reading documentation and videos and forums (this takes a lot of input to “get” this concept without a live instructor face to face), I believe what is missing is a secondary tunnel mapping the two NHS routers BR1 and BR2 together.
MPLS – BR1 – Secondary DMVPN Tunnel config
MPLS-BR1(config)#int tu1
MPLS-BR1(config-if)#ip add 172.168.100.11 255.255.255.0
MPLS-BR1(config-if)#ip nhrp auth DmVpN
MPLS-BR1(config-if)#ip nhrp map 172.16.100.10 101.101.101.10
MPLS-BR1(config-if)#ip nhrp map 172.16.200.10 201.201.201.10
MPLS-BR1(config-if)#ip nhrp map multicast 101.101.101.1
MPLS-BR1(config-if)#ip nhrp map multicast 201.201.201.1
MPLS-BR1(config-if)#no ip redirects
MPLS-BR1(config-if)#ip mtu 1400
MPLS-BR1(config-if)#ip nhrp shortcut
MPLS-BR1(config-if)#ip nhrp nhs 172.16.100.10
MPLS-BR1(config-if)#ip nhrp nhs 172.16.200.10
MPLS-BR1(config-if)#tunnel source fa0/0
MPLS-BR1(config-if)#tunnel mode gre multipoint
MPLS-BR1(config-if)#
MPLS-BR1(config-if)#
MPLS-BR1(config-if)#
*Dec 11 07:38:35.875: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up
MPLS-BR1(config-if)#
I left the IPSec Profile off because it will throw errors if the Tunnel Source for a shared IPSec Profile does not match, so for the sake of testing I just didn’t use one.
MPLS – BR2 – Secondary DMVPN Tunnel config
MPLS-BR2(config)#int tu1
MPLS-BR2(config-if)#ip add 172.168.200.21 255.255.255.0
MPLS-BR2(config-if)#ip nhrp auth DmVpN
MPLS-BR2(config-if)#ip nhrp map 172.16.100.10 101.101.101.10
MPLS-BR2(config-if)#ip nhrp map 172.16.200.10 201.201.201.10
MPLS-BR2(config-if)#ip nhrp map multicast 101.101.101.1
MPLS-BR2(config-if)#ip nhrp map multicast 201.201.201.1
MPLS-BR2(config-if)#no ip redirects
MPLS-BR2(config-if)#ip mtu 1400
MPLS-BR2(config-if)#ip nhrp shortcut
MPLS-BR2(config-if)#ip nhrp nhs 172.16.100.10
MPLS-BR2(config-if)#ip nhrp nhs 172.16.200.10
MPLS-BR2(config-if)#tunnel source fa0/0
MPLS-BR2(config-if)#tunnel mode gre multipoint
MPLS-BR2(config-if)#
MPLS-BR2(config-if)#
MPLS-BR2(config-if)#
*Dec 11 07:40:53.815: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, c hanged state to down
*Dec 11 07:40:54.455: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, c hanged state to up
MPLS-BR2(config-if)#
MPLS-BR2(config-if)#
So we have this portion done, however all sites still appear exactly as pictured above, so we are missing a step somewhere in the network which I have this feeling it must be on the spokes as there are two concurrent DMVPN instances runing with only one tunnel interface per spoke – Which means the Tunnel IP Address will basically bond it to whichever VPN its common subnet is in.
Now I will actually create a secondary DMVPN Tunnel for each spoke site
I’ll spare the output as it was more or less mimicing Tunnel0 configs with tweaks to the IP Address, also I found this out along the way I hadn’t seen before:
HST-BR2(config-if)# tunnel protection ipsec profile ProtectTheTunnel
Error: All interfaces sharing this IPSec profile must be configured using the ‘shared’ keyword’.
Eg: tunnel protection ipsec profile foo shared
In Cisco’s example line, they pity the foo who doesn’t know you need “shared” at the end of your IPSec Profile statement on Tunnels that share common profiles 🙂
However again I ran into ANOTHER issue with IPSec Profiles on Tunnels:
HST-BR2(config-if)# tunnel protection ipsec profile ProtectTheTunnel shared
Error: There are other interfaces with the same IPSec profile but different tunnel sources.
Shared IPSec profiles must share the same tunnel source.
Although I may realistically not get to Performance Routing in this lab, I am learning every single way there is not to configure a DMVPN Tunnel 🙂
This was just a goof up that was easily corrected on the Tunnel1 interface.
So out of all this have I gotten myself absolutely anywhere closer to a DMVPN(s)?
Really my test case for this is HST – BR2 as that last showed one Tunnel stuck in IKE and the other stuck in NHRP so lets have another look:
HST-BR2#sh dmvpn
Legend: Attrb –> S – Static, D – Dynamic, I – Incomplete
N – NATed, L – Local, X – No Socket
# Ent –> Number of NHRP entries with same NBMA peer
NHS Status: E –> Expecting Replies, R –> Responding, W –> Waiting
UpDn Time –> Up or Down Time for a Tunnel
==========================================================================
Interface: Tunnel0, IPv4 NHRP Details
Type:Spoke, NHRP Peers:2,
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
—– ————— ————— —– ——– —–
1 101.101.101.10 172.16.100.10 IKE 01:15:45 S
1 201.201.201.10 172.16.200.10 NHRP 01:15:45 S
Interface: Tunnel1, IPv4 NHRP Details
Type:Spoke, NHRP Peers:2,
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
—– ————— ————— —– ——– —–
1 101.101.101.10 172.16.100.10 NHRP 00:12:38 S
1 201.201.201.10 172.16.200.10 NHRP 00:12:38 S
HST-BR2#
So that did absolutely nothing, I kind of feel as though I’m through stuff at the wall to see what sticks at this point, as Tunnel0 states haven’t changed, but now we have Tunnel1 that have both “NHRP” states on them.
The one thing I haven’t really tried or wanted to let loose to reek havoc on the lab is full Phase 3 by issue “ip nhrp shortcut” everywhere, but it might be that time – So I did add “ip nhrp shortcut” to each Tunnel interface on every router but still having issues.
I am hitting the breaks here and doing Front Door VRFs on my skeletal DMVPN Lab
This has been great to revisit a lot of CCNA / CCNP Topics, however I need to begin making forward progress again, and I believe after so much of that I will come back to this Topology with a renewed sense of how to get things working.
So it will be on freeze for now, but I shall be back!
The DEVNET GRIND! 