Part 6: Troubleshooting of sub-optimal routing via route-maps / redist/ policy routing, and an old friend OSPF distance comes to save the day! (GREAT review!)

labbers_delight_rev3

Not /fin with this Topology of course, after this lab of fine tuning some sub-optimal routing I am taking copies of all “sh run” to be able to spin this lab up again if it ever gets the “wr er”, however it will be /fin for review and onto the subject of VPN’s.

So, Part 6, I am so ready to get this review over with – it’s almost taking as long as the initial learning!

As I recall our Local Policy Routing uncovered a case of sub-optimal routing, where OSPF paths are being preferred over much better link speeds, because it’s AD of 110 is lower than RIP’s 120. There are 2 different ways to address this:

  • Create a Policy Route on R2 setting R3 as the next hop for certain networks
  • Change the AD itself either via route-map or redistribution

So my initial thoughts is Policy Route on S0/0 directing traffic to a next-hop of 172.12.23.3 (Ethernet segment) would almost almost definitely introduce more sub-optimal routing to track down and fix, however I am not quite sure the best way to change that AD.

 

I haven’t seen it done before in a route map, so I’m going to try to tack it onto the Route-Map on R3 Redistributing those EIGRP routes into OSPF

 

So to get this configured, I need to check out the route-map for R3 to see where to insert my clause for changing the AD:

R3#show route
route-map EIGRP2RIP, deny, sequence 10
  Match clauses:
    tag 120
  Set clauses:
  Policy routing matches: 0 packets, 0 bytes
route-map EIGRP2RIP, permit, sequence 20
  Match clauses:
  Set clauses:
    tag 200
  Policy routing matches: 0 packets, 0 bytes
route-map RIP2EIGRP, deny, sequence 10
  Match clauses:
    tag 200
  Set clauses:
  Policy routing matches: 0 packets, 0 bytes
route-map RIP2EIGRP, permit, sequence 20
  Match clauses:
  Set clauses:
    tag 120
  Policy routing matches: 0 packets, 0 bytes
route-map EIGRP2OSPF, deny, sequence 5
  Match clauses:
    tag 110
  Set clauses:
  Policy routing matches: 0 packets, 0 bytes
route-map EIGRP2OSPF, permit, sequence 10
  Match clauses:
  Set clauses:
    tag 200
  Policy routing matches: 0 packets, 0 bytes
route-map OSPF2EIGRP, deny, sequence 10
  Match clauses:
    tag 200
  Set clauses:
  Policy routing matches: 0 packets, 0 bytes
route-map OSPF2EIGRP, permit, sequence 20
  Match clauses:
  Set clauses:
    tag 110
  Policy routing matches: 0 packets, 0 bytes
route-map OSPF2RIP, deny, sequence 5
  Match clauses:
    tag 120
  Set clauses:
  Policy routing matches: 0 packets, 0 bytes
route-map OSPF2RIP, permit, sequence 10
  Match clauses:
  Set clauses:
    tag 110
  Policy routing matches: 0 packets, 0 bytes
route-map RIP2OSPF, deny, sequence 10
  Match clauses:
    tag 110
  Set clauses:
  Policy routing matches: 0 packets, 0 bytes
route-map RIP2OSPF, permit, sequence 20
  Match clauses:
  Set clauses:
    tag 120
  Policy routing matches: 0 packets, 0 bytes
R3#

Oh yeah, it’s like that, once you get to route-mapping this output gets long and confusing fast! That is why show run is helpful as well, but probably not available come exam day. I located and highlighted in red our EIGRP2OSPF route-map, so I will put it smack dab in the middle, except I have no idea the output to look for but know that I am doing a “permit” on the sequence and “set”ing something:

RR3#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R3(config)#route-map EIGRP2OSPF permit 7
R3(config-route-map)#set ?
as-path           Prepend string for a BGP AS-path attribute
  automatic-tag     Automatically compute TAG value
  clns              OSI summary address
  comm-list         set BGP community list (for deletion)
  community         BGP community attribute
  dampening         Set BGP route flap dampening parameters
  default           Set default information
  extcommunity      BGP extended community attribute
  interface         Output interface
  ip                IP specific information
  ipv6              IPv6 specific information
  level             Where to import route
  local-preference  BGP local preference path attribute
  metric            Metric value for destination routing protocol
  metric-type       Type of metric for destination routing protocol
  mpls-label        Set MPLS label for prefix
  nlri              BGP NLRI type
  origin            BGP origin code
  tag               Tag value for destination routing protocol
  traffic-index     BGP traffic classification number for accounting
  vrf               Define VRF name
  weight            BGP weight for routing table

R3(config-route-map)#set ip ?
address     Specify IP address
  default     Set default information
  df          Set DF bit
  next-hop    Next hop address
  precedence  Set precedence field
  qos-group   Set QOS Group ID
  tos         Set type of service field

R3(config-route-map)#set metric ?
+/-<metric>     Add or subtract metric
  <0-4294967295>  Metric value or Bandwidth in Kbits per second
  <cr>

R3(config-route-map)#set metric

I color coded in red where my commands are on the CLI, and the output from the ? as there is so much output available for “set” options, however we do NOT have anything in there for Administrative Distance. I thought it might be under “set ip” or “set metric” however I was wrong, so very very wrong.

 

Trying using “distance …” command on R2 / Redistribution options

 

Looking back on my notes from 10 months ago (which is why it is good to make your own blog for studies), the administrative distance for OSPF routes can be changed locally right on the router, and the changes will only be locally significant which will be perfect for this scenario we are running into! First let us look at R2’s sub-optimal route table once more:

R2#sh ip route

Gateway of last resort is not set

     1.0.0.0/32 is subnetted, 1 subnets
O       1.1.1.1 [110/65] via 172.12.123.1, 00:42:05, Serial0/0
     2.0.0.0/32 is subnetted, 1 subnets
C       2.2.2.2 is directly connected, Loopback2
     100.0.0.0/13 is subnetted, 1 subnets
O E1    100.0.0.0 [110/84] via 172.12.123.1, 00:42:05, Serial0/0
     33.0.0.0/24 is subnetted, 1 subnets
O E2    33.33.33.0 [110/2] via 172.12.123.3, 00:42:05, Serial0/0
     3.0.0.0/32 is subnetted, 1 subnets
O       3.3.3.3 [110/65] via 172.12.123.3, 00:42:05, Serial0/0
     4.0.0.0/32 is subnetted, 1 subnets
O E2    4.4.4.4 [110/20] via 172.12.123.3, 00:04:54, Serial0/0
     172.12.0.0/24 is subnetted, 4 subnets
O E2    172.12.34.0 [110/20] via 172.12.123.3, 00:04:56, Serial0/0
O E1    172.12.15.0 [110/84] via 172.12.123.1, 00:42:08, Serial0/0
C       172.12.23.0 is directly connected, FastEthernet0/0
C       172.12.123.0 is directly connected, Serial0/0
     22.0.0.0/24 is subnetted, 1 subnets
C       22.22.22.0 is directly connected, Loopback22
     11.0.0.0/24 is subnetted, 1 subnets
O E1    11.11.11.0 [110/84] via 172.12.123.1, 00:42:08, Serial0/0
R2#

I just got even MORE EXCITED because I completely forgot, I left RIP and EIGRP AS 200 Redistribution as default E2 external routes, while EIGRP AS 100 is E1 – So if I can change it by External route type that would route traffic exactly right! Lets check it out:

R2(config-router)#distance ?
  <1-255>  Administrative distance
  ospf     OSPF distance

Ah yes, I remember this now, we will either have to make all external routes with an AD of 121, or make an access-list that allows certain routes to get an AD of 121, referenced here:

https://loopedback.com/2016/06/15/ospf-to-rid-4-ways-to-change-ad-sub-optimal-routing-route-loops/

Being that I am currently lazy and a bit fried from work / VPN theory, I’m going to try to just use the “distance ospf # …” command in OSPF configuration to change the local external AD, I will need to review and re-lab that mentioned page at some point but not as another part of this lab session:

R2(config-router)#distance ospf external 121
R2(config-router)#do sh ip route

Gateway of last resort is not set

     1.0.0.0/32 is subnetted, 1 subnets
O       1.1.1.1 [110/65] via 172.12.123.1, 00:00:26, Serial0/0
     2.0.0.0/32 is subnetted, 1 subnets
C       2.2.2.2 is directly connected, Loopback2
     100.0.0.0/13 is subnetted, 1 subnets
R       100.0.0.0 [120/2] via 172.12.23.3, 00:00:26, FastEthernet0/0
     33.0.0.0/24 is subnetted, 1 subnets
R       33.33.33.0 [120/1] via 172.12.23.3, 00:00:26, FastEthernet0/0
     3.0.0.0/32 is subnetted, 1 subnets
O       3.3.3.3 [110/65] via 172.12.123.3, 00:00:26, Serial0/0
     4.0.0.0/32 is subnetted, 1 subnets
R       4.4.4.4 [120/2] via 172.12.23.3, 00:00:00, FastEthernet0/0
     172.12.0.0/24 is subnetted, 4 subnets
R       172.12.34.0 [120/1] via 172.12.23.3, 00:00:08, FastEthernet0/0
R       172.12.15.0 [120/2] via 172.12.23.3, 00:00:08, FastEthernet0/0
C       172.12.23.0 is directly connected, FastEthernet0/0
C       172.12.123.0 is directly connected, Serial0/0
     22.0.0.0/24 is subnetted, 1 subnets
C       22.22.22.0 is directly connected, Loopback22
     11.0.0.0/24 is subnetted, 1 subnets
R       11.11.11.0 [120/2] via 172.12.23.3, 00:00:08, FastEthernet0/0
R2(config-router)#

Well, I guess I will be reviewing that old page sooner than I thought, eh? So I removed the distance command, and will read through the link posted above quick to see what needs to be done here.

So after a quick skim, we are going to need an access-list to reference in OSPF, this uses the “distance # (ip address) …” command in OSPF config, and I know we need the RID this route is learned off of but being the other spoke I don’t know if it needs to be the hub or R3 / other spoke’s RID, so my first though is to check our neighbor table to see if we even have the ASBR locked and loaded as a neighbor:

R2(config)#do sh ip ospf nei

Neighbor ID     Pri   State           Dead Time   Address         Interface
11.11.11.1        1   FULL/DR         00:01:58    172.12.123.1    Serial0/0
R2(config)#

Nope, on a Hub and Spoke OSPF network, your only ally (neighbor) is the Hub, so we will need to use it’s RID right there in the neighbor table to configure this as follows:

R2(config)#access-list 11 permit host 4.4.4.4
R2(config)#access-list 11 permit 172.12.34.0 0.0.0.255
R2(config)#router ospf 1
R2(config-router)#distance 121 ?
  A.B.C.D  IP Source address
  <cr>

R2(config-router)#distance 121 11.11.11.1 ?
  A.B.C.D  Wildcard bits

R2(config-router)#distance 121 11.11.11.1 0.0.0.255 ?
  <1-99>       IP Standard access list number
  <1300-1999>  IP Standard expanded access list number
  WORD         Standard access-list name
  <cr>

R2(config-router)#distance 121 11.11.11.1 0.0.0.255 11 ?
  <cr>

R2(config-router)#distance 121 11.11.11.1 0.0.0.255 11
R2(config-router)#

I have no idea if this is going to work, but excellent review I hadn’t even though of. DRUM ROLL PLEASE, as here we see the new and optimally routing table for R2:

(Failure, same routes). I won’t even bother with the output. It took doing a “clear ip ospf proc” and a “clear ip route *” to finally get these results:

R2#sh ip route

Gateway of last resort is not set

     1.0.0.0/32 is subnetted, 1 subnets
R       1.1.1.1 [120/2] via 172.12.23.3, 00:00:25, FastEthernet0/0
     2.0.0.0/32 is subnetted, 1 subnets
C       2.2.2.2 is directly connected, Loopback2
     100.0.0.0/13 is subnetted, 1 subnets
R       100.0.0.0 [120/2] via 172.12.23.3, 00:00:25, FastEthernet0/0
     33.0.0.0/24 is subnetted, 1 subnets
R       33.33.33.0 [120/1] via 172.12.23.3, 00:00:26, FastEthernet0/0
     3.0.0.0/32 is subnetted, 1 subnets
R       3.3.3.3 [120/2] via 172.12.23.3, 00:00:27, FastEthernet0/0
     4.0.0.0/32 is subnetted, 1 subnets
R       4.4.4.4 [120/2] via 172.12.23.3, 00:00:27, FastEthernet0/0
     172.12.0.0/24 is subnetted, 4 subnets
R       172.12.34.0 [110/20] via 172.12.123.3, 00:00:02, Serial0/0
O E1    172.12.15.0 [110/84] via 172.12.123.1, 00:00:02, Serial0/0
C       172.12.23.0 is directly connected, FastEthernet0/0
C       172.12.123.0 is directly connected, Serial0/0
     22.0.0.0/24 is subnetted, 1 subnets
C       22.22.22.0 is directly connected, Loopback22
     11.0.0.0/24 is subnetted, 1 subnets
O E1    11.11.11.0 [110/84] via 172.12.123.1, 00:00:02, Serial0/0
R2#

So I have managed to turn all OSPF into RIP routes again, and I am not sure how I did that with this command only specifying those 2 routes learned from 11.11.11.1 to have an AD of 121. Time to review exactly what I did here.

I can’t see any glaring mistakes, so I am wondering if maybe due to how the ACL is being called out, if that implicit deny is not kicking in quite right, so I put an explicit deny on there:

R2#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R2(config)#access-list 11 deny any
R2(config)#do show access-list 11
Standard IP access list 11
    10 permit 4.4.4.4
    20 permit 172.12.34.0, wildcard bits 0.0.0.255
    30 deny   any
R2(config)#

Now lets clear ip ospf proc again and see what we get:

R2#sh ip route

Gateway of last resort is not set

     1.0.0.0/32 is subnetted, 1 subnets
O       1.1.1.1 [110/65] via 172.12.123.1, 00:00:32, Serial0/0
     2.0.0.0/32 is subnetted, 1 subnets
C       2.2.2.2 is directly connected, Loopback2
     100.0.0.0/13 is subnetted, 1 subnets
O E1    100.0.0.0 [110/84] via 172.12.123.1, 00:00:32, Serial0/0
     33.0.0.0/24 is subnetted, 1 subnets
O E2    33.33.33.0 [110/2] via 172.12.123.3, 00:00:32, Serial0/0
     3.0.0.0/32 is subnetted, 1 subnets
O       3.3.3.3 [110/65] via 172.12.123.3, 00:00:32, Serial0/0
     4.0.0.0/32 is subnetted, 1 subnets
O E2    4.4.4.4 [110/20] via 172.12.123.3, 00:00:33, Serial0/0
     172.12.0.0/24 is subnetted, 4 subnets
O E2    172.12.34.0 [110/20] via 172.12.123.3, 00:00:36, Serial0/0
O E1    172.12.15.0 [110/84] via 172.12.123.1, 00:00:36, Serial0/0
C       172.12.23.0 is directly connected, FastEthernet0/0
C       172.12.123.0 is directly connected, Serial0/0
     22.0.0.0/24 is subnetted, 1 subnets
C       22.22.22.0 is directly connected, Loopback22
     11.0.0.0/24 is subnetted, 1 subnets
O E1    11.11.11.0 [110/84] via 172.12.123.1, 00:00:36, Serial0/0
R2#

I just cannot win with this method… WAIT A MINUTE! THAT WILDCARD MASK SHOULD BE 0.0.0.0 NOT THE NETWORK MASK OF 0.0.0.255! LETS TRY THIS AGAIN:

R2#sh ip route

Gateway of last resort is not set

     1.0.0.0/32 is subnetted, 1 subnets
R       1.1.1.1 [120/2] via 172.12.23.3, 00:00:09, FastEthernet0/0
     2.0.0.0/32 is subnetted, 1 subnets
C       2.2.2.2 is directly connected, Loopback2
     100.0.0.0/13 is subnetted, 1 subnets
R       100.0.0.0 [120/2] via 172.12.23.3, 00:00:09, FastEthernet0/0
     33.0.0.0/24 is subnetted, 1 subnets
R       33.33.33.0 [120/1] via 172.12.23.3, 00:00:09, FastEthernet0/0
     3.0.0.0/32 is subnetted, 1 subnets
R       3.3.3.3 [120/2] via 172.12.23.3, 00:00:10, FastEthernet0/0
     4.0.0.0/32 is subnetted, 1 subnets
R       4.4.4.4 [120/2] via 172.12.23.3, 00:00:10, FastEthernet0/0
     172.12.0.0/24 is subnetted, 4 subnets
R       172.12.34.0 [110/20] via 172.12.123.3, 00:00:00, Serial0/0
O E1    172.12.15.0 [110/84] via 172.12.123.1, 00:00:00, Serial0/0
C       172.12.23.0 is directly connected, FastEthernet0/0
C       172.12.123.0 is directly connected, Serial0/0
     22.0.0.0/24 is subnetted, 1 subnets
C       22.22.22.0 is directly connected, Loopback22
     11.0.0.0/24 is subnetted, 1 subnets
O E1    11.11.11.0 [110/84] via 172.12.123.1, 00:00:00, Serial0/0
R2#

The oddity is, only the E1 routes are remaining OSPF, there might be something to that but for now I am going to remove the distance command from R2 and see if there are any options in the redistribute command on R3.

So I wasn’t able to touch AD in Redistribution, but I was able to change the metric-type (as I’d had been able to in the route-map for EIGRP2OSPF as well, so lets see if applying that same command to R2 that allowed O E1 routes to stay holds steady.

Aaaaand, it did not. I have a feeling that not having a neighbor relationship to that ASBR is making things difficult, so I am resetting the works and putting a policy route on S0/0 as I said I would not be viable in the beginning of the lab as we are running out of options 🙂

 

Using Policy Routing to accomplish my task, and end this never ending lab

 

I’m going brain dead for the night, and while I could review past material for days on end, I need to wrap up the review (for now) and finish this lab tonight – So I will use Policy Routing on R2 to accomplish overcoming the sub-optimal routing we set out to destroy:

R2(config)#ip access-list extended GOTOYOURHOME
R2(config-ext-nacl)#10 permit ip host 11.11.11.1 host 4.4.4.4
R2(config-ext-nacl)#exit
R2(config)#route-map GOHOMEBALL permit 10
R2(config-route-map)#match ip add GOTOYOURHOME
R2(config-route-map)#set ip next-hop 172.12.23.4
R2(config-route-map)#exit
R2(config)#int s0/0
R2(config-if)#ip policy route GOHOMEBALL ?
  <cr>

R2(config-if)#ip policy route GOHOMEBALL

(I hope you enjoyed the Happy Gilmore references) Aaaaand:

R1#traceroute 4.4.4.4 source 11.11.11.1

Type escape sequence to abort.
Tracing the route to 4.4.4.4

  1 172.12.123.2 36 msec 32 msec 33 msec
  2  *  *  *
  3  *  *  *
  4  *  *  *
  5  *  *  *
  6  *  *  *
  7  *
ASR#2
[Resuming connection 2 to r2 … ]

R2(config-route-map)#^Z
R2#de
*Mar  1 17:24:20.014: %SYS-5-CONFIG_I: Configured from console by console
R2#debug ip pack
IP packet debugging is on
R2#
*Mar  1 17:24:25.624: IP: tableid=0, s=11.11.11.1 (Serial0/0), d=4.4.4.4 (Serial0/0), routed via FIB
*Mar  1 17:24:25.624: IP: s=11.11.11.1 (Serial0/0), d=4.4.4.4 (FastEthernet0/0), g=172.12.23.4, len 28, forward
*Mar  1 17:24:25.624: IP: s=11.11.11.1 (Serial0/0), d=4.4.4.4 (FastEthernet0/0), len 28, encapsulation failed
R2#

SO THIS HAS ONCE AGAIN FAILED, BUT I FINALLY GOT IT, USING THE DISTANCE COMMAND ON R2, AND THIS WRAPS THIS LAB ON UP!

After looking at the extended ip route command for the network, I noticed in its configuration for the route it was learned via 33.33.33.3, not via our only neighbor 11.11.11.1, so I repeated the same syntax only with 33.33.33.3 as the remote RID:

R2#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R2(config)#access-list 11 permit host 4.4.4.4
R2(config)#access-list 11 permit 172.12.34.0 0.0.0.255
R2(config)#router ospf 1
R2(config-router)#distance 121 33.33.33.3 0.0.0.0 ?
  <1-99>       IP Standard access list number
  <1300-1999>  IP Standard expanded access list number
  WORD         Standard access-list name
  <cr>

R2(config-router)#distance 121 33.33.33.3 0.0.0.0 11
R2#clear ip ospf proc
Reset ALL OSPF processes? [no]: yes
R2#
*Mar  1 17:40:54.304: %OSPF-5-ADJCHG: Process 1, Nbr 11.11.11.1 on Serial0/0 from FULL to DOWN, Neighbor Down: Interface down or detached
R2#
*Mar  1 17:41:10.094: %OSPF-5-ADJCHG: Process 1, Nbr 11.11.11.1 on Serial0/0 from LOADING to FULL, Loading Done

AAAAAAAAAND:

R2(config)#do sh ip route

Gateway of last resort is not set

     1.0.0.0/32 is subnetted, 1 subnets
O       1.1.1.1 [110/65] via 172.12.123.1, 00:02:16, Serial0/0
     2.0.0.0/32 is subnetted, 1 subnets
C       2.2.2.2 is directly connected, Loopback2
     100.0.0.0/13 is subnetted, 1 subnets
O E1    100.0.0.0 [110/84] via 172.12.123.1, 00:02:16, Serial0/0
     33.0.0.0/24 is subnetted, 1 subnets
O E2    33.33.33.0 [110/2] via 172.12.123.3, 00:02:16, Serial0/0
     3.0.0.0/32 is subnetted, 1 subnets
O       3.3.3.3 [110/65] via 172.12.123.3, 00:02:16, Serial0/0
     4.0.0.0/32 is subnetted, 1 subnets
R       4.4.4.4 [120/2] via 172.12.23.3, 00:00:08, FastEthernet0/0
     172.12.0.0/24 is subnetted, 4 subnets
R       172.12.34.0 [120/1] via 172.12.23.3, 00:00:10, FastEthernet0/0
O E1    172.12.15.0 [110/84] via 172.12.123.1, 00:02:19, Serial0/0
C       172.12.23.0 is directly connected, FastEthernet0/0
C       172.12.123.0 is directly connected, Serial0/0
     22.0.0.0/24 is subnetted, 1 subnets
C       22.22.22.0 is directly connected, Loopback22
     11.0.0.0/24 is subnetted, 1 subnets
O E1    11.11.11.0 [110/84] via 172.12.123.1, 00:02:19, Serial0/0
R2(config)#

I honestly I did not think I would be able to get this, but there it is, made possible by the distance command in OSPF config in R2. I am saving all routers and running for the door before I find another issue with the config – See you next time for some VPN configuration and theory!

EDIT:

To note for future reference, this is what led me to my answer:

R2#show ip route 4.4.4.4 255.255.255.255
Routing entry for 4.4.4.4/32
  Known via “ospf 1”, distance 110, metric 20
  Tag 200, type extern 2, forward metric 64
  Last update from 172.12.123.3 on Serial0/0, 00:00:22 ago
  Routing Descriptor Blocks:
  * 172.12.123.3, from 33.33.33.3, 00:00:22 ago, via Serial0/0
      Route metric is 20, traffic share count is 1
      Route tag 200

That was a great save, just goes to show, any problem can be worked through if you work at it hard enough. I will also note it tickles me that a configuration I didn’t even think of or mention about as a solution was what ended up saving the day 🙂 Pretty awesome!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s