I figured I would take it easy on a Friday night, and do a full blow lab on CEF 🙂
This Topology is a good example to review CEF FIB and Adjacency tables, as you don’t need a really large Topology to demonstrate, provides “Packet Switching” information meaning it just wants to read the packet / frame as rapidly as possible then forward it along out its destination port to the next device to perform the same CEF lookup.
So technically speaking,CEF is sort of like CDP in the way CEF stores details stores “Forwarding Information” about for directly connected neighbors, but still knows about remote networks and which interface to kick the packet out to get there.
CEF also does the same checksum lookup, decrement to TTL, and re-encapsulation of the ethernet frame as any of its predecessors as we will see in a demo below!
So first lets take a look at a switch newly write erased switch’s CEF table
As CEF is running by default, I wanted to show what it looks like on a fresh switch:
SW3
SW3#sh ip cef
Prefix Next Hop Interface
0.0.0.0/0 receive
0.0.0.0/8 drop
0.0.0.0/32 receive
127.0.0.0/8 drop
224.0.0.0/24 receive
240.0.0.0/4 drop
255.255.255.255/32 receive
SW3#
So we have three header fields, and two different “Next Hop” types by default, and NO interfaces are listed as this switch hasn’t learned where to send anything.
All it knows by default is to receive, and to drop.
- Receive = Locally known Network or Host Address
- Drop = Known bad packet to be dropped (explained in previous post here), this may also be due to a bad check, expired TTL in the receive Ethernet Frame, etc.
So it has a default route it will receive any traffic on, if it see’s traffic from a network starting with 0.x.x.x it will be dropped, along with the reserved spaces we all know and love such as the Loopback range 127.0.0.0/8 and Experimental Range 240.0.0.0/4.
However this shows by default it will receive any sensible traffic via 0.0.0.0/0 (any), MultiCast Traffic on 224.0.0.0/24, and Broadcasts on 255.255.255.255.
You can disable CEF, and it will look a lot like this:
SW3(config)#no ip cef distributed
%Cannot disable CEF on this platform <—- SAY WHAAAAAT?
SW3(config)#
Back in my day when we used to route packets uphill both ways we could disable CEF using that command, but apparently not anymore, so never mind that demo!
*Note this error came from a 3750X running IOS 15.x*
Now to review SW1’s CEF table to see how it has populated from the Topology
SW1
SW1#sh ip cef
Prefix Next Hop Interface
0.0.0.0/0 no route <—- Because SW1 has “ip routing” enabled
0.0.0.0/8 drop
0.0.0.0/32 receive
10.0.0.0/24 attached Port-channel1 <– L3 EC Network
10.0.0.0/32 receive Port-channel1 <– Network Number
10.0.0.10/32 receive Port-channel1 <– Host Addy for Po1 Int
10.0.0.255/32 receive Port-channel1 <– Po1 Broadcast Addy
10.1.1.0/24 attached Vlan10 <—- VLAN 10 network
10.1.1.0/32 receive Vlan10 <—- Network Number
10.1.1.1/32 receive Vlan10 <—- Host Addy for Host A
10.1.1.255/32 receive Vlan10 <—- Broadcasts for LAN
10.2.2.0/24 attached Port-channel1 <— Remote Net Prefix
127.0.0.0/8 drop
224.0.0.0/4 drop
224.0.0.0/24 receive
240.0.0.0/4 drop
255.255.255.255/32 receive
SW1#
Color coded it a bit this time, which I will discuss bullet point style:
- Colored in Blue is the default entries seen on SW3
- Colored in Red indicates there is no default route configured
- Colored in Orange is the L3 EtherChannel information
- Colored in Green is our the local LAN information
- Colored in Pink is the Static Route configured to the 10.2.2.0/24 network
As can be seen the FIB / CEF table is fairly easy to read, in order it shows:
- Network Prefix is the “Header” at the top with all its Host Addys below
- Network Number for Network Prefix
- Host Addresses of attached devices to the Network Segment
- Network Broadcast Address
So the table kind of parses itself by the Network Prefix first, then the information below it is the known Host Addresses and Broadcast Address, and if you see a single “attached” Network Prefix like the one in pink that means it is non-attached route to another network and the Interface it can be reached off of.
Everything else with the Network Prefix as a Header to host addresses is attached / local to the switch, anything kind of floating on its own is a statically configured or dynamically learned Route Prefix to a remote network.
Also we see a new Next Hop not seen on SW3 called “attached” which means that it may not directly connected to this Switch / Router, but it knows of that network!
The 10.1.1.0/32 Prefix I cannot explain, except maybe that it is shown that the Network Number (or starting number of the usable IP range).
The “No Route” business for the default route is because we have IP Routing enabled on SW1 where we didn’t on SW3, so until a route is defined it will show “No Route” rather than “receive” so lets change that:
SW1(config)#ip route 0.0.0.0 0.0.0.0 null0
SW1(config)#^Z
SW1#
SW1#sh ip cef
Prefix Next Hop Interface
0.0.0.0/0 attached Null0
Again this would generally point upstream towards that sweet freedom of the open internet, however in my lab it goes to the nowhere interface Null0, however now it does show “attached” and all is well.
So can’t hurt to know, if you see “No Route” instead of “receive” where your Default Route is at the top, your switch is has IP Routing enabled!
Time to review Packet Flow / MAC re-writes as the frame traverses the network
This is where things get clearer but also a little more murky, as when I think of an Adjacency Table I think of a nicely formatted table like FIB, but that couldn’t be less true!
The AT hides in the background of this front facing FIB table, however it will provide Layer 2 information for the Layer 3 Prefixes found in the FIB Table.
Here is the Topology used filled in with all MAC addy’s in use for clarity:
I simply looked at each interface that will be involved in the path to grab all these addy’s as shown below:
HostA#sh int fa0/1
FastEthernet0/1 is up, line protocol is up
Hardware is Gt96k FE, address is 001e.f797.f14b (bia 001e.f797.f14b)
SW1#sh int vlan 10
Vlan10 is up, line protocol is up
Hardware is EtherSVI, address is 1ce6.c7c1.c845 (bia 1ce6.c7c1.c845)
SW1#sh int po1
Port-channel1 is up, line protocol is up (connected)
Hardware is EtherChannel, address is 1ce6.c7c1.c841 (bia 1ce6.c7c1.c841)
And so on. Now before we go looking at the Adjacency table, remember that if you are on a home lab or maybe even in the exam room, it will NOT populate by itself, it will only know about traffic flows that it has inspected – So send those pings around!
After a ping from Host A to Host B, both the FIB and AT tables filled in a bit more:
SW1
SW1#sh ip cef
Prefix Next Hop Interface
0.0.0.0/0 attached Null0
0.0.0.0/8 drop
0.0.0.0/32 receive
10.0.0.0/24 attached Port-channel1
10.0.0.0/32 receive Port-channel1
10.0.0.10/32 receive Port-channel1
10.0.0.20/32 attached Port-channel1
10.0.0.255/32 receive Port-channel1
10.1.1.0/24 attached Vlan10
10.1.1.0/32 receive Vlan10
10.1.1.1/32 receive Vlan10
10.1.1.2/32 attached Vlan10
10.1.1.255/32 receive Vlan10
10.2.2.0/24 attached Port-channel1
10.2.2.2/32 attached Port-channel1
127.0.0.0/8 drop
224.0.0.0/4 drop
224.0.0.0/24 receive
240.0.0.0/4 drop
255.255.255.255/32 receive
SW1#
I have highlighted in red the entries that will show in the Adjacency table, which you will notice are HOST Address that are “Attached” (Not Local), as that is what the AT is for is gathering the forwarding information to the next hop router for remote networks!
We have 3 remote Host Addresses that are reachable, 10.0.0.20 / 10.2.2.1 / 10.2.2.2, so lets look at the details for these with more look at the Topology. First we will look at their individual output and I’ll re-post the Topology to compare MAC addresses to, but it is fairly obvious they are SRCDESTEthType for MAC addresses:
SW1
Entry for Remote L3 EtherChannel interface
SW1#sh adj detail
Protocol Interface Address
IP Port-channel1 10.0.0.20(8)
0 packets, 0 bytes
epoch 0
sourced in sev-epoch 0
Encap length 14
58971EABC8411CE6C7C1C8410800
L2 destination address byte offset 0
L2 destination address byte length 6
Link-type after encap: ip
ARP
Entry for Host B
IP Port-channel1 10.2.2.2(8)
0 packets, 0 bytes
epoch 0
sourced in sev-epoch 0
Encap length 14
58971EABC8411CE6C7C1C8410800
L2 destination address byte offset 0
L2 destination address byte length 6
Link-type after encap: ip
ARP
Entry for Host A
IP Vlan10 10.1.1.2(8)
0 packets, 0 bytes
Protocol Interface Address
epoch 0
sourced in sev-epoch 0
Encap length 14
001EF797F14B1CE6C7C1C8450800
L2 destination address byte offset 0
L2 destination address byte length 6
Link-type after encap: ip
ARP
SW1#
From the Adjacency Table you can see Host A’s SRC MAC, which will always be pointed at its Default Gateway for its Destination MAC, however IP Addresses in this packet flow WILL NEVER BE CHANGED LIKE MAC ADDRESSES WILL!!!
However when Host A send a ping to Host B, it looks at both the FIB entires for it and the Adjacency Table information for it:
SW1 Information to 10.2.2.2:
FIB:
10.2.2.2/32 attached Port-channel1
AT:
58971EABC8411CE6C7C1C8410800
In red it shows first the next-hop MAC Address (remote side of Po1) as its next hop device, so this will be re-written into the packet as it traverses the network
In blue it shows that the local exit interface MAC is that of the local end of Po1, and that will be the SRC MAC Address re-written onto the packet when it forwards it off
The Green 0800 is the Ethertype of the packet, not entirely relevant for CCNP studies.
So lets take a look at the beginning steps of a Ping from Host A to Host B visually:
This will arrive on Port 10.0.0.20 (receive) in the FIB table, and CEF will dissect it for details as to its destination, to make sure it continues on the correct path to its Dest.
So for this we would really want to focus on the 10.2.2.2 entry Adjacency details, as this is what is going to be used to re-write the Src and Dst MAC Addresses:
SW2:
IP Vlan20 10.2.2.2(8)
0 packets, 0 bytes
Protocol Interface Address
epoch 0
sourced in sev-epoch 0
Encap length 14
001B5336F2CD58971EABC8450800
L2 destination address byte offset 0
L2 destination address byte length 6
Link-type after encap: ip
ARP
By now its second nature, right? Red means Dest MAC to be re-written, Blue means the source to be re-written before forwarding the packet, and the green EtherType field really should have no bearing on your SWITCH exam at the CCNP level.
So the final stop illustrated here is that SW2 “receive” the frame off 10.0.0.20(receive), does an Adjacency lookup, and finds the information it needs to re-write the frame or packet before sending it to Host B:
I hope through these illustrations that you (and definitely I) have learnt that CEF is “PACKET SWITCHING!” and not routing, so really it just inspects a packet to make sure it is not malformed or otherwise needs to be dropped, then re-writes the Src and Dst MAC to get it to the next device to perform its own CEF lookup.
To wrap this up with a few troubleshooting tips for CEF and AT details:
If you are having trouble seeing your Destination entry in your CEF tables, look at “sh arp” and see if it is in there, as CEF only knows about routes that it has learned via forwarding traffic to those remote routes and receiving it back!
You will not see any details under (receive) entries in FIB, as they are local to the Router or Switch, you will only see Adjacency details for remote hosts / networks.
When looking for the re-write information, you’ll first want to look to FIB for the Destination information for your network, then look at its Next Hop in “… detail” in the Adjacency Table for exact information on what the new Src / Dst MAC will be.
Src and Dest IP Addresses never change, can’t say that enough.
And that is all I got for CEF Hardware Packet switching for you guys and gals!
Really it boils down to Adjacent devices passing having skeletal info to forward packets, but enough to do it efficiently, so that the next device can look at the IP that never changes and check its Layer 2 information on what Int to Fwd it out!
I am going to go try to do something other than study, much more to come! 🙂
*Apologies for typos or derps, I study after an 8 hour work day of network troubleshooting, and this week I am especially fried being on call so there may be some things in their that I re-worded and no longer make sense*
Please comment if you find any of these types of derps – It is appreciated!
The DEVNET GRIND! 