MLS Packet Switching – A brief history, CEF, Adjacency Table, Terminology, Hardware explanations and SDM Template considerations!

This is meant to be more geared “CEF” Packet Switching technology, but you can’t really appreciate the evolution of Packet Switching, without first knowing its history along with the Cisco hardware modules on their current devices that make CEF possible.

The being said, a quick review of Packet Switching methods leading up to CEF!

• Process-Switching = When a data flow enters an interface, the Route Processor (the CPU) must be involved in every packet forwarding decision
• Fast-Switching = When a data flow enters an interface, if the destination is not stored in the “Route Cache” for that interface, it is “Punted” to the “Route Processor” (CPU) to check the IP Route table for a destination
• Cisco Express Forwarding (CEF) – Uses dedicated hardware resources, that build a Packet Switching Database from L3 IP Route Table and L2 ARP Table info (AT)

Process-Switching is pretty irrelevant, just knowing that it exists, and that the CPU had to process entire Packets to route every single one of them is enough for me.

Fast-Switching was a recent predecessor that seemed like a pretty good deal, having to bug the CPU once per Data Flow, then have the route cached to forward all other packets.

CEF however is a whole different animal, deriving its information from the IP Route Table for Packet Switching (Forwarding), minus all the Routing Protocol stuff.

(CEF is enabled by default on Cisco devices!)

For example, CEF doesn’t need AD or Metric, it is only interested in forwarding packets as fast as possible without needing to disrupt the CPU unless absolutely necessary.

The “Route Processor” / CPU that builds the IP Route Table for CEF usage

With CEF we want to avoid using the “Route Processor” or CPU as little as possible!

When using CEF, the only type of traffic that requires CPU attention is the following:

• ARP Requests
• Routing Protocol Packets
• CDP
• Packets needing encryption
• Packets requiring a response from the CPU

While the Route Processor / CPU builds the IP Route Table and the ARP Table, CEF is taking stripped down version of this information, to already have Packet Decisions defined so it does not need to consult the CPU as to handling the Packet!

Talking about the Switch Fabric for a moment before proceeding

The “Switch Fabric” really refers to the way that the interconnects all these different Hardware Modules attached (CPU, RAM, DRAM, TCAM, ASIC, etc), both logically and physically, for example with a Shared BUS Topology within a PC.

It knows the some component needs a few milliseconds of its time, so that BUS is allotted to that physical component for that time, then another component uses the Shared BUS.

There are many different types of Fabric / Ways switches interoperate, however I just wanted to clarify the term “Switch Fabric” and also apply it here, as there really is software somewhere within this chaos to keep those Packets Switching at light speed!

Why CEF even exists, and an example of SW1 with a Default CEF (FIB) Table

CEF exists due to Cisco devices now having extra ASIC and TCAM modules available for newer devices, those can be utilized for storing these tables / performing these decision look ups rapidly, while the CPU takes care of keeping ALL Switch processes running!
For example on SW1 I did a “sh ip cef” BEFORE turning on IP Routing to review defaults:

SW1#sh ip cef
Prefix Next Hop Interface
0.0.0.0/0 receive = Receive any traffic
0.0.0.0/8 drop = Drop traffic with 0.x.x.x for a subnet
0.0.0.0/32 receive = Receive any Destined for this Host
127.0.0.0/8 drop = Drop Loopback traffic
224.0.0.0/24 receive = Receive Multicast traffic
240.0.0.0/4 drop = Drop traffic in the experimental IP range
255.255.255.255/32 receive = Etc.
SW1#

Note that any traffic not in the FIB will be dropped, as it derives its information from the IP Route Table, it will not do another lookup – Just drop it!

There was no IP Route Table or ARP Table to pull info from when this was created, so all it has is the default Receive and Drop lines, but to fill this in we need some Host Traffic!
Explaining that first lost ping when pinging a new Host – AT LONG LAST!!!

Remember that ping response starting with a single dot before !’s when pinging a new Host? Remember seeing that all those times and never thinking twice about it?

HostA#ping 10.0.10.102

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.10.102, timeout is 2 seconds:
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 1/1/4 ms
HostA#

We have always just accepted that first ! wouldn’t go through, business as usual.

WRONG!

That first dot is CEF not finding a matching Host IP Entry between its FIB / AT, and the AT “Punts” up an ARP request to the CPU (ARP’s must be performed by the CPU), until it gets a response that the Adjacency Table then adds an entry for… or it gets no ARP Response.
Which brings me to a quick, bullet point style of Terminology for CEF on exam day:

• Glean = The state an AT entry will remain in once it Punts an ARP to the CPU, and is awaiting a response, should only be there for less than a second, usually
• Punt = Packet that needs to go to CPU (as mentioned above) CDP, ARP, Dynamic Routing, Encryption, ETc.
• Null = I am not really sure what causes it, but pretty sure it leads to being dropped
• Drop = The inbound Packet does not match any Prefix in the FIB, it is dropped
• Discard = If a Security Feature prevents the traffic from forwarding

Not to break any Cisco NDA agreements, but I would know those for all 3 CCNP exams!

So currently on SW1 we have not enabled IP Routing as I was using it for Security DHCP labs, so I’ll turn on IP Routing and review some FIB / Adjaceny details!

Configuring CEF for Layer 3 on an MLS Switch!

CEF does run by default, but to get L3 info we must enable ip routing:

SW1#conf t
Enter configuration commands, one per line. End with CNTL/Z.
SW1(config)#ip routing
SW1(config)#ip cef
% Incomplete command.

SW1(config)#ip cef ?
distributed Distributed Cisco Express Forwarding
load-sharing Load sharing
optimize Optimizations
traffic-statistics Enable collection of traffic statistics

SW1(config)#ip cef distributed
SW1(config)#

This is easy to get confused because IP Routing is just “ip routing”, however CEF does require that extra “ip cef distributed” on the end, though it should be on by default.

A Topology created to review the FIB and Adjacency Table at Layer 3!

Everything has pinged each other device (minus that first ! we discussed), so lets take a look at the CEF (FIB) table now to see how thing are looking:

SW1#sh ip cef
Prefix Next Hop Interface
0.0.0.0/0 no route
0.0.0.0/8 drop
0.0.0.0/32 receive
10.0.0.0/24 attached Vlan1
10.0.0.0/32 receive Vlan1
10.0.0.1/32 receive Vlan1
10.0.0.2/32 attached Vlan1
10.0.0.255/32 receive Vlan1
10.0.10.0/24 attached Vlan10
10.0.10.0/32 receive Vlan10
10.0.10.1/32 receive Vlan10
10.0.10.250/32 attached Vlan10
10.0.10.255/32 receive Vlan10
127.0.0.0/8 drop
224.0.0.0/4 drop
224.0.0.0/24 receive
240.0.0.0/4 drop
255.255.255.255/32 receive
SW1#

I wanted to highlight in blue the Host Addresses as that is ultimately what is needed for CEF to perform Packet Switching to a Destination, however highlighted in orange are subnets that will cause CEF to bring in the Adjacency Table to get ARP info on the Host.

Again if the traffic does not match even a subnet prefix in the FIB, it is dropped immediately and will not do a route lookup in the IP Route Table!

We don’t see much of anything from Host A or Host B, because they are Layer 2 devices at this point, both on the same VLAN so I will use SVI’s to demonstrate L3 concepts next.

The CEF’s Layer 2 information database, then Adjacency Table (AT)!

So lets check it out on SW1 and see what info we can see:

SW1#sh adj
Protocol Interface Address
IP Vlan1 10.0.0.2(8)
IP Vlan10 10.0.10.250(8)
SW1#

So not a lot of great information here, however we now know the “Interface” we want to delve into further is not physical one, but an SVI (VLAN 1 and VLAN 10)!

SW1#sh adj vlan 10 det
Protocol Interface Address
IP Vlan10 10.0.10.250(8)
0 packets, 0 bytes
epoch 0
sourced in sev-epoch 0
Encap length 14
58971EABC8411CE6C7C1C8410800
L2 destination address byte offset 0
L2 destination address byte length 6
Link-type after encap: ip
ARP
SW1#

This is part is so important for exam success I’m going to make it it’s own little section!

Where to find the Frame MAC Over-write information using the Adjacency Table

Consider the above output, with that one line of HEX smack dab in the middle:
58971EABC8411CE6C7C1C8410800 – (Not sure what the 0800 is for but its there)

Lets see how this is derived, and especially, what it means for exam day!

SW1#sh int vlan 10
Vlan10 is up, line protocol is up
Hardware is EtherSVI, address is 1ce6.c7c1.c841 (bia 1ce6.c7c1.c841)

SW2#sh int vlan 10
Vlan10 is up, line protocol is up
Hardware is EtherSVI, address is 5897.1eab.c841 (bia 5897.1eab.c841)

The first Hex MAC Address (SW2 SVI 10) comes first, meaning if SW1 SVI 10 needs to send information to that neighbor, it will re-write the MAC Address with that neighbors MAC.
Its sooo not going to let me use Host A or Host B for an example, that I get this on SW1:

SW1#sh adj fa1/0/1 det
Protocol Interface Address
SW1#

I would think there would be some info in there, but as long as Packet Switching is happening faster than the speed of light, I am a happy man (so, so happy)!

***ONE MORE IMPORTANT NOTE – THE CEF PROCESS DOES NOT MAKE USE OF THE LOCAL MAC / CAM TABLES, THOSE ARE INDEPENDENT OF CEF, AND ONLY THE ARP TABLE WILL BE REFERENCED FOR LAYER 2 INFORMATION BY CEF!!!***

Everything about the Cisco Hardware Modules and how CEF utilizes them!

We’ll start at the bottom here with the switch Interfaces, constantly receiving and transmitting data, and the first Hardware in line for that duty is the ASIC.

The ASIC is a specialized Chip or Module piece of hardware on the MLS, that is referred to as the “Forwarding Engine” (not to be confused with Route Processor / CPU), it directly talks to the interfaces with instructions on what to do with certain traffic.
There can be one or more ASICs to perform certain functions like Routing, Security, etc.

The ASICs works directly with another specialized piece of hardware called “TCAM” or Ternary Content Address Memory, which CEF populates with both the FIB and AT information, specifically for the ASICs to reference when making their decisions!
There can also be more than one TCAM module installed on (big) Cisco switches!
Now for the red arrows in the diagram, that represents CEF pulling the stripped down Packet Switching data from the IP Route Table, and passing it down to the TCAM for access by ASICs that can perform incredibly fast Packet Switching decisions!

This process does run through the Switch Fabric, but so does everything else!

Also added was a box conjoining the ASICs and the TCAM, as together they must make decisions on how to forward traffic, ASICs do the leg work of re-writing MAC Addresses to destinations but they do not have that info without CEF / TCAM!

That seemed to be a very contentious point as I remember in my studies, the re-writing of the MAC, that is why I wanted to demonstrate beyond doubt how to verify AND THAT THE ASIC DOES DO THE RE-WRITING EVEN THOUGH IT GETS INFO FROM TCAM FOR IT!

Remember to factor in SDM Templates into your switches role in the network!

Here is a table of your resource allocation per template courtesy of apprize.info :

The only one I’d REALLY be concerned with is the VLAN option shown here:

SW1(config)#sdm prefer ?
access Access bias
default Default bias
dual-ipv4-and-ipv6 Support both IPv4 and IPv6
routing Unicast bias
vlan VLAN bias

That gives absolutely zero zip ZILCH resources to IPv4 routing, I highlighted the dual-ipv4-and-ipv6 as that is EASY POINTS on exam day if you are looking for an SDM Template for an IPv6 environment!

AND WITH THAT FINAL PIECE OF INFORMATION, THIS POST IS DONE FOR!

I have an older post on this subject I may just remove, as I feel this one is much more concise and easy to understand, as I have struggled putting this mental picture together of the Switch Chassis / Data Flow / Decision Making on an MLS!