MultiLayer Switching / CEF – Packet processing explained in depth, and visually presented for clarity!

The above illustration is exactly how a MultiLayer Switch interconnects logically

The MultiLayer Switching segment confused of my studies confused me to no end, and when studying some material today it clicked what is meant by “Hardware Switching” vs Process or Fast Switching, and wanted to review it here as well in a more visual manner.

First lets discuss CEF and MultiLayer Switching and EXACTLY how it works!

When using CEF (on by default), you have the following components in use:

• Route Processor / CPU – This populates the IP Address Table and ARP Table with entries that are not yet known by CEF and its Hardware Switching components
• IP Address Table = “sh ip route”, this populates the FIB Table (Forwarding Information Base) with the skeletal info of routes it has learned, no Metric / AD info
• ARP Table = “sh arp”, the table that correlates known IP Addresses to their respective MAC Addresses
• FIB Table = Populated from IP Route Table
• Adjacency Table = The mapping of of CEF Table entries with ARP information
• “Feature Manager” is the software responsible for taking FIB / AT Tables info and populating them into TCAM!
• TCAM = Specialized Hardware Memory that stores L2/L3/L4/ACL/QOS information for rapid lookup by the Hardware ASIC that Frames come in on
• ASIC = The “Forwarding Engine” that searched each frame for its L2/L3/L4 info, to perform a lookup on TCAM memory to make forwarding decisions

So going down that list, essentially that is how things are populated, more visually:

• Route Processor -> IP Address + ARP Tables
• IP Address + ARP Tables -> FIB + AT
• FIB + AT -> TCAM (Performed by Feature Manager)
• TCAM is then used for RAPID lookup by ASICs

To better understand this, some more bullet points of what happens when a route enters an MLS running CEF (on by default):

• Frame enters port and ASIC does L3 lookup in TCAM
• If a valid entry is found in TCAM, Frame is forwarded without touching the CPU
• If valid entry is not found, the ASIC “punts” the Frame to the RP / CPU, which is responsible for creating an ARP request for the route
• If an ARP reply is received, the RP / CPU adds the IP Address / ARP table entries, which then trickle down via CEF to FIB and AT tables, and finally to TCAM

This is why it is called “Hardware Switching” with CEF, because the lookup is done on separate specialized hardware memory modules, the CPU only need to be involved for the following types of traffic:

•  Arp Queries (sent by CPU)
• CDP / NTP / Basically all Service Packets
• Routing Protocol Update traffic
• Packets that require encryption

Lets take a look at how a traffic flow is processed by CEF and Route Processor!

Any traffic that is just communication from one host to another will be performed by hardware lookup, by the ASIC looking up all the forwarding info it needs in the TCAM Memory modules (which there may be multiple ASICs and TCAMs), otherwise Protocol traffic generally will need to bother the Route Processor / CPU of the Switch or Router as it will need to update its tables which then trickle down to TCAM as pictured here:

So now that the CPU has updated it’s tables, CEF populates its own tables, which then populates the TCAM Memory Module so the next flow will be Hardware Switched.

Again remember “Feature Manager” is the software in the IOS responsible for populating TCAM from the FIB and AT tables!

Clear as mud? Great! I will finish this off for the night with some AT states for exam day.

For the Adjacency Table, there are 5 different states that you will need to know what they are, and what they mean to you in this whole process of Hardware Switching:

• Punt = CPU is required for certain traffic such as Routing Protocols, Service Protocols, so the packet is punted for the Route Processor to handle
• Glean = Waiting on ARP Response from CPU Punt diagram, will eventually age out, should only be in this state for a second or there is an issue!
• Null = A route is either configured or created by default to send traffic from this Destination to Null0 (Nowhere land)
• Drop = Packet is somehow malformed or encapsulated incorrectly and cannot be read, so the packet is dropped
• Discard = A Security Policy (ACL / VACL) is found in TCAM preventing this flow of traffic, so it is discarded

Speaking from experience, these states are critical to know, so be sure to re-read this and review the diagram as much as needed and ask any questions in the comments!

And that is it for tonight!

I was going to follow up with some CEF / AT labbing and demonstrations, but I’ll leave that for a separate post, I am glad to have put together such a good “at a glance” visual explanation of MLS / CEF operation.

Hope that helps all my fellow CCNP Grinders get that Pass on Exam day! 🙂