by Richard Hijinx
This article is intended for those new to CEF and its impact on the way Multi Layer Switching is done in Cisco hardware. This article can also serve as a review for those familiar with the concepts but are looking for a refresher. In this first article we are going to go over the components that make up this switching architecture followed by some fundamental examples to illustrate these components and concepts at work. Before we get started be sure to download the topology we are going to be using in the lab examples for clarity.
Modern Catalyst Multilayer switches utilize CEF-based MLS. The terminology and architecture of this switching model can be tough to understand at first, but trust me, you are going to love it once you start working with it.
There are two distinguished functions provided by a Cisco Express Forwarding-based Multilayer Switching. The first function is building routing information. This routing information is built by the Layer 3 engine within the control plane. The second function provided is hardware switching of packets. Hardware switching of packets is done by the Layer 3 Forwarding Engine within the data plane. The data plane is where Cisco Express Forwarding works its magic. The control plane is where layer 3 decisions are made, when those layer 3 packets can NOT be switched in hardware.
Since Cisco Explress Forwarding’s provides it’s magic in the data plane, we will start with it. It is the most fun anyway. The Layer 3 Forwarding Engine within the data plane has two distinct components of its own.
The first component is the Cisco Express Forwarding FIB, and the second is the Cisco Express Forwarding Adjacency table. The Cisco Express Forwarding FIB is just a reformatted routing table put in order with the most specific routes first. The FIB contains next hop information for each prefix. The routing and next-hop information is built in software within the control plane, and passed to the Layer 3 forwarding engine and placed in the Forwarding Information Base. I can’t stress enough how important it is to understand that this is basically a reordered routing table with some additional entries in it. When a packet enters the switch, the switch consults the FIB and finds the longest match prefix and obtains the next hop address. I know this doesn’t sound like magic yet, but stay with me, there is more and this stuff is pretty cool.
The second component, the CEF adjacency table, contains and maintains layer 2 addresses for every entry in the FIB. This table is built the same way the Forwarding Information Base is built. It is built from information contained within the Layer 3 engine, in this case, the ARP table, which is then passed to the Layer 3 Forwarding Engine and placed in the Cisco Express Forwarding Adjacency table. If you know how packets are encapsulated and rewritten as they make their way across a layer 3 network, you are probably beginning to develop an idea of what is going to happen with the CEF adjacency table.
Since the FIB and Adjacency tables are both handled in hardware, we’re starting to see how Cisco Express Forwarding can improve the performance of layer 3 forwarding operations. It receives a copy of the work the Layer 3 Engine does in software, and the Layer 3 Forwarding Engine uses it to make multilayer switching decisions in hardware. Between the Forwarding Information Base having next hop layer 3 information, and the adjacency table having both the layer 3 and layer 2 information, Cisco Express Forwarding has at its disposal everything it needs to forward packets without consulting a routing table running in software. It is all in hardware and it all happens at line speed. I sure love it when tidbits of information come together.
Now, let’s take a look at two scenarios to see the paths packets take through a CEF-enabled MLS. In scenario 1, we have a valid FIB entry and associated adjacency table entry. A packet comes in the ingress interface, the FIB is consulted and an entry is found. The FIB is matched on the longest prefix. The layer 2 information is retrieved from the adjacency table and the packet is then forwarded through the packet rewrite engine, which rewrites the appropriate packet and frame header information at line speed and sends the packet out the egress interface. Notice that no ARP requests are made, no software based processing is performed, and frame information is written in hardware.
In scenario 2, as a packet comes ingress on an interface, the FIB is consulted and is unable to be CEF switched because of one of several different reasons. At this point the packet is sent to the Layer 3 engine for further processing, which is known as a CEF punt. We aren’t going to cover all the scenarios in which a CEF Punt occurs here. We’ll save those more in depth scenarios for Part 2.
It should be obvious, but it is worth mentioning here for clarity. As changes happen in the routing and ARP tables that are maintained by the Layer 3 Engine, those changes are automatically sent to the Layer 3 Forwarding Engine. This updates the Cisco Express Forwarding FIB and the Adjacency tables instantaneously.
Now that we have all of the basics out of the way, it is time to start looking at the relationship between the routing table, ARP table, the Cisco Express Forwarding FIB table, and the Cisco Express Forwarding adjacency table. Let’s start by looking at the IP addresses of the connected interfaces of the two devices used in these demonstrations.
MPLS1#show ip interface brief
Interface IP-Address OK? Method Status Protocol
FastEthernet0/0 unassigned YES NVRAM administratively down down
FastEthernet1/0 172.16.13.1 YES NVRAM up up
FastEthernet1/1 172.16.12.1 YES NVRAM up up
FastEthernet2/0 172.16.15.1 YES NVRAM up up
FastEthernet2/1 unassigned YES NVRAM administratively down down
FastEthernet3/0 unassigned YES NVRAM administratively down down
FastEthernet3/1 unassigned YES NVRAM administratively down down
Loopback010.0.0.1YESNVRAMupup
Tunnel7 10.0.0.1 YES TFTP up down
Tunnel70210.0.0.1YESTFTPupdown
Tunnel703 10.0.0.1 YES TFTP up down
MPLS2#showipinterfacebrief
InterfaceIP-AddressOK?MethodStatusProtocol
FastEthernet0/0unassignedYESNVRAMadministrativelydowndown
FastEthernet1/0172.16.12.2YESNVRAMupup
FastEthernet1/1 172.16.23.2 YES NVRAM up up
FastEthernet2/0 172.16.24.2 YES NVRAM up up
FastEthernet2/1 172.16.25.2 YES NVRAM up up
FastEthernet3/0unassignedYESNVRAMadministrativelydowndown
FastEthernet3/1 unassigned YES NVRAM administratively down down
Loopback010.0.0.2YESNVRAMupup
Now we are going to look at the routing table on MPLS1:
MPLS1#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1-OSPFexternaltype1,E2-OSPFexternaltype2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia-IS-ISinterarea,*-candidatedefault,U-per-userstaticroute
o - ODR, P - periodic downloaded static route
Gateway of last resort is not set
172.16.0.0/28 is subnetted, 6 subnets
O 172.16.24.0 [110/2] via 172.16.12.2, 01:12:32, FastEthernet1/1
O172.16.25.0[110/2]via172.16.12.2,01:12:32,FastEthernet1/1
O 172.16.23.0 [110/2] via 172.16.12.2, 01:12:32, FastEthernet1/1
C172.16.12.0isdirectlyconnected,FastEthernet1/1
C 172.16.13.0 is directly connected, FastEthernet1/0
C 172.16.15.0 is directly connected, FastEthernet2/0
10.0.0.0/32issubnetted,2subnets
O 10.0.0.2 [110/2] via 172.16.12.2, 01:12:32, FastEthernet1/1
C 10.0.0.1 is directly connected, Loopback0
…And now the FIB on MPLS1. Take note of the similarities and in particular the next hop addresses.
MPLS1#show ip cef
Prefix Next Hop Interface
0.0.0.0/0dropNull0(defaultroutehandlerentry)
0.0.0.0/8 drop
0.0.0.0/32receive
10.0.0.1/32receive
10.0.0.2/32 172.16.12.2 FastEthernet1/1
127.0.0.0/8drop
172.16.12.0/28attachedFastEthernet1/1
172.16.12.0/32 receive
172.16.12.1/32 receive
172.16.12.2/32 172.16.12.2 FastEthernet1/1
172.16.12.15/32 receive
172.16.13.0/28 attached FastEthernet1/0
172.16.13.0/32 receive
172.16.13.1/32 receive
172.16.13.15/32 receive
172.16.15.0/28 attached FastEthernet2/0
172.16.15.0/32 receive
172.16.15.1/32 receive
172.16.15.15/32 receive
172.16.23.0/28172.16.12.2FastEthernet1/1
172.16.24.0/28 172.16.12.2 FastEthernet1/1
172.16.25.0/28 172.16.12.2 FastEthernet1/1
224.0.0.0/4drop
224.0.0.0/24receive
240.0.0.0/4 drop
255.255.255.255/32 receive
Next, is the ARP table on MPLS1..followed by the Cisco Express Forwarding Adjacency table.
MPLS1#show ip arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet172.16.13.1-ca00.0bd0.001cARPAFastEthernet1/0
Internet172.16.12.1-ca00.0bd0.001dARPAFastEthernet1/1
Internet172.16.12.273ca01.0bd0.001cARPAFastEthernet1/1
Internet172.16.15.1-ca00.0bd0.0038ARPAFastEthernet2/0
MPLS1#showadjacencydetail
ProtocolInterfaceAddress
TAG FastEthernet1/1 172.16.12.2(7)
0 packets, 0 bytes
CA010BD0001C
CA000BD0001D8847
TFIB 02:48:53
Epoch: 0
IPFastEthernet1/1172.16.12.2(17)
0 packets, 0 bytes
CA010BD0001C
CA000BD0001D0800
ARP 02:48:53
Epoch: 0
The correlations here should all be apparent. Notice the last 4 digits on the line under the bolded MAC addresses. These are ethertype codes. 8847 is MPLS-IP. 0800 is Ethernet.
That about brings Cisco Express Forwarding Part I to a conclusion. That should provide you with a foundational knowledge of what CEF does and how it works. There are quite a few more details to be covered in later articles. Right now I just to get this introduction out there because we will be needing it for MPLS Part III.
About the Author:
Joe Doran is a Cisco certified engineer working toward the prestigious
Cisco CCIE Certification. His blog details the journey and provides free tips and insight into this highly sought after certification track. Joe has a new article on
MPLS