For sometime now, I’ve gotten several questions on how a data flows from one end-device to another. Whether it’s from a student, colleague, and even customers. Interestingly enough – I’ve heard that this question is also asked as part of technical interviews for various IT positions (Pre-Sales, NOC, SOC, SD-WAN, etc…)
In all likelihood this question comes up all the time, unconsciously, in our minds when we’re troubleshooting the fabric for connectivity issues. Aside from the ever-so faithful PING test – understanding how a packet travel to-n-fro is the second item on the troubleshooting agenda.
NOTE: It’s obvious that the physical connectivity is in place or else you don’t have a fabric… So for all intent an purpose for this blog, the physical connectivity is working as proposed.
DISCLAIMER: The material presented in this blog is for educational and training purposes only. Neither the author(s) nor Ahaliblogger© and AHA-VTS.com© assume any liability or responsibility to any person or entity with respect to loss or damages incurred from the information contained in this blog.
So the question – How does a Packet walk through a Fabric from end-host to end-host?
We’ll look at it from a Layer-2 perspective and then from a Layer-3 perspective
Layer-2 Perspective
Layer-2 Diagram
The diagram consists of four [4] end devices connected to a switch. All devices reside on the same subnet. The IPs themselves are moot for the moment with the exception of the last octet. The MAC addresses are repeating sequences [i.e. aaaa.aaaa.aaa, etc.]
Layer-2 Packet Walk
Step 1: Host OO has data is want to send to Host RR
Host-OO has some data it needs to send to Host-RR. Host-OO needs to do is to determine ‘if’ the data to be sent is for its own subnet or not. An AND operation is performed between source IP address, source subnet mask and destination IP address, destination subnet mask. If the resultant of both is the same, then the destination on the same network otherwise on a different network.
Along with the data, Host-OO then constructs an Layer-3 header consisting of the source and destination IPs; .13 & .22, respectively.
Once this is done, Host-OO needs to construct the layer-2 header. So it check its ARP table for IP to MAC address mapping. In our case – there is no entry for Host-RR. Therefore, Host-OO will need to send an ARP request.
Step 2: ARP Request
The ARP [Address Resolution Protocol] request from Host-OO asks: ‘Does anyone know or has a MAC address that corresponds to IP .22?’ Host-OO sends the ARP request.
The ARP request reaches Switch-X on port 5. At this point Switch-X will update its MAC table with the Port & MAC address of Host-OO.
Switch-X will next send the ARP request to all devices, except the one it received it from. This is called ARP flooding.
The request reaches Hosts: AA, RR, and CC. Since Hosts AA & CC doesn’t own the IP address of .22, they will both discard the request. Host-RR sees its for itself and learns the ARP mapping of the device that sent it [Host-OO]. Host-RR will update its own ARP table.
Step 3: ARP Reply
Host-RR will generate an ARP reply to Host-OO. This will be a unicast frame. Switch-X receives the response on port 22. At this point, Switch-X will update its MAC table with an entry for Host-RR.
Since this is a unicast frame destined for Host-OO, Switch-X already has an entry in its MAC table for Host-OO from the previous ARP request. Based on the MAC table, Switch-X will forward the frame out Port 5.
Step 4: Host-OO updates its ARP table
Once Host-OO receives the ARP reply – it will update its ARP table with the MAC address of Host-RR.
Step 5: Completion of Datagram
Now Host-OO has all the information it needs to build the Layer-2 header with the source and destination MAC address for this data[gram].
Step 6: Host-OO can now send data to Host-RR
Once all the pieces are assembled… Host-RR can send the data to Host-RR.
And we’re done! Now – you may think this may take a long time for this process to occur, but it happens in mere milliseconds.
Let’s move onto Layer 3 Packet Walk.
Layer-3 Perspective
Layer-3 Diagram
This diagram, similar to the previous Layer-2 diagram, consists of additional devices: a router and another subnet network.
The IPs are more significant than in the previous. The network to the left of the router has the IP subnet of 33.33.33.0/24 and to the right of the router, 108.108.108.0/24. This time data needs to be sent from Host-OO to Host-HH.
Because another network has been added to where data will be sent. Six tables will be looked at by their respected devices as shown in the diagram below.
Router-1’s routing table is populated with the corresponding networks to which it’s directly attached to; Eth-23 has the IP address of 33.33.33.1/24 and Eth-46 has the IP address of 108.108.108.1/24.
Layer-3 Packet Walk
Step 1: Host OO has data is want to send to Host-HH
Again, Host-OO needs to determine ‘if’ the data to be sent is for its own subnet or not. An AND operation is performed between source IP address, source subnet mask and destination IP address, destination subnet mask. If the resultant of both is the same then the destination on the same network otherwise on a different network. This time, the destination host resides on another network…
Because the destination host is on different [foreign] network, Host-OO will need to send the data to its default gateway [Router-1].
Along with the data, Host-OO then constructs a Layer-3 header consisting of the source and destination IPs; 33.33.33.13 & 33.33.33.1, respectively.
At this point, Host-OO needs to construct the layer-2 header. So Host-OO will check its ARP table for Router-1’s IP to MAC address mapping. In our case – there is no entry for Router-1. Therefore, Host-OO will need to send an ARP request.
Step2: Host-OO sends ARP request
Host-OO creates an ARP request and send it out to the switch it’s connected to, Switch-X.
Switch-X receives the requests and makes note of the end devices that sent it in its own MAC table. It will record Host-OO on Port-5.
From here, Switch-X will send the request out all ports with the except of the port it was received on. This is call ARP Flooding.
Since the ARP request doesn’t contain the IP for Hosts AA, CC, and RR – they will immediately drop the request while Router-1 accepts it.
NOTE: Even though these Hosts may not have the IP/MAC mapping for Host-OO, they will not update their ARP tables.
Step 3: Router-1 send ARP reply to Host-OO
Now, Router-1 accepts the ARP request and will update its ARP table with Host-OO’s MAC address then returns an ARP reply with its MAC address. At this point – the reply will be unicast as Router-1 knows the IP/MAC mapping for Host-OO.
Switch-X received the, now, unicast ARP reply and populates its MAC table with the Port/MAC address of Router-1.
Since Switch-X already have in its MAC table the Port/MAC address of Host-OO, it forwards the ARP reply to Host-OO on Port-5.
Once Host-OO receives the ARP reply – it can now construct the L2 header with source and destination MAC of Host-OO and Router-1, respectively.
Step 4: Host-OO sends the data to Router-1
With the data, Layer-3 header, and Layer-2 header in hand, Host-OO can now send the data to Router-1.
Switch-X will receive the information, update its MAC table [if needed] and forward the information to Router-1.
Router-1 will strip the Layer-2 header and look at the Layer-3 content. Router-1 sees that the data is destined for the IP of 108.108.108.2. Router-1 consults its Routing table and determines it needs to forward the information out Eth-46, which is directly connected to the network 108.108.108.0/24.
Because Router-1 is directly connected to the destination network, Router-1 must deliver the data to Host-HH. However, Router-1 doesn’t know the MAC address of Host-HH, therefore it must send an ARP request.
NOTE: The data and Layer-3 header has NOT changed and is intact!
Step 5: Router-1 send ARP request
Router-1 creates an ARP request and send it out to the switch it’s connected to, Switch-Y.
Switch-Y receives the requests and makes note of the end devices that sent it in its own MAC table. It will record Router-1 on Port-48.
From here, Switch-Y will send the request out all ports,except the port it was received on. This is call ARP Flooding.
Since the ARP request doesn’t contain the IP for Hosts DD & PP – they will immediately drop the request.
NOTE: Even though these Hosts may not have the IP/MAC mapping for Router-1, they will not update their ARP tables.
Step 6: Host-HH send ARP reply to Router-1
Now, Host-HH accepts the ARP requests, will update its ARP table with Router-1’s MAC address and returns an ARP reply with its MAC address. At this point – the reply will be unicast as Host-HH knows the IP/MAC mapping for Router-1.
Switch-Y received the, now, unicast ARP reply and populates its MAC table with the Port/MAC address of Host-HH.
Since Switch-Y already have in its MAC table the Port/MAC address of Router-1, it forwards the ARP reply to Router-1 on Port-48. Once Router-1 receives the ARP reply – it can now construct the L2 header with source and destination MAC of Router-1 and Host-HH, respectively.
Step 7: Router-1 sends the data to Host-HH
With the data, Layer-3 header, and Layer-2 header in hand, Router-1 can now send the data to Host-HH. Router-1 sends the information to Switch-Y.
Switch-Y will receive the information, update its MAC table [if needed] and forward the information to Host-HH.
Once Host-HH gets the information – it will strip off the Layer-2 and Layer-3 headers, as they’re no longer needed and process the information from Host-OO.
All done! This may seem like a lot of steps, but again – it’s done in a matter of milliseconds. 😊
What if Host-HH needs to send return traffic to Host-OO?
Excellent question! Sending information back to Host-OO will be a lot easier as all the devices in the path between both Hosts have all the required information.
Hopefully I can sum it up in this one diagram below:
Here we go:
- Host-HH has return information for Host-OO
- Host-HH constructs the data and Layer-3 header. Because it knows that Host-OO is on a different network, it must send the information to its default gateway of 108.108.108.1 (1a)
- The Layer-2 header has the source MAC of Host-HH and destination MAC of Router-1, as per its ARP table (1b)
- Host-HH send the traffic to Switch-Y which has an entry for Router-1’s MAC in its MAC table and forwards the information out Eth-46
- Router-1 strips the L2 header and review the Layer-3 information
- Router-1 consult it’s routing table and determines the IP 33.33.33.13 resides on the network 33.33.33.0/24 on Eth23 (3a)
- Since Router-1 is directly connected to this network – it must forward the information to the final destination. (3b)
- Router-1 has the MAC for Host-OO from its ARP table and create the new Layer-2 header with source MAC [Router-1] and destination MAC [Host-OO] (3c)
- Router-1 send the frame to Switch-X which has an entry for Host-OO in its MAC table and forwards the information out Port-5
- Host-OO gets the information, strip off the Layer-2 & 3 headers and process the data…
And there you have it! Complete end-to-end Packet Walk from both Layer-2 and Layer-3 perspectives.
Check out the video below on Layer-2 and Layer-3 Packet Walks! https://web.archive.org/web/20200201151613if_/https://www.youtube.com/embed/qIECIOj0chw?feature=oembed
Hope you enjoyed this Blog!
