Networking can be a pain in the ass if you’ve don’t have a deep knowledge.
This is the same regarding virtual networking.
So this post series will focus on virtual switches, how they work, configuration and operation/troubleshoot.
- A bit more than basic knowledge about networking
- Using the ESXi shell
- Knowledge about vSphere environment
- Distributed switch
Frames – Data + header of Layer 2, aka ethernet frame for instance.
What Is Virtual Switch?
A virtual switch is a software layer inside the hypervisor (ESXi) which can forward and filter L2 frames to virtual machines.
Not like a physical switch, the virtual switch learns about the MAC addresses by two ways:
- from the vmx file. it just read the MAC when powering on a VM and then it knows to which port to forward frames.
- by MAC changes from the guest OS (will be explained later).
What can we configure on virtual switch?
- Security settings (L2)
- Quality of Service in means of bandwidth (not the standard QoS – 802.3ad tag)
- Port groups (multiple ports which have the same configuration) -VLAN / Trunk is the main difference between port groups
- Uplink topology (LACP, Active-Passive, etc.)
Types of Virtual Switch
In vSphere there are several kinds of virtual switches:
- Standard Switch – switch with local configuration for single ESXi. you have to manually configure each standard switch to get the results you want.
- Distributed Switch – A switch configuration which is deployed to multiple ESXi. it’s important to know:
- Distributed switch is in fact a control plane for all the instances inside the ESXi. that means that the configuration is done inside VC and then deployed to all instances which is registered with the DVS
- Distributed switch is also an API so 3rd party vendors can implement their own DVS with advanced features. Nexus 1000v is an example, which implements its control plane outside the VC (the control plane is VS. M and the instance/module in ESXi called VEM).
Standard switch is simple whereas distributed switch has advanced features (like bandwidth limitation, LACP, etc.).
So you ask why to use standard switch?
- You don’t have the appropriate license (enterprise plus).
- Easier to manage from ESXi shell. in fact by time I found the standard switch the most suitable for the management vmkernel, because one can troubleshoot network connectivities easier without found himself restoring to standard switch and then reconfigure DVS.
Not to be confused – vCNS / NSX creates logical switch which is not exactly the same. they are parallel to the virtual switch and as a matter of fact uses virtual switch as their uplinks. (NSX explained in another post).
Background to vnic and MAC addresses
for each vnic of virtual machine, there is a default MAC address which is defined in the vmx file.
guest OS can change this default MAC when they boot (or whenever you run a command to change it). this includes Linux and
In Linux you can run the command ifconfig to change interface MAC:
ifconfig eth0 hw ether “new mac seperated by : “
In windows you can change MAC from device manager in the properties of your ethernet card (in the advanced tab).
Virtual switch does not have the same mac address table like Cisco switches. it only holds current MAC addresses of each vnic, so one port cannot be relate to several MAC addresses like physical switch. you need to know that if you’re going to receive several MAC addresses to the same virtual machine. unfortunately it’s the only way. it will put more load on each virtual machine and will be less secured, but at least you will be able to send packets to several MAC addresses.
You’ll see that when you configure nested ESXi. from each vnic you need to receive several destination MAC packets:
- for each vmkernel port. yes! VMKernel port has different MAC (except vmk0????? [validation needed]).
- for each virtual machine.
without that, you will be able to receive packets for vmk0 only! (might be another vmkernel port but this is the default behaviour).
vmk0 gets by default the MAC address of the first vmnic (vmnic0 or eth0 if you’re a linux guy and wants to understand).
by the way, if you clone nested ESXi, the vmknic0 will keep the previous MAC so you will have duplicate MAC.
Port Group Security and Network Packets
Mac Address Changes
As I told before, it’s possible to change the adapter’s MAC address from the guest OS. this parameter determine if the virtual switch will accept MAC changes for each vnic.
refers only to inbound packets from outside to the virtual machine (receive, not transmit). in the vmx file there is a MAC address for each vnic. if the packets go outside the VM have different source MAC, then the packet will be dropped!
use this whenever there are manual/software changes in MAC addresses. (Load Balancer, nested ESXi).
avoid accepting it will drop all inbound packets!
refers only to outbound packets from the virtual machine outside (transmit, not receive). the MAC address of the source is compared to the current MAC of the virtual machine (the one that is configured in the guest OS, not the one that is written in the VMX).
use this whenever the VM is sending ethernet frames with different source MAC address.
avoid accepting it will drop all outbound frames which has a different MAC than the main MAC registered with the vnic.
Promiscuous Mode Operation
all packets which received by the port group will be forwarded to all its ports.
use this whenever the VM has to listen to more than one MAC.
avoid accepting it will drop all inbound packets which has a different MAC than the main MAC registered with the vnic.
Port Groups and VLANs – how it works?
Each port group can be configured for VLAN tagging, without VLAN tagging or trunk.
What are the differences exactly and how each one of them acts?
None VLAN tagging
Each one of the inbound/outbound packets will be forwarded as is, as long as there is no VLAN tagging inside the header. there is no changes at the ethernet frame.
use this setting if you have a flat network (without VLANs), if your physical NIC connected to the virtual switch configured for tagging or configured for native VLAN.
Each one of the inbound/outbound packets will be manipulated before forwarded.
For inbound packet the tagging will be removed.
For outbound packet the tagging will be added.
That means the VM assume there is no tagging at all! just like working with flat network.
use this setting when the physical NICs configured for tagging and you want to configure mulitple VLANs for your VMs.
Each one of the inbound/outbound packets will be forwarded as is. there is no changes at the ethernet frame.
use this setting when you have VMs which needs to manipulate several VLANs. (Software router for instance).
In the first post we’ve learned what is the virtual switch exactly, how it works, what are the features.
So now I hope it will be easier for you to plan how to build it and what are the options you might enable.
In the next post I’ll review how to troubleshoot and run commands refers to the virtual switch.