“vCloud Software-Defined Networking and Security” or “vCNS” is collective term for series of technologies that come together to deliver on the software-defined datacenter vision. Before I get into the nuts and bolts of how it works, I think it’s worth restating why we have taken this direction. But before I do want to praise the folks how have develop the technology to this state so far. It was announced at last year VMworld, and within a year its technology that customers can use starting with vSphere5.1 – that is some achievement.
So firstly, lets begin with a honest assessment of the way networking is currently handled. Ostensibly the status quo hasn’t be significantly challenged in number of years. In fact virtualization its first incarnation was still pretty much tied to the way physical infrastructure had been managed from some decades. That’s created two main pain-points. In some cases its stymied the rate of virtualization – and it can also cause an issue when it comes with offering the flexibility demand by the cloud. Lets look at some very practical examples by the existing world. Very often you will find the resources that make up a series of clusters different racks. Although vCloud Director 5.1 can aggregate the resources across more than one cluster – these different cluster often represent separate network and storage silos or domains. If I have resources in ClusterA in RackA that becoming scarce, and I have resources available in ClusterB in RackB – how do I make them seamlessly available to all of my tenants in the cloud. Remember my tenants don’t care about the underlying resources or the “plumbing” as some folks have called it. There are some other challenges – each new VM or vApp needs an IP allocation and VLAN. Its not as if IP address are particularly portable which makes moving that workload to another environment very easy, and once it is super-glued to a VLAN relocating to different network segment becomes troublesome. It’s worth saying there are some limits imposed by the physical world such as the number of VLAN that are addressable currently 4096, and there limits on the number of nodes you can have in Layer2 domain. So simply flattening the network by reducing the number of networks using the existing physical constructs isn’t particular viable. Finally, putting these technical issues aside – these limits have to be filtered through the prism of change management. With folks in the virtualization layer or cloud layer potentially making requests to other silos of specialism who plates are already full with other tasks. Let just say your request for change may not be at the top of their respective “to do” list.
So it from this perspective that we have been developing technologies with the vCNS aspect of the Cloud Suite – were being very flexible here. In both working with partners (like Cisco) to develop these technologies that one-day maybe recognised standards, whilst at the same time allowing our partners (such as F5 Networks) to operate freely in this area. Whilst I hope customers will buy into our entire stack (it would be bit made not to have that hope) its by no means mandatory that you do. So I don’t we are creating a monolith here. In this post I want to focus on the VXLAN technology – or Virtual Extensible Local Area Network to give its full . But before I do let see where it sits in the architecture of vCNS.
As you can see there’s three layers to the model. VXLAN sits within the “logical” network, whereas the Edge, Endpoint and App components reside within the “Logical Network Services”. At the top is the “extensible platform” that allows for our third-party partners to interface with vCNS. If you like VXLAN sits between the vSphere layer and higher level constructs of the Organizational VDC. Perhaps this diagram will make it a little clearer.
Yes, I know that image does have the word “elastic”. Sorry about that!
Moving up the stack you can see “Edge Gateway” as providing services like VPN and Load-balancing to the vDC; App providing firewall services that are designed for the objects you find in vCenter and Data Security to protect against leaks of data. So conceptually VXLAN sits on top of the Distributed vSwitch – and it adds data to the Ethernet frame – and its encapsulated and de-encapsulated on the fly. If you like just as ESXi hypervisor introduced a compute virtualization layer that separated the OS from the underlying hardware, VXLAN is adding a network virtualization layer to properly isolate the cloud tenant from the underlying plumbing. This model has advantages in the cloud environment but it also pays dividend elsewhere in making the VM portable (from a network perspective) from site to another – so there’s definitely benefits from DR perspective. There’s the promise not having to re-IP VMs or monkey around with router configurations, NAT or stretched VLANs. So how does work?
VXLAN adds an additional 50bytes to the frame, and essentially encapsulates a Layer2 Frame inside a UDP Frame. The VXLAN ID value is 24bits long and allows for 2x24 networks/subnets. That’s about 16 million subnets (or 16,777,216 to be more exact) in total that compares quite favourably to the 4096 VLANs that current possible in the physical world. It does require multicast IP to work – but the important thing is that it is seamless to the VM. It has no idea that VXLAN is at work – and physical switch has no idea that VXLAN is present either – in that it doesn’t see the MAC addresses or IP’s in use. The other important thing to note that the work on VXLAN has been with other stakeholders in the industry most notably Cisco, Citrix, Red Hat, Broadcom, Arista and others – and its been submitted to the IETF. I personally that’s a pattern you will see a lot in the coming years. Vendor innovation in the world of software-defined networking becoming recognised as industry standards – as opposed to a fractured open-source initiatives that struggle to become recognised global standards as they lack the engineering clout or industry contacts to make it to the table. For customers the important thing is we arrive at standard – how we get there is perhaps less important to them.
Of course for VXLAN to work their needs to be module added to the ESX host so it can interpret the frames. I personally see it in the same light that VLAN Tagging happens now except that was industry standard long before ESX became the dominant platform for server-based virtualization. It’s the Edge Gateway services that responsible for creating the “virtual wires” that make up the VXLAN identifies. Due to this design there some pre-requites required to get it up and running. Firstly, IP Multicast Forwarding. Secondly, increasing the MTU value to 1550bytes to accommodate the additional VXLAN data without fragmentation of packets taking place. Finally, if you going through a router – proxy ARP enabled. The good news is we have worked with my partners (Cisco, Emulex and so on). These partners are looking to bake in support into their NICs and Switches for support for VXLAN. This should ease the adoption, and help offload some of the overhead associated with encapsulation/de-encapsulation. You should watch out for announcement at VMworld as well, later this year and next…