Note: This is based on the R2 Preview – so stuff could change before Microsoft GA their product. I guess they have 6 months to resolve any issue that can arise. So this is not production ready code, but “preview” or what we used to call a “beta” or “release candidate”. I was going to start my evaluation on production version, but I thought it might be more interesting to be exposed to the new features promised by the end of the year.
The phase “Go Team” was one a manager of mine used to say when I was employed in the ‘90s. Whenever she had dropped the ball, and was trying to inspire us to pick up her mess and sort it out for her…. She’d say “Go Team!”
As part of my Cloud Journey, I’ve been inspired to dip my toes into foreign waters and see what life is like on the other side of the fence. I must say I’ve been rather spoiled in the last ten years or so using VMware technologies. It’s only when you start using alternatives that you begin to see how much these competitors are mired in the past.
I recently bought a couple of books to help me learn more about Microsoft Server 2012 Hyper-V. It’s by Finn, Lownds, Luescher and Flynn called “Windows Server 2012 Hyper-V: Installation & Configuration Guide”. I think its good read, and it’s thankfully quite honest from a technical standpoint about the advantages and disadvantages of the platform. The only disappointment is that its main focus is Hyper-V Manager, rather than System Center Virtual Machine Manager (SCVMM). That’s a shame because I would have thought that’s how most sizable organizations would use that technology. In case you don’t know there’s a cheap and cheerful way to manage Hyper-V that is called “Hyper-V Manager”. In reality I would have thought most professional shops would prefer to use SCVMM. So I back filled this book with a copy of Microsoft System Center Virtual Machine Manager 2012 Cookbook by Edvaldo Cardoso. My ideal would be both books sort of mashed-up together.
Reading the Hyper-V book I was taken about how different NIC Teaming support compared to VMware. That was re-enforced by a recent trip to my colocation. I’ve repurposed my hosts in my “silver” cluster to run two instances of Windows 2012 Hyper-V R2, and two instance of RHEL6.x to run KVM. The main reason is to learn more about the technologies in their own right, as well as seeing how management tools like vCloud Automation Center and Multi-Hypervisor Manager work with these foreign virtualization platforms. My goal was to get the colo, install Windows Hyper-V and RHEL and ensure I could communicate to them remotely, so I could do any further configuration from the comfort of my home office. Sadly, some of my Leveno’s don’t have the expensive “media key” that allows complete remote management.
In case you don’t know Windows Server 2012 finally added built-in NIC teaming capabilities. Previously Microsoft offloaded this to 3rd party NIC vendors with customers having to find the exact right driver specifically for their network card. Even after doing that, if there was a problem – your problem was with Intel, Broadcom or who ever made your network cards. It’s their software, not Microsoft’s after all. So from a Microsoft Admins perspective, Windows Hyper-V having native NIC teaming is a step up for them. With advent of Windows 2012 Server Hyper-V, you’d be forgiven to think all is right in the world. Not quite. I personally still don’t think its been done the right way…
Firstly, by default NIC Teaming is disabled in a new installation of Windows 2012 Hyper-V. NIC teaming is buried away in the configuration Windows 2012 Hyper-V host; you’d think the NIC Teaming management would be in the same location as NIC property pages.
That’s because what with Microsoft’s “OS-centric” view of the world where NIC teaming has always been a function of the OS, its not owned/controlled by the management plane of the hypervisor (which is in my view, where it should be, so long as you don’t have some legacy architecture to work around!). I’ve occasionally called this “Operating System Oriented Hypervisors”, and you’d expect to see it in any OS that’s been retrofitted with a virtualization layer or role. Anyway, here’s the steps I had to go through…
1. Logon to the Windows Server 2012 Hyper-V host
2. Wait for Server Manager to load and interrogate the system
3. Select Local Server
4. Under NIC Teaming, click Disabled
5. Under Teams, select the Task button
6. Select New Team
7. Type a name for the NIC Team
8. Select the NICs to be included
9. Configure any additional advanced properties
10. Click OK
11. Repeat and Rinse for all the hosts.
[Yes, I know there’s thing called Powershell. I’m a big fan VMware’s PowerCLI plug-ins for vSphere, View, and vCloud Director – but remember not everyone has the time to master a command-line interface]
Before I started to setup my NIC team I wanted work out the relationship between the physical devices and “friendly” labels as known to Windows Hyper-V. I’m using some Lenovo T200 servers with an additional dual-port card add to the PCI bus. One of the odd things is how ESX, Windows Hyper-V and RHEL enumerate these cards in weird way.
It seems like vmnic0 and vmnic3 (Ethernet and Ethernet4 in Windows Hyper-V) get enumerated differently to vmnic1 and vmnic2 (Ethernet 2 and Ethernet3 in Windows Hyper-V). I think this might come from using non-production, non-standard hardware. But certainly it does seem that regardless of vendor there’s sometimes some funkiness that goes on when enumerating PCI devices correctly. Anyway, I guess this isn’t such a bad outcome. Using different PCI based NICs configured to different switches probably offers more redundancy in the long run.
Using documentation of the previous configuration, and the judicious use of removing NIC cables to identify the devices – I did some renaming of the NICs in Windows Hyper-V. Sure there are probably funkier ways to do this with blades or with the switch management software – but if you dealing with 1Gps interfaces on a conventional 2U/4U rack mount server, and you have physical access its still the simplest way.
I guess my goal here was to create two NIC teams one for “infrastructure” communications (management, IP storage and so on) and separate dedicated NIC for virtual machine traffic that will be most likely tagged for VLAN traffic. By doing the rename process at least when I was making the NIC team in Windows Hyper-V I could at least identify the right NICs to select.
The Additional properties does allow you to indicate the teaming mode (switch independent, static teaming, LACP), Load-balancing algorithm (Dynamic, Hyper-V Port, Address Hash) or whether to have all the NIC active or have an active/standby configuration. Sadly, Microsoft NIC Team does not support “route based on physical NIC load” which was a fourth additional load-balancing method introduced in vSphere5.1
The process of creating a NIC Team can take several minutes to complete, and if you’re are doing this remotely from the management network over Microsoft RDP expect to be disconnected from the server. For example I decided to rename a team called “Management” to be “Management-Team0”, and that caused my Microsoft RDP session to hang. I mean come on Microsoft; it’s just a piece of text! The other thing I noticed is every time you do a rename of NIC team, and error message is generated – which does eventually disappear:
When the NIC team is created you do get a number of errors. I think this is caused by it stripping out the previous IP configuration, and then bringing each of the members of NIC team online. This is bit disconcerting when you see it – but it does eventually indicate that both NIC are active. This process took about 40 second to complete, and did have me double-checking the physical NIC connections due to the erroneous messages – I didn’t have a “Failed Media disconnected”.
Over the couple of hours it took me to write this blogpost, I got just one “Not Responding” style error from the software. I don’t think we can make much of that – R2 is a preview (aka Beta) and so this isn’t production code after all
Doing the NIC teaming from the physical box or using an ILO/DRAC/BMC card is a bit of must. I did try to do this remotely (just to see what it was like) using Microsoft RDP that didn’t shake out as smoothly as I would hope. Creating a NIC team in Windows means the members of the team get their TCP/IP protocol disabled.
That means if “Ethernet” was where your IP settings were, and you were using them to connect remotely to create a NIC team. Things could get a bit sticky. Think of man sitting on a branch with an axe in his hand. For this reason it’s pointless in doing IP configuration upfront before you have the NIC team in place. That’s because in the Microsoft world – you’re TCP configuration is intimately bonded to the interfaces.
In vSphere NIC Teaming is turned on by default. The mere act of adding more than one NIC enables the feature and makes anything connected to that switch have network redundancy. For vSphere the very act of modifying the built-in vSwitch0 or creating a new vSwitch (either distributed or standard) allows you to add network redundancy along the way – on any vSphere host that is under management of vCenter – accessed with single logon. VMware vSphere has no particular requirements to reserve a NIC purely for management usage; instead you merely patch additional NIC’s to the default vSwitch created for management.
The IP configuration for such thing as management, VMotion, IP Storage, Fault Tolerance, High-Heartbeats and so are backed behind the portgroup. You can add remove NIC, create and destroy network teams without worrying about management of the physical layer, screwing up your IP configuration.
As for virtual machine networking – simply creating a NIC team in Windows Hyper-V doesn’t make it accessible to VMs. From a different UI (either Windows Hyper-V manager or SCVMM) you have to handle the networking from there. In contrast with vSphere the VM networking is held all in one place. Create new Standard vSwitch (if necessary), and add in the NIC for teaming:
And then in the same wizard create portgroup with a friendly label, and VLAN Tag ID.
Of course if it was Distributed vSwitch, one would merely have to add portgroup with the appropriate VLAN settings – and that configuration would available on every single ESX hosts that was member of the DvSwitch.
Conclusions
You can’t help feeling that the reason NIC Teaming lies within the control of the “Management OS” (Microsoft new name for an old thing – the Parent Partition) is because that’s where historically for lot of Operating System vendors it has always been located – in the Operating System. For all the grandstanding around Microsoft’s “extensible” virtual switch, the more you look at the architecture of Windows Server 2012 Hyper-V the more it looks just like the same old architecture of Windows Server 2008 Hyper-V. For example, its still the case with this NIC teaming that “child” VM network traffic goes via the “parent” management operating system via what Microsoft calls the “VMbus”.
Link: Overview of the Hyper-V Extensible Switch
Sadly, it’s rather telling that this sort of NIC Teaming has come into the Microsoft station last year, for VMware its been baked in “feature” since ESX 2.x days (circa 2003/4). That means it’s taken nearly 10 years for Microsoft to play catch-up. You’ve got to wonder if it takes Microsoft a decade to play catch-up where will VMware be in 2024?
IMPORTANT: I later discovered this sort of configuration would give me issues with having both MPIO enabled, and meeting the stringent requirements for Microsoft Failover Clustering. I feel I could do with more physical NICs to get my personal, ideal configuration and meet the Microsoft pre-requisites – something I don’t currently need to do with VMware ESX.