Virtualization Oversubscription - What’s so scary? 18 of 20

Happy New Year!

If you’ve been following my series then you’ll know that just before the Holidays I said that I’d deal with what’s the worst that can happen when you oversubscribe.

So what’s the worst that can happen?

Well if you push things too far, all those things that the Hypervisor can do to try and keep things running will eventually be overwhelmed.

If you try to use too much memory you’ll start to see ballooning on a consistent basis, then swapping.  At that point performance will degrade rapidly.  Watch active memory values and take ballooning increasing as the indication things are getting tight.

CPU is as always a more gentle decay in performance.  CPU also has it’s indicators that the limits are being approached.  CPU Ready and Co-Stop are indicators that VMs are finding it tricky to find CPUs when they want to do some processing.

The reason CPU degrades differently to Memory is that it’s used differently.  A process is in memory all the time, but only uses a CPU when it needs so CPU busy is dictated by how frequently the CPU is required and for how long.  The performance of a transaction will be dictated by the ‘chance’ that a CPU will not be available when the transaction arrives.  If all the CPUs are busy it’ll enter a queue and this is where queueing theory comes in.

Contention and Queuing

Any system has a finite set of resources.  If you only have a single user trying to use one workstation then there is no contention for the use of that workstation.  As soon as you have more than one user then there is a chance that they will want to use the workstation at the same time.  That’s contention.  It’s perfectly normal and happens inside every OS all the time.  There are lots more process threads than there are CPUs, and when there is contention, then the processes queue.  Poor performance only occurs when queueing becomes excessive.

On Friday I'll go in to more detail about the basic ideas of queuing. In the meantime register for our first webinar of 2017 'Performance Management made easy'

http://www.metron-athene.com/services/webinars/capacity-management-webinars.html

Phil Bell

Consultant

Virtualization Oversubscription - What’s so scary? VMware CPU and Memory Maximums (5 of 20)

CPU and Memory are the main items people consider for Virtualized systems, so let’s lay down the maximums.

•        Virtual Machine Maximum

–       128 vCPUs per VM

•        Host CPU maximums

–       Logical CPUs per host 480 (Logical CPUs being simultaneous threads so that might be 240 hyper-threaded cores.)

–       Virtual machines per host 1024

–       Virtual CPUs per host 4096

–       Virtual CPUs per core 32

•        Maximum with a caveat: The achievable number of vCPUs per core depends on the workload and specifics of the hardware. For more information, see the link below for the latest version of Performance Best Practices for VMware vSphere

https://www.vmware.com/pdf/vsphere6/r60/vsphere-60-configuration-maximums.pdf

This raises 2 points.

·        Clearly it’s ok to oversubscribe CPUs 

·        There is no set number to tell you how much oversubscription is OK.

Memory VMware Maximums

Memory is a lot simpler.

•        6TB per Host

–       Well 12TB on specific hardware

•        4TB per VM

Having set out those few ground rules we can now look at memory oversubscription and I'll be doing just that on Monday.

There are still a few places left on our VMware vSphere Capacity & Performance Essentials online workshop so don't forget to book your place.

http://www.metron-athene.com/services/online-workshops/capacity-management-workshops.html

Phil Bell

Consultant

Virtualization Oversubscription - What’s so scary? (2 of 20) Fear and Misunderstanding

Carrying on from Wednesday I mentioned that ultimately there is a fear in these departments that oversubscription = poor performance. It’s considered to be a 1:1 relationship.  The reason for this is, to some extent, a misunderstanding of what oversubscription is.  It’s got the word ‘over’ in it then it must be bad.  Nothing in our department is ‘over’.  We’re all looking at the same word, they see something bad and I see an opportunity to save some money.  Correct me if you think I’m wrong, but saving money is typically thought to be a good thing.

Flying Navigation by Dead Reckoning

Avoiding oversubscription is a bit like navigating by dead reckoning.

• You know what you started with
• You know what you provisioned
• You know how much is left

In WW2 a bomb was considered to be on target if it was within 5 miles of the actual target and we only managed that with 1 in 5.  Dead reckoning isn’t very accurate on its own.  The situation is much more complex than that and the same remains true for people avoiding oversubscription.

Virtualization Used Capacity by Dead Reckoning

•        You know what you started with

•        You know what you provisioned

•        You know how much is left

This is not especially efficient. The essence of what these sites seem to be doing is this:

We start with a 5 Host cluster that has 120 Logical CPUs, and 180GB RAM.  We’re then going to issue no more than 96 vCPUs and 144GB RAM across the VMs.  This allows for a host to fail and we can still run everything.  We’ll also have great performance because VMs will get a CPU whenever they want it, because it’s theirs, and the same with memory.  All the memory a VM wants is real RAM.

I’m not going to deny that performance will be about as good as it can be, but it’s not going to be terribly efficient.  Chances are you could turn off 2 hosts and still see no impact in performance.  Who wouldn't like to reduce their ESX licence, and related power costs by 20%, while still having a spare host?

So what is oversubscription? I'll deal with this on Monday.

Phil Bell

Consultant 

Virtualization Oversubscription - What’s so scary? (1 of 20)

Overcommit vs Oversubscribe

When I wrote the Title and Synopsis for this blog series I happened to choose the word oversubscription.  It’s been pointed out to me that many people refer to “overcommit” rather than “oversubscribe”. 

Both words appear to be in use to describe this and Google does some nice work to return results using either option.

What led me here? 

In my role with Metron I get to visit lots of different people working in lots of different industries.  They are mostly capacity managers working in IT departments though, so there is some common ground.

Clients

–       “Oh, we don’t oversubscribe”

Over the past year I’ve had a number of mildly frustrating conversations with organisations.  These tend to be newer and/or don’t have a good history of Capacity Management.  The frustration has been around ‘Oversubscription’ in virtualised environments. 

I’ll start talking about how we can monitor the environment to ensure good performance in the future.  When they’ll stop me and say, quite proudly, “Oh, we don’t oversubscribe, we don’t want to impact performance”.  The pride is almost the worst part.  They’ll beam a smile at the senior staff as if to say “We’ve got this, don’t worry”.  They aren’t concerned about the overspend that’s required to have that attitude.

Ultimately there is a fear in these departments that oversubscription = poor performance. I'll be looking at the fear and misunderstanding around this on Friday.

In the meantime there are still a few places available at our VMware Capacity and Performance essentials online workshop, don't forget to book your place now.

http://www.metron-athene.com/services/online-workshops/capacity-management-workshops.html

Phil Bell

Consultant