Comment 3 for bug 1635367

PiN == provisioned in Nova, PiI == provisioned in Ironic.

How it used to work:
Broke PiN PiI UsedMem AvailMem Reported
    Y Y Y 100 100 0
    Y Y N 100 0 -100
    Y N Y 0 100 100
    Y N N 0 0 0
    N Y Y 100 100 0
    N Y N 100 100 0
    N N Y 0 100 100
    N N N 0 100 100

These changes were made:
https://review.openstack.org/#/c/306670/2
https://review.openstack.org/#/c/321907/1

How it works now:
Broke PiN PiI UsedMem AvailMem Reported
    Y Y Y 100 0 -100 -> 0
    Y Y N 100 0 -100 -> 0
    Y N Y 0 0 0
    Y N N 0 0 0
    N Y Y 100 0 -100 -> 0
    N Y N 100 100 0
    N N Y 0 0 0
    N N N 0 100 100

Basically, only schedule to an Ironic node if it's not broken and neither Nova
or Ironic have it marked as provisioned.

If negative values are allowed, these cases would be considered
over-subscriptions, but they are not. Allowing negative RAM causes confusion,
and misrepresents available memory (controlled by the hypervisor / node). This
can lead to subtle bugs when the memory is not available and oversubscribing is
allowed. One example is the possibility of scheduling to a node that has no
available memory. Another example, negative memory is subtracted from
available memory in cells reporting causing builds to fail even though capacity
is available.

At the time of this writing, I'm not certain to what level over-subscriptions
are broken since the available memory limit is stored at a higher value using
the ram allocation factor. It will take a bit more digging before I fully grok
the impact of disallowing negative values here.

One potential way to solve for both cases is to rework the filters to use the
ram_allocation_factor to reduce the allocation instead of increasing the
available memory limit (https://github.com/openstack/nova/blob/master/nova/scheduler/filters/ram_filter.py#L51).
I believe this would eliminate the need for negative values and still allow for
over subscriptions. I can go down this path if it makes sense, but probably good
to have a conversation first.