When a VM is scheduled on a certain compute-node the VM's resources are reserved on the compute-node. The scheduler takes care to not schedule more VM's than the compute-node is able to host. The placement-service keeps track of these resources, and it allows a certain amount of control onto which VMs are hosted on a certain compute node. You can basicly tune:
- Overcommitment: To what extent do we allow to reserve more resources than what is physically present?
- When allowing to hand out more resources than what we have (ie: overcommitment) you should keep an eye to the real resource-usage of the compute-node.
- Minimal size: Do we restrict how small resource-reservations we allow on the node?
- To avoid having a trillion small reservations on large providers.
- Maximum size: Do we restrict how large resource-reservations we allow on the node?
- To avoid having too few requiring users sharing a small node.
- Static reservations:
- Reservations for the provider itself. For instance some RAM for the hypervisor.
Display current resource inventory
To manipulate the inventory of a compute-node you need the uuid of that compute-node's resource-provider in placement:
$ openstack resource provider list +--------------------------------------+-------------------------------------------------+------------+ | uuid | name | generation | +--------------------------------------+-------------------------------------------------+------------+ | 038a659d-6577-4479-b450-a6953ebcae13 | compute02.infra.skylow.iik.ntnu.no | 4620 | | e5956494-254f-4556-b942-2a899700f173 | compute01.infra.skylow.iik.ntnu.no | 5245 | | 2b349c50-da83-402b-a95c-87df42125041 | compute03.infra.skylow.iik.ntnu.no | 5365 | | 84beb0c5-e1d1-48eb-b545-cc091812c335 | gpu01.infra.skylow.iik.ntnu.no | 604 | | 0f9253b5-089e-400c-9b49-2957b0aa6668 | gpu01.infra.skylow.iik.ntnu.no_pci_0000_3f_00_0 | 9 | | 193f92d9-d46f-4d8e-93ba-08283c285e4d | gpu01.infra.skylow.iik.ntnu.no_pci_0000_3d_00_0 | 16 | | b648ecbf-50b4-4f76-ba23-26404b8517c0 | gpu01.infra.skylow.iik.ntnu.no_pci_0000_3e_00_0 | 10 | | 16ad55c0-aeda-4d4d-a2ff-9eba51dbfdc4 | gpu01.infra.skylow.iik.ntnu.no_pci_0000_40_00_0 | 18 | +--------------------------------------+-------------------------------------------------+------------+
To see the resource-settings for compute01 you can use the following command:
$ openstack resource provider inventory list e5956494-254f-4556-b942-2a899700f173 +----------------+------------------+----------+----------+----------+-----------+-------+ | resource_class | allocation_ratio | min_unit | max_unit | reserved | step_size | total | +----------------+------------------+----------+----------+----------+-----------+-------+ | VCPU | 16.0 | 1 | 16 | 0 | 1 | 16 | | MEMORY_MB | 1.0 | 1 | 48281 | 512 | 1 | 48281 | | DISK_GB | 1.0 | 1 | 8518 | 0 | 1 | 8518 | +----------------+------------------+----------+----------+----------+-----------+-------+
From the output above we can see that the compute-node have three types of resorces: VCPU, MEMORY and DISK. Each of these resources have som properties:
- allocation_ratio: What amount of overcommitment do we allow? For the example above we see a ratio of "16.0" for VCPU, meaning that we allow reserving 16 times as many virtual CPU's as we have physical CPUs. Each megabyte of memory we only allow to reserve once.
- min_unit: The smallest reservation we allow of a certain resource. This is per default 1 for all resources. Setting this to "4096" for the MEMORY_MB resource would for instance make sure that the specific provider only allows VM's with at least 4GB of memory.
- max_unit: The largest reservation we allow of a certain resource. This is per default all the resources available, allowing a single VM to take all available resources, but can be changed as we like. For instance we could set the VCPU's max_unit to 4 to only allow VM's with up to 4 CPU's on this provider.
- reserved: Some resources can be reserved for the hypervisor. Defaults to 512 MB RAM, but this value should probably be a bit higher for our large compute-nodes to consider the amount of memory used for page-tables and such.
- step_size: What increments do we allow in resource reservations? Do we allow 1, 2, 3, 4 CPU's, or only 2 and 4? Step size can limit the sizes we allow.
- total: The amount of resources the provider have of the given type. The example above have 16 VCPU's, and with an allocation_ratio of 16 that means that the provider allows a total reservation of 256 VCPUs.
Modify resource inventory
To change the allocation-ratio for memory you could use the following command:
$ openstack resource provider inventory set --resource MEMORY_MB:allocation_ratio=1.1 --amend e5956494-254f-4556-b942-2a899700f173 +----------------+------------------+----------+----------+----------+-----------+-------+ | resource_class | allocation_ratio | min_unit | max_unit | reserved | step_size | total | +----------------+------------------+----------+----------+----------+-----------+-------+ | VCPU | 16.0 | 1 | 16 | 0 | 1 | 16 | | MEMORY_MB | 1.1 | 1 | 48281 | 512 | 1 | 48281 | | DISK_GB | 1.0 | 1 | 8518 | 0 | 1 | 8518 | +----------------+------------------+----------+----------+----------+-----------+-------+
Now the compute-node allows an 10% overcommitment with the regards to memory.
It is also possible to change the allocation_ratio for all compute-nodes with a oneliner:
$ for id in $(openstack resource provider list --resource VCPU=1 -f value -c uuid); do openstack resource provider inventory set --resource MEMORY_MB:allocation_ratio=1.1 --amend $id; done