...

1. Set apache::service_ensure: 'stopped' in hiera for the node that you are upgrading
2. Run puppet with the yoga modules/tags, run apt-get dist-upgrade, and run puppet again
   1. The first puppet-run complains a lot; as it changes its logic for openstack auth all openstack-changes fails; run puppet once more if this bugs you
3. Run keystone-manage doctor and ensure nothing is wrong
4. Run keystone-manage db_sync --expand
   1. Returns nothing
5. Run keystone-manage db_sync --migrate
   1. Returns nothing
6. At this point, you may restart apache2 on this node
   1. Remove the  apache::service_ensure: 'stopped' previously set in hiera.
7. Upgrade keystone on the other nodes, one at a time
   1. Basically run step 1, 2 and 6 on the other nodes
8. When all nodes are upgraded, perform the final DB sync
   1. keystone-manage db_sync --contract

Glance

To upgrade glance without any downtime you would need to follow the following procedure:

1. Select which glance-server to upgrade first.
   1. In the node-specific hiera for this host you should set: glance::api::enabled: false 
2. Run puppet with the xena modules/tags, run apt-get dist-upgrade, and run puppet again
3. Run glance-manage db expand
4. Run glance-manage db migrate
5. Remove the glance::api::enable: false from the node-specific hiera, and run puppet again. This would re-start the glance api-server on this host.
   1. Test that this api-server works.
6. Upgrade the rest of the glance hosts (ie; step 2 for each of the remaining glance hosts)
7. Run glance-manage db contract on one of the glance-nodes.

Enable glance quotas through keystone unified limits

Register If you want to add quotas to limit tenants possibility to use too much storage for their images you need to register default-quotas in keystone. Substitute "SkyLow" with the relevant region-name:

...

ntnuopenstack::glance::endpoint::internal::id: '<GLANCE INTERNAL ENDPOINT ID>'
ntnuopenstack::glance::keystone::limits: true

Cinder

To upgrade cinder without any downtime, follow this procedure

1. Add the following three lines to the node-file of the first node you would like to upgrade:
   

   1. apache::service_ensure: 'stopped'

   2. cinder::scheduler::enabled: false

   3. cinder::volume::enabled: false

2. Run puppet with the xena modules/tags, run apt-get dist-upgrade, and run puppet again
3. Run cinder-manage db sync && cinder-manage db online_data_migrations
4. Remove the lines added at step 1, re-run puppet, and test that the upgraded cinder version works.
5. Perfom step 2 for the rest of the cinder nodes