...

• This documents expects that your cloud is deployed with a recent ussuri tag of the ntnuopenstack repository.
• You have a recent mysql backup in case things go south.
• If you want to do a rolling upgrade, the following key should be set in hiera long enough in advance that all hosts have had an puppet-run to apply it:

  • nova::upgrade_level_compute: 'trainussuri'

  • When the upgrade is finished - set this key to 'ussurivictoria'

The recommended order to upgrade the services are listed below:

...

1. Add the following three lines to the node-file of the first node you would like to upgrade:
   

   1. apache::service_ensure: 'stopped'

   2. cinder::scheduler::enabled: false

   3. cinder::volume::enabled: false

2. Do one of these two alternatives:
   1. Run puppet with the victoria modules/tags, run apt-get dist-upgrade, and run puppet again
   2. Reinstall the node with the victoria modules/tags
3. Run cinder-manage db sync && cinder-manage db online_data_migrations
4. Remove the lines added at step 1, re-run puppet, and test that the upgraded cinder version works.
5. Perfom step 2 -5 for the rest of the cinder nodes

...

1. Pick the first node, and do one of the following:
   1. run puppet with the victoria modules/tags, Run apt-get autoremove && apt-get dist-upgrade
   2. Reinstall the node with victoria modules/tags.
2. Run neutron-db-manage upgrade --expand
3. Run neutron-db-manage --subproject neutron-fwaas upgrade head
4. Restart neutron-server.service and rerun puppet
5. Upgrade the rest of the API-nodes (repeating step 1, and 43)
6. Stop all neutron-server processes for a moment, and run:
   
   1. neutron-db-manage upgrade --contract
7. Re-start the neutron-server processes

...

1. Reinstall the server to CentOS 8
2. Stop all magnum-services by adding the following keys to node-specific hiera, and then make sure to run puppet on the magnum hosts:

   1. magnum::conductor::enabled: false

   2. apache::service_ensure: 'stopped'

3. Run puppet with the victoria modules/tags

4. Run yum upgrade

5. Run su -s /bin/sh -c "magnum-db-manage upgrade" magnum

6. Re-start magnum services by removing the keys added in step 1 and re-run puppet.

Octavia

Octavia must be stopped for upgrades, and can thus be performed on all octavia-hosts at the same time. It might be an idea to keep one set of hosts stopped at old code in case of the need for a sudden roll-back.

1. Stop all octavia-services by adding the following keys to hiera, and then make sure to run puppet on the octavia hosts:

   1. octavia::housekeeping::enabled: false

   2. octavia::health_manager::enabled: false

   3. octavia::api::enabled: false

   4. octavia::worker::enabled: false

2. Do one of:

   1. Reinstall the node with victoria modules/tags

   2. Run puppet with the victoria modules/tags, Run apt-get dist-upgrade && apt-get autoremove, Run puppet

3. Run octavia-db-manage upgrade head


4. Re-start octavia services by removing the keys added in step 1 and re-run puppet.

5. Build a victoria-based octavia-image and upload to glance. Tag it and make octavia start to replace the amphora.

Horizon

1. Run puppet with the victoria modules/tags
2. run yum upgrade

3. Run puppet again
4. restart httpd

Compute-nodes

When all APIs etc. are upgraded, it is time to do the same on the compute-nodes. Compute nodes are simple to upgrade:

1. Do one of:
   1. Reinstall the node with victoria modules/tags
   2. Run puppet with the victoria modules/tags, Run apt dist-upgrade && apt-get autoremove
2. Reboot the compute-node

GPU-nodes

1. Run puppet with the victoria modules/tags
2. Run yum upgrade && yum autoremove
3. Run puppet again
4. Restart openstack services and openvswitch-services

Finalizing

• Run nova-manage db online_data_migrations on a nova API node. Ensure that it reports that nothing more needs to be done.
• Rotate octavia images.