...
If all of the infrastructure is going to be upgraded to a tested configuration, it is a matter of merging the new configuration to the current branch of the r10k repo. For instance, to upgrade the whole skyhigh platform to the Liberty release a merge of the master-branch at tag "v0.3.0" is performed into the skyhigh branch by the following commands:
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
$ git checkout skyhigh $ git merge "v0.3.0" |
After the merge it can be smart to edit the Puppetfile to also download the same versions of the role and profile repos:
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
mod 'role', :git => 'https://github.com/ntnusky/role.git', - :tag => 'master' + :tag => 'v0.3.0' mod 'profile', :git => 'https://github.com/ntnusky/profile.git', - :tag => 'master' + :tag => 'v0.3.0' |
When one is ready to start the upgrade, the modules of the environment skyhigh can be deployed on the manager like so:
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
root@manager:~# r10k deploy environment skyhigh -vp |
...
Upgrade the Openstack Installation
The openstack installation is currently not in a state that it allows current architecture in our clouds will in a large degree allow live upgrades. The virtual machines can be running, but all the openstack services on the controllers needs to be disabled before one of the controllers are updated.
Upgrade to a previously tested configuration.
This is what we do in production environments. The general upgrade procedure are:
- Stop puppet on all controller nodes, and merge in the correct configuration using r10k.
- Stop ALL openstack services on all controllers.
- Do a puppet-run on the "primary" controller (The one doing the db-sync etc; in SkyHiGh its controller01), which will fail.
- Perform an installation of the newer packages using 'apt-get -y dist-upgrade -o Dpkg::Options::="--force-confnew"'
- Test all api's, and fix db sync errors where those appear. (Typically visible by having errors in the api's, and log messages complaining about missing fields)
- Perform step 2+3 on the other controllers
- Test
- Start puppet on compute-node, let it run once before the 'apt-get -y dist-upgrade -o Dpkg::Options::="--force-confnew"' is issued to install newer packages.
- Test and verify a successful upgrade.
Upgrade to a new verison, and test it in skylow.
Unfinished notes:
...
upgrades are performed openstack project for openstack project, and the specifics for each upgrade will be available in the openstack release-notes. We should also write important considerations in our puppet release-notes, and in specific articles in this wiki (child-pages of the page you are currently reading). Various openstack-projects depends on eachother, and you should thus ensure that projects providing services for other projects should be upgraded first.