Puppet 5 -> Puppet 7

The puppet7 upgrade needs to be done before we can install Focal. The procedure is generally:

Upgrade puppetservers
Upgrade puppetagents
Upgrade postgres
Upgrade puppetdb

This article should guide you through this process.

Upgrade puppetservers

First, you need a puppetmaster that is not behind your loadbalancers. This is possible through one of two ways:

Use one of your existing puppetmasters:
- Add profile::haproxy::register: false to the on of your puppetmasters node-file
- Perform a puppet-run on the puppetmaster
- Perform a puppet-run on the adminlb's.
- Manually upgrade the puppetmaster by installing the puppet7-repos (apt.puppetlabs.com) and perform a dist-upgrade
Install a new puppetmaster with profile::haproxy::register: false in its node-file. Make sure to use the puppet7-repos in the post-install script to make it a puppet7 master.

To verify that the puppet7-master works you can start to direct puppetrequests to it. Update for some nodes in your environment the profile::puppet::hostname to the puppet7 master and see everything works as expected. Remeber to deploy the needed environments to it first.

Point the rest of the puppet-masters (including the CA) to the puppet7-master, add the two following keys to hiera node-files, and then run puppet to upgrade the puppet-agent:

profile::puppet::collection: 'puppet7'
puppet_agent::package_version: 'auto'

At this point your puppetmasters have puppetagent version 7, and puppetmaster version 5. Install puppetmaster version 7 by running a dist-upgrade, and finish by restarting the puppetserver process.

Making sure the CA is allowed to sign new certificates

Starting in puppet 6 the CA commands is using the puppetserver API, and is thus requiring proper authorization. This means one of two:

Either the puppetCA's client cert needs to contain the authorization-extension "pp_cli_auth: true"
or the /etc/puppetlabs/puppetserver/conf.d/auth.conf needs to authorize the puppetca-server by hostname to access the CA-related endpoints.

As we need a functioning CA to sign a new cert with new extensions the upgrade-path uses alternative 2. When a new puppet-server ca is installed you should make sure that the new server gets the extension in its cert. For now update the /etc/puppetlabs/puppetserver/conf.d/auth.conf to contain (replacing the PUPPETCA-FQDN with the real fqdn of your CA:

...
        {
            # Allow the CA CLI to access the certificate_status endpoint
            match-request: {
                path: "/puppet-ca/v1/certificate_status"
                type: path
                method: [get, put, delete]
            }
            allow: {
               extensions: {
                   pp_cli_auth: "true"
               }
            }
            allow: PUPPETCA-FQDN
            sort-order: 500
            name: "puppetlabs cert status"
        },
...
        {
            # Allow the CA CLI to access the certificate_statuses endpoint
            match-request: {
                path: "/puppet-ca/v1/certificate_statuses"
                type: path
                method: get
            }
            allow: {
               extensions: {
                   pp_cli_auth: "true"
               }
            }
            allow: PUPPETCA-FQDN
            sort-order: 500
            name: "puppetlabs cert statuses"
        },
...

After restarting the puppetserver service you should be able to use commands like "puppetserver ca list --all".

Upgrading puppet-agent

To upgrade the puppet-agents in the infrastructure, add the following two keys to global hiera (and if you want, remove them from the puppetmaster node-files) and wait for two puppet-runs.

profile::puppet::collection: 'puppet7'
puppet_agent::package_version: 'auto

Upgrading postgres-servers

Puppetdb for puppet7 needs a newer postgres-version than what we are currently running. To upgrade, do the following:

Reinstall one of the postgres-slaves with 18.04 with role::base
In the node-file of this postgres-slave, set the following keys:
1. profile::postgres::keepalived::enable: false
2. profile::postgres::version: '13'
3. profile::postgres::masterserver: '<fqdn-of-the-new-postgres-master>'
Install role::postgres::master on the postgres-machine
Take a backup from the old postgres-master machine using the command sudo -u postgres pg_dumpall > postgresdb.pgsql
Move this backup to the new postgres-master, and restore it to the server using the command sudo -u postgres psql -f /tmp/postgresdb.pgsql
Change the global hiera-key profile::postgres::ipv4 to point to the IP of the new postgres master, and run puppet on the puppetdb servers.
Install Ubuntu 18.04 with the role role::postgres::slave on the rest of the postgres-machines. While this installation is going, move the hiera-keys in step 2 from the node-file to a global file (key c already exist, so make sure that there is no duplicate).
Use postgres-joinMaster.sh on the slaves to have them replicating their master.

Upgrading puppetdb

At this point, with postgres13 running and puppet7 agents, upgrading the puppetdb-machines are simply a dist-upgrade and restart of the service (or host to get new kernels as well).

Page tree