Page History

...

To verify that the puppet7-master works you can start to direct puppetrequests to it. Update for some nodes in your environment the profile::puppet::hostname to the puppet7 master and see everything works as expected. Remeber to deploy the needed environments to it first.

Shftleader might not pick up the new puppetmaster. In that case, run this on the new puppetmaster:
/opt/shiftleader/manage.py puppet_env2update tulleenv

It will throw an error, but also register the register it in Shiftleader at the same time.. /o\ 

Point the rest of the puppet-masters (including the CA) to the puppet7-master, add the two following keys to hiera node-files, and then run puppet to upgrade the puppet-agent:

...

...
        {
            # Allow the CA CLI to access the certificate_status endpoint
            match-request: {
                path: "/puppet-ca/v1/certificate_status"
                type: path
                method: [get, put, delete]
            }
            allow: {
               extensions: {
                   pp_cli_auth: "true"
               }
            }
            allow: PUPPETCA-FQDN
            sort-order: 500
            name: "puppetlabs cert status"
        },
...
        {
            # Allow the CA CLI to access the certificate_statuses endpoint
            match-request: {
                path: "/puppet-ca/v1/certificate_statuses"
                type: path
                method: get
            }
            allow: {
               extensions: {
                   pp_cli_auth: "true"
               }
            }
            allow: PUPPETCA-FQDN
            sort-order: 500
            name: "puppetlabs cert statuses"
        },
...

Also ensure there is something like this in /etc/hosts. It won't work with only the localhost definitions

<ip-address> PUPPETCA-FQDN

After restarting the puppetserver service you should be able to use commands like "puppetserver ca list --all".

Later, to create a client cert with the pp_cli_auth extension added (typically when you install a new CA), do the following:

Install a new server with role::base
On new CA:
# systemctl stop puppet
# rm /etc/puppetlabs/puppet/ssl/certs/FQDN.pem /etc/puppetlabs/puppet/ssl/private_keys/FQDN.pem

On old CA:
# systemctl stop puppetserver

# puppetserver ca generate --ca-client --certname NEW-CA-FQDN --subject-alt-names cool.name.foo,cooler.name.foo
That command will tell you that it has some existing files. Delete all of thoses, and re-run the command

Finally:
Copy the generated /etc/puppetlabs/puppet/ssl/private_keys/NEW-CA-FQDN.pem to the same folder on the new CA, and re-run puppet on the new CA.

You should se something like this from "puppetserver ca list --all": (pp_cli_auth: true is the key part here)
puppetca3.infra.pile.it.ntnu.no              (SHA256)  9C:22:1F:89:C4:C8:C3:BF:F9:59:64:2D:CA:5A:F8:A9:12:02:9C:3E:DB:3D:F7:BD:03:D1:15:F7:BC:6F:84:2E    alt names: ["DNS:puppet.pile.it.ntnu.no", "DNS:puppetca.pile.it.ntnu.no", "DNS:puppetca3.infra.pile.it.ntnu.no"]    authorization extensions: [pp_cli_auth: true]

Upgrading puppet-agent

To upgrade the puppet-agents in the infrastructure, add the following two keys to global hiera (and if you want, remove them from the puppetmaster node-files) and wait for two puppet-runs.

profile::puppet::collection: 'puppet7'
puppet_agent::package_version: 'auto'

Upgrading postgres-servers

Puppetdb for puppet7 needs a newer postgres-version than what we are currently running. To upgrade, do the following:

1. Reinstall one of the postgres-slaves with 18.04 with role::base
2. In the node-file of this postgres-slave, set the following keys:
   1. profile::postgres::keepalived::enable: false
   2. profile::postgres::version: '13'
   3. profile::postgres::masterserver: '<fqdn-of-the-new-postgres-master>'
3. Install role::postgres::master on the postgres-machine
4. Take a backup from the old postgres-master machine using the command sudo -u postgres pg_dumpall > postgresdb.pgsql
5. Move this backup to the new postgres-master, and restore it to the server using the command sudo -u postgres psql -f /tmp/postgresdb.pgsql
6. Stop puppet on the old postgres master before next step. Just to be sure it does not use the new postgres master's IP for keepalived.
7. Change the global hiera-key profile::postgres::ipv4 to point to the IP of the new postgres master, and run puppet on the puppetdb servers.
8. Install Ubuntu 18.04 with the role role::postgres::slave on the rest of the postgres-machines. While this installation is going, move the hiera-keys in step 2 from the node-file to a global file (key c already exist, so make sure that there is no duplicate).
9. Use postgres-joinMaster.sh on the slaves to have them replicating their master.

Upgrading puppetdb

At this point, with postgres13 running and puppet7 agents, upgrading the puppetdb-machines are simply a dist-upgrade and restart of the service (or host to get new kernels as well).