...

1. Set apache::service_ensure: 'stopped' in hiera for the node that you are upgrading
2. Run puppet with the 2023.1 modules/tags, run apt-get dist-upgrade, and run puppet again
3. Run keystone-manage doctor and ensure nothing is wrong
4. Run keystone-manage db_sync --expand
   1. Returns nothing
5. At this point, you may restart apache2 on this node
   1. Remove the  apache::service_ensure: 'stopped' previously set in hiera.
6. Upgrade keystone on the other nodes, one at a time
   1. Basically run step 1, 2 and 5 on the other nodes
7. When all nodes are upgraded, perform the final DB sync
   1. keystone-manage db_sync --contract

Glance

To upgrade glance without any downtime you would need to follow the following procedure:

1. Select which glance-server to upgrade first.
   1. In the node-specific hiera for this host you should set: 
      1. apache::service_ensure: 'stopped'
2. Run puppet with the 2023.1 modules/tags, run apt-get dist-upgrade, and run puppet again
3. Remove the apache::service_ensure: 'stopped' from the node-specific hiera, and run puppet again. This would re-start the glance api-server on this host.
   1. Test that this api-server works.
4. Upgrade the rest of the glance hosts (ie; step 2 for each of the remaining glance hosts)