Ceph Luminous to Nautilus

1. Set the noout flag
   1. ceph osd set noout
2. Upgrade ceph-mon's, one at a time. For each do:
   
   1. Set puppet-env to a branch/tag containing nautilus-ceph
   2. Do a puppet-run
   3. Update the ceph-packages on a ceph-mon
      1. apt-get install ceph-mon (or; apt-get dist-upgrade)
   4. Restart the ceph-mon
      
      1. systemctl restart ceph-mon.target
3. When all monitors are upgraded and restarted, verify that the command 'ceph mon dump | grep min_mon_release' returns 'min_mon_release 14 (nautilus)'
4. On each monitor-host: restart ceph-mgr
   1. systemctl restart ceph-mgr.target
5. Upgrade ceph-osd hosts one at a time. For each do:
   1. Set puppet-env to a branch/tag containing nautilus-ceph
   2. Do a puppet-run
   3. Update the ceph-packages
      1. apt-get install ceph-osd (or; apt-get dist-upgrade)
   4. restart the ceph osd daemons
      1. systemctl restart ceph.target
   5. Verify that the osd-daemons still restart after a reboot (Point 7 in the official upgrade-guide (see ref. 1))
6. Upgrade radosgw's
   1. Set puppet-env to a branch/tag containing nautilus-ceph
   2. Do a puppet-run
   3. Upgrade the ceph-packages
      1. apt-get install radosgw (or apt-get dist-upgrade)
   4. Restart the ceph daemons (systemctl restart ceph.target)
7. Enable all nautilus functionality by disallowing any pre-nautilus OSD's (Make sure to upgrade all OSD's first, and verify that the correct versions are running using the command 'ceph versions')
   

   1. ceph osd require-osd-release nautilus

8. Clear the noout flag
   
   1. ceph osd unset noout
9. Enable straw2
   1. ceph osd getcrushmap -o backup-crushmap
   2. ceph osd crush set-all-straw-buckets-to-straw2
   3. (To revert if problems: ceph osd setcrushmap -i backup-crushmap)
10. Enable the new v2 network protocol:

    1. ceph mon enable-msgr2

References:

1. https://docs.ceph.com/docs/master/releases/nautilus/#nautilus-old-upgrade