Versions Compared

Old Version 13

changes.mady.by.user Lars Erik Pedersen

Saved on

compared with

New Version Current

changes.mady.by.user Lars Erik Pedersen

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Her er kommandoer brukt for å opprette public cluster templates

Train

...

Set a template as private/hidden

When a new template is rolled out, set the old version to hidden:

Code Block
openstack coe cluster template update <id> replace hidden=true

2023.1 (Antelope)

In Antelope there is hack/workaround the users has to apply after the cluster is CREATE_COMPLETE, if they want to use the Cinder CSI. NOTE: Only needed for single master clusters, for some reason.....

Code Block
kubectl edit deployment csi-cinder-controllerplugin
# Find the key "dnsPolicy: ClusterFirst" and change it to "dnsPolicy: Default"
# Save and exit. The pod will rebuild


kubernetes-template for v1.24.17

Code Block
openstack coe cluster template create kubernetes-v1.24.17 \
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --master-flavor gx1.2c4r \
  --flavor gx1.2c4r \
  --image fedora-coreos35-stable \
  --labels keystone_auth_enabled=False,kube_tag=v1.24.17-rancher1,container_runtime=containerd,containerd_version=1.6.6,containerd_tarball_sha256=a64568c8ce792dd73859ce5f336d5485fcbceab15dc3e06d5d1bc1c3353fa20f,flannel_network_cidr=172.29.0.0/16,cloud_provider_tag=v1.24.6,cinder_csi_plugin_tag=v1.24.6 \
  --public

kubernetes-template-ha for v1.24.17

Code Block
openstack coe cluster template create kubernetes-v1.24.17-ha \
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --master-flavor gx3.4c8r \
  --flavor gx1.2c4r \
  --image fedora-coreos35-stable \
  --labels keystone_auth_enabled=False,kube_tag=v1.24.17-rancher1,container_runtime=containerd,containerd_version=1.6.6,containerd_tarball_sha256=a64568c8ce792dd73859ce5f336d5485fcbceab15dc3e06d5d1bc1c3353fa20f,flannel_network_cidr=172.29.0.0/16,cloud_provider_tag=v1.24.6,cinder_csi_plugin_tag=v1.24.6 \
  --master-lb-enabled \
  --public


Zed

No kubernetes changes needed (nor supported) for Zed. But we will change back to using flannel from now on. We are also bumping clustered masters to a default flavor with more CPU and RAM.

kubernetes-template for v1.23.16

Code Block
openstack coe cluster template create kubernetes-v1.23.16 \
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --master-flavor gx1.2c4r \
  --flavor gx1.2c4r \
  --image fedora-coreos35-stable \
  --labels keystone_auth_enabled=False,kube_tag=v1.23.16-rancher1,container_runtime=containerd,containerd_version=1.6.6,containerd_tarball_sha256=a64568c8ce792dd73859ce5f336d5485fcbceab15dc3e06d5d1bc1c3353fa20f,flannel_backend=vxlan,flannel_network_cidr=172.29.0.0/16,csi_snapshotter_tag=v4.0.0 \
  --public

kubernetes-template-ha for v1.23.16

Code Block
openstack coe cluster template create kubernetes-v1.23.16-ha \.11 \ 
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --master-flavor m1gx3.small4c8r \
  --flavor m1gx1.small2c4r \
  --image fedora-coreoscoreos35-stable \
  --labels keystone_auth_enabled=False,kube_tag=v1.23.16-rancher1,container_runtime=containerd,containerd_version=1.6.6,containerd_tarball_sha256=a64568c8ce792dd73859ce5f336d5485fcbceab15dc3e06d5d1bc1c3353fa20f,flannel_backend=vxlan,flannel_network_cidr=172.29.0.0/16,csi_snapshotter_tag=v4.0.0 \
  --master-lb-enabled \
  --public


Yoga

kubernetes-template for v1.23.16

Code Block
openstack coe cluster template create kubernetes-v1.23.16 \
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --network-driver calico \
  --master-flavor gx1.2c4r \
  --flavor gx1.2c4r \
  --image fedora-coreos35-stable \
  --labels keystone_auth_enabled=False,kube_tag=v1.23.16-rancher1,container_runtime=containerd,containerd_version=1.6.6,containerd_tarball_sha256=a64568c8ce792dd73859ce5f336d5485fcbceab15dc3e06d5d1bc1c3353fa20f,csi_snapshotter_tag=v4.0.0 \
  --public

kubernetes-template-ha for v1.23.16

Code Block
openstack coe cluster template create kubernetes-v1.23.16-ha \
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --network-driver calico \
  --master-flavor gx1.2c4r \
  --flavor gx1.2c4r \
  --image fedora-coreos35-stable \
  --labels keystone_auth_enabled=False,kube_tag=v1.23.16-rancher1,container_runtime=containerd,containerd_version=1.6.6,containerd_tarball_sha256=a64568c8ce792dd73859ce5f336d5485fcbceab15dc3e06d5d1bc1c3353fa20f,csi_snapshotter_tag=v4.0.0 \
  --master-lb-enabled \
  --public

Xena

kubernetes-template for v1.21.14

Code Block
openstack coe cluster template create kubernetes-v1.21.14 \
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --network-driver calico \
  --master-flavor gx1.2c4r \
  --flavor gx1.2c4r \
  --image fedora-coreos33-stable \
  --labels keystone_auth_enabled=False,kube_tag=v1.21.14-rancher1,hyperkube_prefix=docker.io/rancher/ \
  --public

kubernetes-template-ha for v1.21.14

Code Block
openstack coe cluster template create kubernetes-v1.21.14-ha \
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --network-driver calico \
  --master-flavor gx1.2c4r \
  --flavor gx1.2c4r \
  --image fedora-coreos33-stable \
  --labels keystone_auth_enabled=False,kube_tag=v1.21.14-rancher1,hyperkube_prefix=docker.io/rancher/ \
  --master-lb-enabled \
  --public


Wallaby

kubernetes-template for v1.21.12

NOTE: With new flavor scheme, tha flavor may vary between platforms

Code Block
openstack coe cluster template create kubernetes-v1.21.12 \
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --network-driver calico \
  --master-flavor gx1.2c4r \
  --flavor gx1.2c4r \
  --image fedora-coreos33-stable \
  --labels keystone_auth_enabled=False,kube_tag=v1.21.12-rancher1,hyperkube_prefix=docker.io/rancher/ \
  --public

kubernetes-template-ha for v1.21.12

Code Block
openstack coe cluster template create kubernetes-v1.21.12-ha \
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --network-driver calico \
  --master-flavor gx1.2c4r \
  --flavor gx1.2c4r \
  --image fedora-coreos33-stable \
  --labels keystone_auth_enabled=False,kube_tag=v1.21.12-rancher1,hyperkube_prefix=docker.io/rancher/ \
  --master-lb-enabled \
  --public


Victoria

kubernetes-template for v1.21.x

Code Block
openstack coe cluster template create kubernetes-v1.21.7 \
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --network-driver calico \
  --master-flavor m1.small \
  --flavor m1.small \
  --image fedora-coreos33-stable \
  --labels keystone_auth_enabled=False,kube_tag=v1.21.7-rancher1,hyperkube_prefix=docker.io/rancher/ \
  --public

kubernetes-template-ha for v1.21.x

Code Block
openstack coe cluster template create kubernetes-v1.21.7-ha \
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --network-driver calico \
  --master-flavor m1.small \
  --flavor m1.small \
  --image fedora-coreos33-stable \
  --labels keystone_auth_enabled=False,kube_tag=v1.21.7-rancher1,hyperkube_prefix=docker.io/rancher/ \
  --master-lb-enabled \
  --public


kubernetes-template

Code Block
openstack coe cluster template create kubernetes-v1.18.20 \
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --network-driver calico \
  --master-flavor m1.small \
  --flavor m1.small \
  --image fedora-coreos33-stable \
  --labels keystone_auth_enabled=False,kube_tag=v1.18.20 \
  --public

kubernetes-template-ha

Code Block
openstack coe cluster template create kubernetes-v1.18.20-ha \
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --network-driver calico \
  --master-flavor m1.small \
  --flavor m1.small \
  --image fedora-coreos33-stable \
  --labels keystone_auth_enabled=False,kube_tag=v1.18.20 \
  --master-lb-enabled \
  --public


Docker swarm doesn't work in Victoria....

Ussuri

kubernetes-template

Code Block
openstack coe cluster template create kubernetes-v1.18.10 \
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --network-driver calico \
  --master-flavor m1.small \
  --flavor m1.small \
  --image fedora-coreos-stable \
  --labels keystone_auth_enabled=False,heat_container_agent_tag=ussuri-stable-1,kube_tag=v1.18.10 \
  --public

kubernetes-template-ha

Code Block
openstack coe cluster template create kubernetes-v1.18.10-ha \
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --network-driver calico \
  --master-flavor m1.small \
  --flavor m1.small \
  --image fedora-coreos-stable \
  --labels keystone_auth_enabled=False,heat_container_agent_tag=ussuri-stable-1,kube_tag=v1.18.10 \
  --master-lb-enabled \
  --public


No changes in docker-swam-mode templates for ussuri.

Train

Note: There is a bug in that occurs in the mix of Fedora CoreOS 32, sytemd 242 and flannel, that is breaking inter-pod communication with flannel. Therefore, we use calico. Link

kubernetes-template

Code Block
openstack coe cluster template create kubernetes-v1.17.11 \ 
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --network-driver calico \
  --master-flavor m1.small \
  --flavor m1.small \
  --image fedora-coreos-stable \
  --labels keystone_auth_enabled=False,kube_tag=v1.17.11 \
  --public

kubernetes-template-ha

Code Block
openstack coe cluster template create kubernetes-v1.17.11-ha \ 
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200,129.241.0.201 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --docker-storage-driver overlay2 \
  --network-driver calico \
  --master-flavor m1.small \
  --flavor m1.small \
  --image fedora-coreos-stable \
  --labels keystone_auth_enabled=False,kube_tag=v1.17.11 \
  --master-lb-enabled \
  --public


No changes in docker-swam-mode templates for train.

Stein

kubernetes-template

Code Block
openstack coe cluster template create kubernetes-v1.15.12 \ 
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --master-flavor m1.small \
  --flavor m1.small \
  --image fedora-atomic-latest \
  --labels heat_container_agent_tag=stein-stable,keystone_auth_enabled=False,flannel_network_cidr=172.29.0.0/16,kube_tag=v1.15.12 \
  --public

kubernetes-template-ha

Code Block
openstack coe cluster template create kubernetes-v1.15.12-ha \ 
  --coe kubernetes \
  --external-network ntnu-internal \
  --dns-nameserver 129.241.0.200 \
  --volume-driver cinder \
  --docker-volume-size 20 \
  --master-flavor m1.small \
  --flavor m1.small \
  --image fedora-atomic-latest \
  --labels heat_container_agent_tag=stein-stable,keystone_auth_enabled=False,flannel_network_cidr=172.29.0.0/16,kube_tag=v1.15.12 \
  --master-lb-enabled \
  --public

...