📖 clarify semantic meaning of the Version fields

[rccrdpccl]

What this PR does / why we need it:
The aim of this PR is to clarify the semantic meaning of the Version fields. At the moment, the documentation heavily implies that version fields should contain a Kubernetes version.
However this is problematic when working with Kubernetes distributions, especially the ones with a versioning schema totally different from Kubernetes (i.e. OpenShift).

UX problem

At the moment, to install a k8s distribution, the user needs to work out what k8s version corresponds to the desired k8s distribution version. However, this is not even always possible: for example, in the OpenShift case, a minor release is based on a given Z patch of k8s, and other Z releases for the same Y version might be based on the same k8s Z patch. As a practical example, OpenShift 4.17.0 is based on 1.30.4 and OpenShift 4.17.1 is also based on 1.30.4, making a bidirectional conversion actually impossible.

Kubernetes version references

Unfortunately this is not straightforward, as there are multiple references to explicit Kubernetes versions.

• machine preflight check explicitly referencing a kubernetes version (since when the skew version policy changes). In this case the referenced version is 1.28.0, which should fall out of support in less than 12 months time
• setting minimum version. In this case, we might be able to drop this check as Cluster API already doesn't support versions below 1.22 (in this case we're toggling between 1.20 and 1.22)

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign fabriziopandini for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

This PR is currently missing an area label, which is used to identify the modified component when generating release notes.

Area labels can be added by org members by writing /area ${COMPONENT} in a comment

Please see the labels list for possible areas.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[k8s-ci-robot]

Welcome @rccrdpccl!

It looks like this is your first PR to kubernetes-sigs/cluster-api 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/cluster-api has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @rccrdpccl. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[fabriziopandini]

I don't mind adding distribution after kubernetes version in go doc, many companies use custom versioning schema, but as a matter of fact the codebase is full of assumptions that this field has a direct correlation with the Kubernetes version being installed.

Top of mind we have similar assumptions in KCP/CABPK, most probably also in the topology controller and the E2E test framework, and I would not be surprised if we have assumption on version in other places as well.

Also, the K8s version being installed is a key information we need to have for implementing support of any new K8s version whenever it introduces a change that requires special handling (and this happens more often that you might expect); also the entire test matrix for our supported versions is based on this information.

This make me wondering if we are going down on a slippery path if we start decoupling the kubernetes distribution version from the actual kubernetes version being installed.

@vincepri @enxebre @sbueringer opinions? should we move this discussion to an issue first?

[fabriziopandini]

Also @chrischdi

[enxebre]

Generally speaking, moving towards having better supportability for distros to handle their versioning makes sense to me. That should hopefully come without any loss or impact for KCP/CABPK/... and for the special handling scenarios, while still enabling other building blocks to replace KCP/CABPK in a meaningful manner. A less convoluted path to satisfy both scenarios might be having a new field for the distro version? so then we have semantics for both and is up to implementers how to leverage them together. I agree with Fabrizio that this deserves thorough discussion.

[sbueringer]

My take on it. Yes, we have a lot of assumptions all over our code base that the version is the actual Kubernetes version (+/- ~ some suffix according to semver is accepted/ignored).

But not only that, we also have a huge ecosystem using Cluster API that up until today can assume that the version is the Kubernetes version. By changing that fact, we are not only breaking our own code that relies on that assumption but also everyone building on top of Cluster API that uses that version. We also can't just simply change the semantic of the field by updating the godoc and then keep all of our usages that assume that the version is the Kubernetes version the same.

About some specific cases:

• Your example from above: "setting minimum version. In this case, we might be able to drop this check as Cluster API already doesn't support versions below 1.22 (in this case we're toggling between 1.20 and 1.22)"
  • => This has nothing to do with the version API field. CAPI sends a request to the APIserver to get the server version and then compares it against a hard-coded value. Does this mean that in OpenShift also the APIserver is not aware of its "Kubernetes version"?
• In addition to various usages in KCP / CABPK, we also at least have the MachinePool Machine implementation that assume Node.status.nodeInfo.kubeletVersion (i.e. the Kubernetes version) is the same as Machine.spec.version

These cases show that it was necessary in the past to rely on the version being the Kubernetes version, while some of those cases are becoming eventually irrelevant because the respective Kubernetes versions go out of support we have other cases where this might never happen (e.g. KCP / CABPK being able to figure out the correct kubeadm config apiVersion based on Machine.spec.version).

In general, I think we have to be able going forward to implement Kubernetes version specific behavior. By changing the semantic of this field to "Kubernetes distribution version" we won't be able to do this anymore, which in the worst case means we won't be able to support a new Kubernetes version.

Furthermore, if we also couldn't rely anymore that the apiserver itself knows it's correct Kubernetes version we also won't be able to leverage new apiserver features in our controller code until all Kubernetes versions that don't support it are out of support (and we couldn't even implement checks if CAPI is running against a supported kube-apiserver version).

That being said, definitely open to discussions about for example adding an additional field. I think this definitely requires an issue and it probably would be good to write a small proposal to get some better idea what we want to achieve.