Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Kind OVNK IPV6 deployment #1751

Open
wants to merge 2 commits into
base: devel
Choose a base branch
from
Open

Add support for Kind OVNK IPV6 deployment #1751

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8702d60
Add support for Kind OVNK IPV6 deployment
yboaron Nov 18, 2024
017a6a1
Workaround until ipv6 bugfix merge
yboaron Jan 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion package/Dockerfile.shipyard-dapper-base
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ FROM fedora:41
# good balance between compression time and resulting image size.
ARG UPX_LEVEL=-5
ENV DAPPER_HOST_ARCH=amd64 SHIPYARD_DIR=/opt/shipyard SHELL=/bin/bash \
DAPPER_RUN_ARGS="--net=kind"
DAPPER_RUN_ARGS="--net=kind --cap-add=NET_ADMIN --privileged=true"
Copy link
Contributor Author

@yboaron yboaron Jan 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only need the extra networking permissions to container for OVN/IPV6 deployment.

@mkolesnik @skitt , is it possible to set DAPPER_RUN_ARGS content based on CNI, IP family env vars ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DAPPER_RUN_ARGS is a bit of a cheat — it’s not used as an environment variable as such inside the container, so nothing running in the container can set it meaningfully. The .dapper scripts extracts the value of the variable from the container image before running it.

However since we decide what .dapper does, anything can be done. Could you explain what the exacts criteria (as in, which values in which files) are and which containers need to run with privileges?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For IPV6 OVN deployment (make clusters using=ipv6-stack,ovn) OVN-K script [1] sets net.ipv6.conf.all.forwarding and sudo sysctl -w net.ipv6.conf.all.disable_ipv6 .
so, container should run with net privileges.

[1]
https://github.com/ovn-kubernetes/ovn-kubernetes/blob/master/contrib/kind.sh#L642

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It only does so if the values don’t match what it expects. Unfortunately GH doesn’t enable IPv6 forwarding by default, so changing that would be necessary.

However it would be possible to do this as an extra step before running the jobs, instead of making the containers themselves privileged…

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See https://github.com/submariner-io/shipyard/actions/runs/12792280698/job/35662349880?pr=1813

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It only does so if the values don’t match what it expects

Yep, without net privileges we can't even read kernel's ipv6 config (sysctl net.ipv6.conf.all.forwarding)

However it would be possible to do this as an extra step before running the jobs, instead of making the containers themselves privileged…

Maybe I'm missing something here, this idea will be useful only on GH, will it be possible to deploy Kind+IPV6,OVN locally without making the containers privileged?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, without net privileges we can't even read kernel's ipv6 config (sysctl net.ipv6.conf.all.forwarding)

That’s surprising, it works for me and should work fine as long as /proc/sys is mounted in the container (sysctl literally only reads from that).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For local deployments, we can check the settings early and give an appropriate error, same as for other sysctl settings. In most cases I expect developer machines will be set up with IPv6 enabled and forwarded anyway.

ENV HOST_ARCH=${DAPPER_HOST_ARCH} ARCH=${DAPPER_HOST_ARCH} PATH=/go/bin:/root/.local/bin:/usr/local/go/bin:$PATH \
GOLANG_ARCH_amd64=amd64 GOLANG_ARCH_arm=armv6l GOLANG_ARCH=GOLANG_ARCH_${DAPPER_HOST_ARCH} \
GOPATH=/go GO111MODULE=on GOPROXY=https://proxy.golang.org \
Expand Down
20 changes: 18 additions & 2 deletions scripts/shared/lib/clusters_kind
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,12 +196,24 @@ EOF

function deploy_kind_ovn(){
export K8S_VERSION
export NET_CIDR_IPV4="${cluster_CIDRs[${cluster}]}"
export SVC_CIDR_IPV4="${service_CIDRs[${cluster}]}"
export SVC_CIDR_IPV4
export NET_CIDR_IPV4
export SVC_CIDR_IPV6
export NET_CIDR_IPV6
export KIND_CLUSTER_NAME="${cluster}"

local ovn_flags=()
[[ "$OVN_IC" != true ]] || ovn_flags=( -ic -npz 1 -wk 3 )

if [[ "$IPV6_STACK" ]]; then
ovn_flags+=( -n4 -i6 -sw)
SVC_CIDR_IPV6="${service_IPv6_CIDRs[${cluster}]}"
NET_CIDR_IPV6="${cluster_IPv6_CIDRs[${cluster}]}"
else
NET_CIDR_IPV4="${cluster_CIDRs[${cluster}]}"
SVC_CIDR_IPV4="${service_CIDRs[${cluster}]}"
fi

delete_cluster_on_fail ./ovn-kubernetes/contrib/kind.sh -ov "$OVN_IMAGE" -cn "${KIND_CLUSTER_NAME}" -ric "${ovn_flags[@]}" -lr -dd "${KIND_CLUSTER_NAME}.local" --disable-ovnkube-identity

[[ "$AIR_GAPPED" = true ]] && air_gap_iptables
Expand Down Expand Up @@ -252,6 +264,10 @@ function prepare_ovn() {

# When updating commit, Update the OVN_SRC_IMAGE to the corressponding commit
git checkout 24b0ae73a996e409bfefad7b90cb42224e34be54
git fetch origin pull/4955/head:kind_ipv6_fix
git config user.email "[email protected]"
git config user.name "fufu"
git cherry-pick kind_ipv6_fix
local OVN_SRC_IMAGE="ghcr.io/ovn-org/ovn-kubernetes/ovn-kube-u:master@sha256:ba102783d520f0474e5c7dd5f2a0a1dce0ec2bda6cd42ac547621892e57c25e2"

docker pull "${OVN_SRC_IMAGE}"
Expand Down
Loading