You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The reason why this happens is because the proxy tries to parse the RPC match but it fails because it is missing. Lets walk through the stack in the policy controller:
The reason why the method in this case is None can be seen in the following piece of code.This is the code that takes the deserialized K8s API representation and turns it into our internal type.
fn deserialize_method_match<'de, D: serde::Deserializer<'de>>(
deserializer: D,
) -> Result<Option<GrpcMethodMatch>, D::Error> {
<Option<GrpcMethodMatch> as serde::Deserialize>::deserialize(deserializer).map(|value| {
match value.as_ref() {
Some(rule) if rule.is_empty() => None,
_ => value,
}
})
}
...
impl GrpcMethodMatch {
fn is_empty(&self) -> bool {
let (method, service) = match self {
Self::Exact { method, service } => (method, service),
Self::RegularExpression { method, service } => (method, service),
};
method.as_deref().map(str::is_empty).unwrap_or(true)
&& service.as_deref().map(str::is_empty).unwrap_or(true)
}
}
It is clear at this point that if method is an empty string or service is an empty string then this is an empty GRPC match and None is returned, even if the rype of the match has been specified.
For the record, that default was removed from the GRPCRoute CRD in Gateway API v0.8.0, so if you update your Gateway API CRDs to v0.8.0 or higher, everything should Just Work. (I'm pretty sure I tested this with Gateway API 1.1.1, but I'll doublecheck that.)
I deleted my previous comment about how Gateway API v1.1.1 makes this Just Work™.
In the very early days of Gateway API, a Route with a Service parentRef but no backendRefs would implicitly use the parentRef as a backendRef, so the GRPCRoute that @zaharidichev cites should indeed have worked.
In both current Gateway API and in Linkerd's implementation, you need to supply backendRefs: the GRPCRoute with no backendRefs at all is accepted, and if you've installed Gateway API v1.1.1 it won't have a weird matches stanza added to it. It won't actually route to anything, though, so it probably shouldn't be accepted in the first place.
What is the issue?
Whenever we create a
GRPCRoute
that has no matches at all, the proxy sees that as invalid outbound policy.How can it be reproduced?
Reproduction:
Create a GRPC server:
Create a GRPCRoute for this server:
Make sure the status of the Route is accepted and backend refs are resolved.
Now, create a simple client (non GRPC), just to be able to fascilitate a policy resolution for this target.
Ssh into the client and issue a simple curl request to the service on port 8080.
If you tail the logs of the client proxy you will see that the policy that has been delivered is invalid:
You can also observe the following error:
The reason why this happens is because the proxy tries to parse the RPC match but it fails because it is missing. Lets walk through the stack in the policy controller:
It is missing because the policy controller contains the following logic for putting together the API response. This indicates that if our method match is
None
, then the RPC will beNone
as well (which is the source of the problem.)https://github.com/linkerd/linkerd2/blob/main/policy-controller/grpc/src/routes/grpc.rs#L23
The reason why the method in this case is
None
can be seen in the following piece of code.This is the code that takes the deserialized K8s API representation and turns it into our internal type.linkerd2/policy-controller/k8s/index/src/routes/grpc.rs
Line 14 in 50e1a09
The K8s representation in turn is deserialized using the following logic in the gateway API repo:
It is clear at this point that if
method
is an empty string orservice
is an empty string then this is an empty GRPC match andNone
is returned, even if the rype of the match has been specified.Now, if we look closely, we will see that the version of the
GRPCRoute
CRDs that we use, has a default for route matches that create a route match of typeMethod
with nothing else specified: https://github.com/linkerd/linkerd2/blob/main/charts/linkerd-crds/templates/gateway.networking.k8s.io_grpcroutes.yaml#L254This makes it so, whenever we create a
GRPCRoute
with no explicit matches, we end up interpreting that as an Invalid one.Logs, error output, etc
already provided
output of
linkerd check -o short
n/a
Environment
Possible solution
No response
Additional context
No response
Would you like to work on fixing this bug?
None
The text was updated successfully, but these errors were encountered: