chore(deps): upgrade to hyper 1.x #3504
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cratelyn
force-pushed
the
kate/hyper-1.x-upgrade
branch
from
daf57fe to
dd6f80f
Compare
cratelyn
added a commit
that referenced
this pull request
Jan 8, 2025
this commit makes a small, subtle change to the `PeekTrailersBody<B>` http response body middleware. to help facilitate upgrading to http-body 1.x, we remove some tricky logic that involves `Body` interfaces that no longer exist after the 0.4 release. currently, a `PeekTrailersBody<B>` is not fully consistent about the conditions in which it will peek the trailers of a response body: the inner body is allowed to yield _either_ (a) **zero** DATA frames, in which case the body will be `.await`'ed and polled until the trailers are obtained, or (b) **one** DATA frame, so long as the inner body immediately yields a trailer. meanwhile, the documentation comment for the type claims: > An HTTP body that allows inspecting the body's trailers, if a > `TRAILERS` frame was the first frame after the initial headers frame. we won't have distinct `data()` and `trailers()` interfaces in the 1.0 release. we have a single [`BodyExt::frame()`](https://docs.rs/http-body-util/latest/http_body_util/trait.BodyExt.html#method.frame) method. consequently, porting this middleware as-is would be somewhat difficult. we might have to hold two frames, should we receive one frame, `now_or_never()` the second frame, and discover that we've been provided a second data frame rather than the trailers. this all runs slightly against the invariants of `Body`, see this comment originally added in 7f817b5: ``` // Peek to see if there's immediately a trailers frame, and grab // it if so. Otherwise, bail. // XXX(eliza): the documentation for the `http::Body` trait says // that `poll_trailers` should only be called after `poll_data` // returns `None`...but, in practice, I'm fairly sure that this just // means that it *will not return `Ready`* until there are no data // frames left, which is fine for us here, because we `now_or_never` // it. ``` this isn't quite true, as `Trailers` is just a wrapper calling `poll_trailers`: <https://docs.rs/http-body/0.4.6/src/http_body/next.rs.html#28-30> so, let's remove this. doing so will make the task of porting this middleware to http-body 1.0 in the short term, and additionally prevents any potential future misbehavior due to inner bodies not handling this eager trailer polling gracefully. see linkerd/linkerd2#8733. see #3504. Signed-off-by: katelyn martin <[email protected]>
cratelyn
added a commit
that referenced
this pull request
Jan 8, 2025
this commit makes a small, subtle change to the `PeekTrailersBody<B>` http response body middleware. to help facilitate upgrading to http-body 1.x, we remove some tricky logic that involves `Body` interfaces that no longer exist after the 0.4 release. currently, a `PeekTrailersBody<B>` is not fully consistent about the conditions in which it will peek the trailers of a response body: the inner body is allowed to yield _either_ (a) **zero** DATA frames, in which case the body will be `.await`'ed and polled until the trailers are obtained, or (b) **one** DATA frame, so long as the inner body immediately yields a trailer. meanwhile, the documentation comment for the type claims: > An HTTP body that allows inspecting the body's trailers, if a > `TRAILERS` frame was the first frame after the initial headers frame. we won't have distinct `data()` and `trailers()` interfaces in the 1.0 release. we have a single [`BodyExt::frame()`](https://docs.rs/http-body-util/latest/http_body_util/trait.BodyExt.html#method.frame) method. consequently, porting this middleware as-is would be somewhat difficult. we might have to hold two frames, should we receive one frame, `now_or_never()` the second frame, and discover that we've been provided a second data frame rather than the trailers. this all runs slightly against the invariants of `Body`, see this comment originally added in 7f817b5: ``` // Peek to see if there's immediately a trailers frame, and grab // it if so. Otherwise, bail. // XXX(eliza): the documentation for the `http::Body` trait says // that `poll_trailers` should only be called after `poll_data` // returns `None`...but, in practice, I'm fairly sure that this just // means that it *will not return `Ready`* until there are no data // frames left, which is fine for us here, because we `now_or_never` // it. ``` this isn't quite true, as `Trailers` is just a wrapper calling `poll_trailers`: <https://docs.rs/http-body/0.4.6/src/http_body/next.rs.html#28-30> so, let's remove this. doing so will make the task of porting this middleware to http-body 1.0 in the short term, and additionally prevents any potential future misbehavior due to inner bodies not handling this eager trailer polling gracefully. see linkerd/linkerd2#8733. see #3504. Signed-off-by: katelyn martin <[email protected]>
cratelyn
force-pushed
the
kate/hyper-1.x-upgrade
branch
5 times, most recently
from
fa62779 to
b41108e
Compare
cratelyn
force-pushed
the
kate/hyper-1.x-upgrade
branch
3 times, most recently
from
a5d21c8 to
5085d02
Compare
|
rebased once more ♻️ |
cratelyn
added a commit
that referenced
this pull request
Jan 22, 2025
this vendors a copy of the `Frame<T>` type. we can use this to preemptively model our peek body in terms of this type, and move to the "real" version of it when we're upgrading in #3504. Signed-off-by: katelyn martin <[email protected]>
cratelyn
added a commit
that referenced
this pull request
Jan 23, 2025
this commit introduces a `compat` submodule to `linkerd-http-retry`. this helps us frontrun the task of replacing all of the finicky control flow in `PeekTrailersBody<B>` using the antiquated `data()` and `trailers()` future combinators. instead, we can perform our peeking in terms of an approximation of `http_body_util::BodyExt::frame()`. to accomplish this, this commit vendors a copy of the `Frame<T>` type. we can use this to preemptively model our peek body in terms of this type, and move to the "real" version of it when we're upgrading in pr #3504. additionally, this commit includes a type called `ForwardCompatibleBody<B>`, and a variant of the `Frame<'a, T>` combinator. these are a bit boilerplate-y, admittedly, but the pleasant part of this is that we have, in effect, migrated the trickiest body middleware in advance of #3504. once we upgrade to http-body 1.0, all of these types can be removed. https://docs.rs/http-body-util/latest/http_body_util/trait.BodyExt.html#method.frame https://docs.rs/http-body-util/0.1.2/src/http_body_util/combinators/frame.rs.html#10 Signed-off-by: katelyn martin <[email protected]>
cratelyn
added a commit
that referenced
this pull request
Jan 23, 2025
this commit introduces a `compat` submodule to `linkerd-http-retry`. this helps us frontrun the task of replacing all of the finicky control flow in `PeekTrailersBody<B>` using the antiquated `data()` and `trailers()` future combinators. instead, we can perform our peeking in terms of an approximation of `http_body_util::BodyExt::frame()`. to accomplish this, this commit vendors a copy of the `Frame<T>` type. we can use this to preemptively model our peek body in terms of this type, and move to the "real" version of it when we're upgrading in pr #3504. additionally, this commit includes a type called `ForwardCompatibleBody<B>`, and a variant of the `Frame<'a, T>` combinator. these are a bit boilerplate-y, admittedly, but the pleasant part of this is that we have, in effect, migrated the trickiest body middleware in advance of #3504. once we upgrade to http-body 1.0, all of these types can be removed. https://docs.rs/http-body-util/latest/http_body_util/trait.BodyExt.html#method.frame https://docs.rs/http-body-util/0.1.2/src/http_body_util/combinators/frame.rs.html#10 Signed-off-by: katelyn martin <[email protected]>
cratelyn
added a commit
that referenced
this pull request
Jan 23, 2025
this commit reworks `PeekTrailersBody<B>`. the most important goal here is replacing the control flow of `read_body()`, which polls a body using `BodyExt` future combinators `data()` and `frame()` for up to two frames, with varying levels of persistence depending on outcomes. to quote #3556: > the intent of this type is to only yield the asynchronous task > responsible for reading the body once. depending on what the inner > body yields, and when, this can result in combinations of: no data > frames and no trailers, no data frames with trailers, one data frame > and no trailers, one data frame with trailers, or two data frames. > moreover, depending on which of these are yielded, the body will call > .await some scenarios, and only poll functions once in others. > > migrating this to the Frame<T> and poll_frame() style of the 1.0 Body > interface, away from the 0.4 interface that provides distinct > poll_data() and poll_trailers() methods, is fundamentally tricky. this means that `PeekTrailersBody<B>` is notably difficult to port across the http-body 0.4/1.0 upgrade boundary. this body middleware must navigate a number of edge conditions, and once it _has_ obtained a `Frame<T>`, make use of conversion methods to ascertain whether it is a data or trailers frame, due to the fact that its internal enum representation is not exposed publicly. one it has done all of that, it must do the same thing once more to examine the second frame. this commit uses the compatibility facilities and backported `Frame<T>` introduced in the previous commit, and rewrites this control flow using a form of the `BodyExt::frame()` combinator. this means that this middleware is forward-compatible with http-body 1.0, which will dramatically simplify the remaining migration work to be done in #3504. see linkerd/linkerd2#8733 for more information and other links related to this ongoing migration work. Signed-off-by: katelyn martin <[email protected]>
cratelyn
added a commit
that referenced
this pull request
Jan 24, 2025
this branch contains a sequence of commits that refactor `PeekTrailersBody<B>`. this branch is specifically focused on making this body middleware forward-compatible with the 1.0 interface(s) of `http_body::Body` and `http_body_util::BodyExt`. it does this in two main steps: (1) temporarily vendoring `http_body::Frame<T>` and providing a compatibility shim that provides a `frame()` method for a body, and (2) modeling `PeekTrailersBody<B>` and its peeking logic in terms of this `Frame<'a, T>` future. --- * feat(http/retry): add `Frame<T>` compatibility facilities this commit introduces a `compat` submodule to `linkerd-http-retry`. this helps us frontrun the task of replacing all of the finicky control flow in `PeekTrailersBody<B>` using the antiquated `data()` and `trailers()` future combinators. instead, we can perform our peeking in terms of an approximation of `http_body_util::BodyExt::frame()`. to accomplish this, this commit vendors a copy of the `Frame<T>` type. we can use this to preemptively model our peek body in terms of this type, and move to the "real" version of it when we're upgrading in pr #3504. additionally, this commit includes a type called `ForwardCompatibleBody<B>`, and a variant of the `Frame<'a, T>` combinator. these are a bit boilerplate-y, admittedly, but the pleasant part of this is that we have, in effect, migrated the trickiest body middleware in advance of #3504. once we upgrade to http-body 1.0, all of these types can be removed. https://docs.rs/http-body-util/latest/http_body_util/trait.BodyExt.html#method.frame https://docs.rs/http-body-util/0.1.2/src/http_body_util/combinators/frame.rs.html#10 Signed-off-by: katelyn martin <[email protected]> * refactor(http/retry): `PeekTrailersBody<B>` uses `BodyExt::frame()` this commit reworks `PeekTrailersBody<B>`. the most important goal here is replacing the control flow of `read_body()`, which polls a body using `BodyExt` future combinators `data()` and `frame()` for up to two frames, with varying levels of persistence depending on outcomes. to quote #3556: > the intent of this type is to only yield the asynchronous task > responsible for reading the body once. depending on what the inner > body yields, and when, this can result in combinations of: no data > frames and no trailers, no data frames with trailers, one data frame > and no trailers, one data frame with trailers, or two data frames. > moreover, depending on which of these are yielded, the body will call > .await some scenarios, and only poll functions once in others. > > migrating this to the Frame<T> and poll_frame() style of the 1.0 Body > interface, away from the 0.4 interface that provides distinct > poll_data() and poll_trailers() methods, is fundamentally tricky. this means that `PeekTrailersBody<B>` is notably difficult to port across the http-body 0.4/1.0 upgrade boundary. this body middleware must navigate a number of edge conditions, and once it _has_ obtained a `Frame<T>`, make use of conversion methods to ascertain whether it is a data or trailers frame, due to the fact that its internal enum representation is not exposed publicly. one it has done all of that, it must do the same thing once more to examine the second frame. this commit uses the compatibility facilities and backported `Frame<T>` introduced in the previous commit, and rewrites this control flow using a form of the `BodyExt::frame()` combinator. this means that this middleware is forward-compatible with http-body 1.0, which will dramatically simplify the remaining migration work to be done in #3504. see linkerd/linkerd2#8733 for more information and other links related to this ongoing migration work. Signed-off-by: katelyn martin <[email protected]> * refactor(http/retry): mock body enforces `poll_trailers()` contract this commit addresses a `TODO` note, and tightens the enforcement of a rule defined by the v0.4 signature of the `Body` trait. this commit changes the mock body type, used in tests, so that it will panic if the caller improperly polls for a trailers frame before the final data frame has been yielded. previously, a comment indicated that we were "fairly sure" this was okay. while that may have been acceptable in practice, the changes in the previous commit mean that we now properly respect these rules. thus, a panic can be placed here, to enforce that "[is] only be called once `poll_data()` returns `None`", per the documentation. Signed-off-by: katelyn martin <[email protected]> * refactor(http/retry): rename `PeekTrailersBody::Buffered` <#3559 (comment)> this is a nicer name than `Unknown` for this case. not to mention, we'll want that name shortly to address the possibility of unknown frame variants. Signed-off-by: katelyn martin <[email protected]> * refactor(http/retry): encapsulate `Inner<B>` enum variants this commit makes the inner enum variants private. #3559 (comment) Signed-off-by: katelyn martin <[email protected]> * refactor(http/retry): gracefully ignore unknown frames #3559 (comment) Signed-off-by: katelyn martin <[email protected]> --------- Signed-off-by: katelyn martin <[email protected]>
cratelyn
added a commit
that referenced
this pull request
Jan 24, 2025
this is a refactor commit, outlined from #3504. this commit is particularly interested in preparing the `PeekTrailersBody<B>` middleware for our upgrade to hyper/http-body 1.0. more specifically: this commit clears up the boundary concerning when it will or will not become inert and delegate to its inner body `B`. currently, a `PeekTrailersBody<B>` is not fully consistent about the conditions in which it will peek the trailers of a response body: the inner body is allowed to yield _either_ (a) **zero** DATA frames, in which case the body will be `.await`'ed and polled until the trailers are obtained, or (b) **one** DATA frame, so long as the inner body immediately yields a trailer. see linkerd/linkerd2#8733. Signed-off-by: katelyn martin <[email protected]>
note: this commit will not compile, code changes are intentionally elided from this commit. this commit upgrades hyper, http, tonic, prost, related dependencies, and their assorted cargo features. see <linkerd/linkerd2#8733>. Signed-off-by: katelyn martin <[email protected]>
Signed-off-by: katelyn martin <[email protected]>
Signed-off-by: katelyn martin <[email protected]>
this commit represents main as of linkerd/linkerd2-proxy-api#421. Signed-off-by: katelyn martin <[email protected]>
Signed-off-by: katelyn martin <[email protected]>
Signed-off-by: katelyn martin <[email protected]>
Signed-off-by: katelyn martin <[email protected]>
Signed-off-by: katelyn martin <[email protected]>
Signed-off-by: katelyn martin <[email protected]>
NOTE: there is a comment noting that the upgrade middleware does not expect to be cloneable. it is unfortunately, however, at odds with the new bounds expected of extensions. so, `Http11Upgrade` is now Clone'able, but a comment is left in place noting this weakened invariant. it's worth investigating how upgrades have changed since, in more detail, but for the current moment we are interested in being especially conservative about changing behavior, and focusing on api changes like `Body::poll_frame(..)`. Signed-off-by: katelyn martin <[email protected]>
a brief note; this commit happened to tickle an unfortunate sharp edge in `BoxBody` and `Full`'s respective constructors. type inference could not figure out how to construct the body, so we refrain from boxing the response body now. Signed-off-by: katelyn martin <[email protected]>
Signed-off-by: katelyn martin <[email protected]>
Signed-off-by: katelyn martin <[email protected]>
Signed-off-by: katelyn martin <[email protected]>
cratelyn
force-pushed
the
kate/hyper-1.x-upgrade
branch
from
5085d02 to
b58ab0b
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
see linkerd/linkerd2#8733.