Skip to content

Support Spring Framework 7 (Spring Boot 4) and Spring Cloud Gateway 5.x - #826

Merged
wu-sheng merged 2 commits into
mainfrom
support-spring-framework-7
Aug 29, 2026
Merged

Support Spring Framework 7 (Spring Boot 4) and Spring Cloud Gateway 5.x#826
wu-sheng merged 2 commits into
mainfrom
support-spring-framework-7

Conversation

@wu-sheng

Copy link
Copy Markdown
Member

Fix apache/skywalking#14047NoSuchMethodError on Spring Framework 7, and extend the Spring plugin family to Spring Boot 4 / Spring Cloud Gateway 5.x

  • Add a test case to verify that the fix works.
  • Explain briefly why the bug exists and how to fix it.
  • Update the documentation to include this new feature.
  • Update the CHANGES log.

Closes apache/skywalking#14047.

Why the bug exists

Spring Framework 7.0 dropped the MultiValueMap contract from HttpHeaders (gh-33913), which removed List get(Object):

spring-web 6.2.19  public class HttpHeaders implements MultiValueMap<String,String>, Serializable
                   public java.util.List<java.lang.String> get(java.lang.Object);
spring-web 7.0.0+  public class HttpHeaders implements java.io.Serializable
                   public java.util.List<java.lang.String> get(java.lang.String);   // only

spring-webflux-6.x-plugin was compiled against the removed overload, emitting
invokevirtual org/springframework/http/HttpHeaders.get:(Ljava/lang/Object;)Ljava/util/List;. InstMethodsInter swallows the throwable and calls through, so the application keeps serving traffic while the entry span — created four lines later — never exists. The result is exactly what the reporter saw: one ERROR per request and no traces at all.

It is not limited to users who enabled the WebFlux plugin. spring-cloud-gateway-4.x-plugin shades a byte-identical copy of that interceptor, and its witness class LocalResponseCacheProperties still resolves on spring-cloud-gateway-server-webflux 5.0.3, so a gateway user who copies only the documented gateway plugin hits the same failure.

The boundary is exact: all of spring-web 6.2.x is fine, every 7.x breaks. Spring Boot 4.0.3 pins Spring Framework 7.0.5.

How it is fixed

A bytecode sweep of every built plugin jar — resolving all org/springframework constant-pool refs against spring-web/webflux 5.3.39, 6.2.0 and 7.0.5 — found only two Spring 7 removals reaching the agent, across three call sites.

Site Change
spring-webflux-6.x / -5.x DispatcherHandlerHandleMethodInterceptor HttpHeaders#getFirst(String), declared on HttpHeaders itself in Spring 5, 6 and 7 with an identical descriptor
mvc-annotation-commons RequestUtil getValuesAsList(String) — this site consumes every value, so getFirst would be wrong
mvc-annotation-commons AbstractMethodInterceptor drop the premature reactive status read

Two traps worth recording: HttpHeaders#get(String) does not fix it (compiled against Spring 6, javac still emits the (Object) descriptor), and getOrEmpty is not cross-version safe ((Object) on 6.2.0, (String) on 7.0.5).

For getRawStatusCode(): it was removed in Spring 7, but it also ran when the controller returned its Mono, before the response was written, so it normally read null. The version-specific InvokeInterceptor already sets http.status_code in Mono#doFinally once the status is known, so the shared module simply stops reading it — no version-specific types leak into mvc-annotation-commons.

Version witnesses

The 6.x plugin's only guard was WitnessMethod(Mono, contextWrite), which bounds Reactor, not Spring — present from 3.4.1 through 3.8.x. That is why it attached to Spring 7 and then failed per request instead of cleanly not attaching. Added:

Plugin Witness class 5.3.39 6.0.0 6.2.0 7.0.5
spring-webflux-5.x web.reactive.resource.GzipResourceResolver
spring-webflux-6.x web.reactive.DispatchExceptionHandler

This also closes a pre-existing hazard: on Reactor 3.4 both Mono#subscriberContext and Mono#contextWrite exist, so on Spring Boot 2.4–2.7 the 5.x and 6.x plugins would both match DispatcherHandler#handle.

⚠️ Breaking: the WebFlux application toolkit is split

Reactor removed Signal#getContext() in 3.5.0, and ContextView does not exist before 3.4.0 — no single artifact can span both. WebFluxSkyWalkingOperators#continueTracing has therefore been broken on all of Spring Boot 3 and 4 since Nov 2022.

Artifact Package Reactor Spring Boot
apm-toolkit-webflux-5.x (renamed) ...toolkit.webflux.v5 3.1 → 3.4 2.x
apm-toolkit-webflux-6.x (new) ...toolkit.webflux.v6 3.5 → 3.8 3.x and 4.x

The 5.x module keeps its code and behaviour byte-for-byte; only the coordinate and package change. Distinct packages are deliberate: the two artifacts have identical public signatures, so a shared FQCN would let a wrong dependency compile and fail only at runtime. This way it is an unresolvable import. The activation still matches the legacy un-versioned names, so applications that never upgrade their toolkit keep working against a newer agent.

The 6.x module compiles against Reactor 3.5 on purpose — the root cause of the three-year silent rot was that this module compiled against the oldest Reactor (3.2, via spring-webflux 5.1), so javac never flagged the removal. Reactor 3.5–3.8 are still Java 8 bytecode, so maven.compiler.release=8 is preserved.

Nothing loses support. Every Spring 3/4/5 plugin, Boot 1.x/2.x plugin and Reactor 3.1–3.4 user keeps working exactly as before.

Tests

Two new scenarios, both driving a full round-trip so the cross-process ref is asserted — the only thing that proves the sw8 header is read back correctly:

  • webflux-6.x-7.x-scenarioGET /case/webflux (Entry, componentId 67) → WebClient (Exit, 99) → /case/receive (Entry, 67, refType: CrossProcess) → WebFluxOperators/onNext (Local, through the new v6 toolkit package). Spring Boot 3.0 → 4.1, one version per minor.
  • gateway-5.x-scenario — Spring Cloud Gateway 5.0.3 on Spring Boot 4, gateway → backend, asserting the componentId 67 entry span, SpringCloudGateway/RoutingFilter / send locals, sendRequest exit, and the downstream CrossProcess ref.

Verified locally against the mock collector and validator on Spring Boot 3.5.16 and 4.0.8 (segment items assert successful, zero intercept failure), and Spring Cloud Gateway 5.0.3 on Boot 4.0.8.

The scenario was also negative-tested: reverting the one-line fix reproduces NoSuchMethodError: 'java.util.List org.springframework.http.HttpHeaders.get(java.lang.Object)', drops CrossProcess refs to 0, and fails the validator — so it genuinely guards this regression.

Documentation

Supported-list.md (WebFlux / WebClient / Spring MVC / RestTemplate → 7.x, Spring Cloud Gateway → 5.0.x), Optional-plugins.md (spring-webflux-6.x was never named there), agent-optional-plugins/spring-gateway.md, and Application-toolkit-webflux.md rewritten with both toolkit usages plus a migration note.

Supported-list.md line 10 said Spring Boot Web 4.x, added in 2017 (e327790c36) when it meant the spring-web module 4.x. In a Spring Boot 4 PR that reads as a support claim, so it is corrected to Spring Web 4.x.

Spring Framework 7.0 dropped the MultiValueMap contract from HttpHeaders
(spring-projects/spring-framework gh-33913), removing List get(Object). The
webflux-6.x plugin was compiled against that overload, so every request through
DispatcherHandler#handle threw NoSuchMethodError before the entry span was
created: the service kept serving traffic but produced no traces at all.
spring-cloud-gateway-4.x shades a byte-identical copy of that interceptor and
its witness class still resolves on Spring Cloud Gateway 5.x, so gateway users
hit it through the documented plugin as well.

A bytecode sweep of every built plugin jar, resolving all org/springframework
constant-pool refs against spring-web/webflux 5.3.39, 6.2.0 and 7.0.5, found
only two Spring 7 removals reaching the agent, across three call sites.

- spring-webflux-6.x / -5.x: read the header via HttpHeaders#getFirst(String),
  which is declared on HttpHeaders itself in Spring 5, 6 and 7 with the same
  descriptor. Note HttpHeaders#get(String) does NOT work: compiled against
  Spring 6 javac still emits the (Object) descriptor. getOrEmpty is not
  cross-version safe either, its descriptor changes between 6 and 7.
- mvc-annotation-commons RequestUtil: use getValuesAsList(String). getFirst is
  wrong here because this site consumes every value. Note it additionally
  splits comma-delimited values.
- mvc-annotation-commons AbstractMethodInterceptor: drop the premature reactive
  status read. ServerHttpResponse#getRawStatusCode() was removed in Spring 7,
  and it ran before the response was written so it normally read null anyway.
  The version-specific InvokeInterceptor already sets http.status_code in
  Mono#doFinally, once the status is actually known.

Add Spring generation witnesses to both webflux plugins. The only guard was
WitnessMethod(Mono, contextWrite), which bounds Reactor and not Spring, which
is why the plugin attached to Spring 7 and then failed on every request rather
than cleanly not attaching. On Reactor 3.4 both Mono#subscriberContext and
Mono#contextWrite exist, so the 5.x and 6.x plugins would both match
DispatcherHandler#handle on Spring Boot 2.4-2.7.

Split the WebFlux application toolkit. Reactor removed Signal#getContext() in
3.5.0 and ContextView does not exist before 3.4.0, so no single artifact can
serve both; continueTracing has been broken on all of Spring Boot 3 and 4 since
Nov 2022. apm-toolkit-webflux becomes apm-toolkit-webflux-5.x (package
...toolkit.webflux.v5, unchanged behaviour) and a new apm-toolkit-webflux-6.x
(package ...toolkit.webflux.v6) compiles against Reactor 3.5 and uses
Signal#getContextView(). Compiling forward is deliberate: the module rotted
silently for three years because it compiled against the oldest Reactor, so
javac never flagged the removal. Distinct packages make picking the wrong
artifact a compile error rather than a runtime NoSuchMethodError. The
activation keeps matching the legacy un-versioned names, so applications that
never upgrade their toolkit still work against a newer agent.

No plugin, framework version or Reactor version loses support.

Add webflux-6.x-7.x-scenario (Spring Boot 3.0 -> 4.1, one version per minor)
and gateway-5.x-scenario (Spring Cloud Gateway 5.0.3 on Spring Boot 4). Both
drive a full round-trip so the cross-process ref is asserted, which is the only
thing that proves the sw8 header is read back correctly.

Closes apache/skywalking#14047
@wu-sheng wu-sheng added this to the 9.8.0 milestone Aug 29, 2026
@wu-sheng wu-sheng added enhancement New feature or request plugin feature labels Aug 29, 2026
…n docs

Only stop the span this invocation created. WebFluxSkyWalkingOperatorsInterceptor
created a local span conditionally — a snapshot has to be present, which it is
not for an empty Reactor context or a ServerWebExchange that was never enhanced
because the optional WebFlux plugin is absent — but afterMethod called
ContextManager.stopSpan() and handleMethodException called activeSpan().log()
unconditionally. That could stop or log against an unrelated active span. The
flaw predates this PR; widening the activation to the v5/v6 packages exposes
both replacement artifacts to it, so fix it here.

Migrated only this interceptor to the V2 API so MethodInvocationContext can
carry the "did I create a span" decision per invocation, which also handles
nesting. The shared WebFluxSkyWalkingStaticMethodsAroundInterceptor base is left
on V1 — its five other subclasses are pure trace-id/correlation getters that
never create or stop spans — with its two stateless helpers made static so the
V2 interceptor can still use them.

Verified without the optional WebFlux plugin installed, so no snapshot ever
reaches the Reactor context: no intercept failure, no span-lifecycle error. With
the plugin installed the WebFluxOperators/onNext span and the cross-process ref
are unchanged and the scenario still validates.

Docs:
- The migration note claimed there is no silent runtime failure. That only holds
  for old-toolkit/new-agent. A 9.8.0+ toolkit against a <= 9.7.0 agent compiles
  and runs but is silently untraced, because the older agent does not know the
  .v5/.v6 class names. Documented as a compatibility table, with the instruction
  to upgrade the agent first or together.
- The continueTracing examples mixed Reactor generations: one uses
  Mono#subscriberContext, absent in 3.5+, so it cannot compile for the Boot 3/4
  users the page now documents; another uses Mono#deferContextual and
  Context#of(ContextView), absent before 3.4. Labelled each by generation, and
  noted that the ServerWebExchange overload is generation-agnostic.
- Corrected the v5 lower bound to Reactor 3.1.3. Signal#getContext() is absent
  in 3.1.0-3.1.2 and first appears in 3.1.3, verified against the published jars.
@wu-sheng
wu-sheng merged commit 25e27a8 into main Aug 29, 2026
290 of 465 checks passed
@wu-sheng
wu-sheng deleted the support-spring-framework-7 branch August 29, 2026 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request feature plugin

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants