OpenTelemetry Java Wrapper Layer doesnt connect to AWS lambda generated segment

[arun-annamalai]

Describe the bug
A clear and concise description of what the bug is.
[Java17 lambda runtime only] When I turn AWS lambda active tracing on, the span that is created the OpenTelemetry Java Wrapper Layer does not connect to the last span that the AWS lambda environment creates. I am invoking the lambda via console GUI. This indicates a problem with the context propagation from the lambda environment to the Java Wrapper Layer.

This bug does not reproduce in java11

Steps to reproduce
If possible, provide a recipe for reproducing the error.
Create a Java17 runtime lambda with java wrapper layer (arn:aws:lambda:us-west-2:184161586896:layer:opentelemetry-javawrapper-0_10_0:1)

What did you expect to see?
A clear and concise description of what you expected to see.
I expect to the span created by the java wrapper layer have the parent set to the span created by the lambda environment.

What did you see instead?
A clear and concise description of what you saw instead.
I saw that the span created by the java wrapper layer did not have a parent and was in a separate trace.

What version of collector/language SDK version did you use?
Version: (e.g., v0.58.0, v1.11.0, etc)
My lambda java application has the following otel sdk versions
opentelemetry-aws-xray-propagator-1.42.0-alpha
opentelemetry-api-1.42.1
opentelemetry-sdk-1.42.1
opentelemetry-aws-sdk-2.2-1.31.0-alpha

Lambda Env Variables
AWS_LAMBDA_EXEC_WRAPPER: /opt/otel-handler

Java: 17

What language layer did you use?
Config: (e.g., Java, Python, etc)
Java Wrapper Layer

Additional context
Add any other context about the problem here.

[tylerbenson]

You need to manually select the xray-lambda propagator instead of the xray propagator.

[arun-annamalai]

ah I see, let me test that out and get back. I think this would warrant a documentation update to make this a bit more clear.

Preferably documented here: https://github.com/open-telemetry/opentelemetry-lambda/blob/main/java/README.md

I can take this work if approved.

[arun-annamalai]

And just to clarify, does this work with Java Wrapper Layer (manual instrumentation)?

OTEL_PROPAGATORS=tracecontext,baggage,xray-lambda

Documentation Source: https://github.com/open-telemetry/semantic-conventions/blob/main/docs/faas/aws-lambda.md#xray-lambda-propagator-configuration

I tried including the above environment variable and encountered this error at runtime upon invoking the lambda:
Unrecognized value for otel.propagators: xray-lambda. Make sure the artifact including the propagator is on the classpath.: io.opentelemetry.sdk.autoconfigure.spi.ConfigurationException io.opentelemetry.sdk.autoconfigure.spi.ConfigurationException: Unrecognized value for otel.propagators: xray-lambda. Make sure the artifact including the propagator is on the classpath. at io.opentelemetry.sdk.autoconfigure.PropagatorConfiguration.getPropagator(PropagatorConfiguration.java:70) at io.opentelemetry.sdk.autoconfigure.PropagatorConfiguration.configurePropagators(PropagatorConfiguration.java:51) at io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdkBuilder.build(AutoConfiguredOpenTelemetrySdkBuilder.java:495) at io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdk.initialize(AutoConfiguredOpenTelemetrySdk.java:36) at io.opentelemetry.instrumentation.awslambdacore.v1_0.TracingRequestStreamWrapper.<init>(TracingRequestStreamWrapper.java:30) at io.opentelemetry.instrumentation.awslambdaevents.v2_2.TracingRequestWrapper.<init>(TracingRequestWrapper.java:32) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Unknown Source) at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source)

This seems to be coming in from the wrapper layer and not my lambda function code since I manually instantiate the otel sdk via code and not via autoconfiguration.

[arun-annamalai]

I noticed that the following SPI file has both the xray and xray-lambda propagator registered: io.opentelemetry.sdk.autoconfigure.spi.internal.ComponentProvider

However this file only has the xray propagator registered: io.opentelemetry.sdk.autoconfigure.spi.ConfigurablePropagatorProvider

Is that intended?

[tylerbenson]

No, I think that's an unintentional oversight. Do you want to submit a PR to fix that in the contrib repo and I can work with the approvers to get it approved?

[arun-annamalai]

yup I opened an issue in the contrib repo and can open a PR there. Does this seem to explain why I am getting an error when using the xray-lambda env variable in the current lambda layer?

[tylerbenson]

That certainly sounds plausible. Thanks for submitting. I approved your PR, but I'm not an "approver".

[arun-annamalai]

I have contacted the xray approvers and they have approved now. Do you know how I would be able to test this out when merged in? Would I need to wait until the latest java contrib release or will this be auto consumed by the otel java wrapper gradle?