Customization of OTel spans for specific endpoints

[anuraaga]

🚀 Feature Proposal

Currently, OTel spans are all created generically as DB spans by the transport

https://github.com/elastic/elastic-transport-js/blob/main/src/Transport.ts#L698

As Elasticsearch is so generic, this isn't entirely precise, notably OTel has genai specific conventions that apply instead

https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/

It would be good if the JS sdk can generate spans following the gen ai conventions for the inference endpoint. These would correspond to ones generated for the openai SDK for example here

https://github.com/elastic/elastic-otel-node/blob/main/packages/instrumentation-openai/src/instrumentation.ts#L246

This mainly requires an actual mechanism for such spans in this SDK as it would be the first one AFAIK. I have a POC that takes an approach of creating a span within the inference method itself, examining the request and response. The transport span is suppressed by just not passing meta to the transport. This is probably a conflation of meta and it may make sense to have a separate suppressTelemetry or similar flag passed along it instead. Open to any thoughts on a clean way to allow endpoints within the SDK to provide custom telemetry for non-DB features.

Motivation

I am working with @codefromthecrypt and @trentm on improving genai observability within elastic products. Notably, we'd like to be able to see these spans when debugging Kibana development that uses the inference plugin, which would use these spans.

Example

No response