OpenTelemetry with fan out and custom protocol

I’m completely new to OpenTelemetry and trying to figure out some things. Assume I have a chain of services:

A ----> B ----> C

The services communicate with each other over a custom binary UDP protocol. There’s also a fan out step at B to C and all processing is asynchronous, so A won’t wait for B to process the data it gets and the same applies to C. I can embed trace IDs in the data, so I’m able to use a propagator to propagate a trace context.

Now my questions:

1. Is the SpanKind of a span just a label or does it actually change the semantics of a span?
2. Since the processing tree starts at A we must create the root span at A. Is the root span somehow special to create? I’m currently just doing tracer.spanBuilder("foo").startSpan()
3. A must call end() on the span it creates before B or C starts processing. The subspan of B must also call end before C starts processing. Will this cause any problem or do I just need to set the correct SpanKind?

>Solution :

Let’s see if this can help you,
SpanKind in OpenTelemetry: SpanKind in OpenTelemetry is an attribute associated with a span that indicates the nature of the span. Spans can be of two primary types: SpanKind.SERVER and SpanKind.CLIENT, which are used to represent incoming and outgoing requests, respectively. Using SpanKind helps clearly distinguish whether a span represents an incoming or outgoing request. Spans in OpenTelemetry are often accompanied by labels that provide additional information.

Root Span: The root span is the top-level span in a trace tree and represents the beginning of a trace. Usually, the root span is created in the main application or at the top of a call hierarchy. Creating the root span is crucial because all other spans within a trace are direct children or descendants of the root span. There’s nothing particularly special about creating a root span, but it’s essential to do so at the beginning of processing a request or transaction.

Order of Span Termination: The order in which spans are terminated is important to accurately establish the duration and temporal relationships between spans. In your scenario, A should finish its span before B or C begin processing. This is important to ensure that timings are correct in the trace. You can do this by using the end() method on spans, as you mentioned. Make sure to end one span before starting another to correctly represent the temporal order of operations.

Regarding your custom binary UDP protocol and trace context propagation, you’ll need to implement custom trace context propagation through your services based on the protocol you use for UDP communication. Additionally, you’ll need to ensure that each service has an OpenTelemetry library configured to collect telemetry data and propagate trace context correctly.