[Playground] OTLP-Mapper - instana to otlp span mapper

[abhilash-sivan]

Summary

Introduces the Instana-to-OTLP Converter Playground.

• Adds support for exporting traces via OTLP (4318/v1/traces).
• Supports OpenTelemetry Semantic Conventions (SemConv) v1.23.

Out of Scope

• Metrics export
• Logs export

TODO

Core Implementation

• Handle agent port usage (data port vs. connection port) (chore: make agent data port configurable independently from the agent connection port #2559)
• Support multiple span data keys (e.g., span.data.peer, span.data.mongo)
• Finalize overall design
• Implement SemConv lookup handling
• Implement span name mapping
• Complete metadata attribute mapping:
• Direct mappings
• Computed mappings
• Research span status mapping
• Research span name mapping (@abhilash-sivan)
• Implement OpenTelemetry instrumentation-specific handling
• Implement log-specific handling
• Implement resource mapping
• Implement required metrics mappings
• Implement agent connection logic
• Error handling
• Resource-service name mapping
• Dual transformation flow in span data @abhilash-sivan
• Implement configuration option for enabling OTLP export (INSTANA_OTLP_ENABLED?) - NOT PART OF THIS

Span Data Mapping

• HTTP
• Kafka
• Databases
• Cloud services
• OpenTelemetry instrumentation

Testing

Unit Tests

• Single-span scenarios
• Multi-span scenarios

End-to-End Testing (@abhilash-sivan)

• SemConv v1.23 validation with the current design
• SemConv v1.41 compatibility validation
• OpenTelemetry wrapper instrumentation testing
• Metadata validation with SemConv v1.23 and span data validation with v1.43
• without metrics

Performance Testing

• Performance testing

Special Cases

• Fix type definitions

Issue Tracking

• UI Issues: https://ibm.box.com/s/0v5wli0pgvrlg8jqpkhi59mb52o28u75
• Results: https://ibm.box.com/s/0v5wli0pgvrlg8jqpkhi59mb52o28u75

Validation Checklist

Before merging, verify the following:

• Traces are successfully exported through the OTLP endpoint (4318/v1/traces).
• Metrics continue to be exported through the existing endpoint.
• The UI correctly identifies and displays applications.

Future Enhancements/TODO

• Support the agent announcement cycle disablement use case.
• Evaluate exporting logs to the OTLP /v1/logs endpoint.
• Implement metrics mapping.
• Refactor the current dual transformation flow in span data (internal mapping and OTLP mapping).

References

• https://opentelemetry.io/blog/2023/http-conventions-declared-stable/
• https://github.com/open-telemetry/opentelemetry-proto/blob/main/opentelemetry/proto/trace/v1/trace.proto

[kirrg001]

test: send traces via otlp port and metrics via standard port - Idenfies 2 diff apps 1. Otel and 2. Node.js

This needs to be communicated in the next WG. Traces & Metrics mapping is required.

[kirrg001]

qs: Why is the OTLP module not part of the tracing folder?

[aryamohanan]

This module handles all telemetry signals- Traces, Metrics, and Logs. Placing it under the tracing folder would unnecessarily couple it to tracing-specific concepts such as spans. Keeping it at the root allows the logic to remain shared and signal-agnostic. It also provides flexibility for future enhancements, such as extending the OTLP implementation to emit additional telemetry signals like logs.

[kirrg001]

IMO it belongs to core/tracing/exporters

[kirrg001]

We can discuss this before merging. No big topic

[kirrg001]

  const originFrom = rawSpan.data?.resource || null;
 const rKey = resourceFactory.getResourceKey(originFrom);

This looks wrong to me? 🤔

This must be rawSpan.f?

[kirrg001]

Any instana span has .f - doesn't matter if its coming from ours instrumentations or from otel integration.

[kirrg001]

// base loopup based on v1.23 with some common mappings as well

Okay I understood the current sem conv structure. So yeah the base mappings are secretly from v1.23.

I'd like to suggest this:

index.js

const VERSIONS = {
  '1.23': require('./v1.23'), // explicit version
  '1.42': require('./v1.42') // explicit version
};

Three folders

• base/
  • mappings.js
• v1.23/
  • index.js
  • mappings.js
• v1.43/
  • index.js
  • mappings.js
• merge.js

base/mappings.js

module.exports = require('../v1.23/mappings');

v1.23/mappings.js

const MAPPINGS = {
  metadata: {
    TRACE_ID: 'traceId',
...

v1.23/index.js

const { merge } = require('../merge');
const base = require('../base/mappings').MAPPINGS;
const { MAPPINGS } = require('./mappings');

module.exports = merge(base, MAPPINGS);

v1.41/index.js

const { merge } = require('../merge');
const base = require('../base/mappings').MAPPINGS;
const { MAPPINGS } = require('./mappings');

module.exports = merge(base, MAPPINGS);

v1.41/mappings.js

const MAPPINGS = {
  http: {
    REQUEST_METHOD: 'http.request.method',
    RESPONSE_STATUS: 'http.response.status_code',
...

The concept of overrides === merge.

[kirrg001]

Is that right?

combineFields(values)

combineFields takes 2 arguments.