Configure OpenTelemetry (v7.1.1+)

Beginning in v7.1.1, the Metric Insights platform can be configured to export application performance telemetry to external monitoring systems using OpenTelemetry. This optional integration allows you to monitor the Metric Insights application in an APM tool of your choice. This article describes how to enable this feature using Splunk Observability Cloud (formerly SignalFx) as an example.

To configure OpenTelemetry integration you will need to obtain a pre-configured web Docker image from Metric Insights Support, copy Splunk Ingest Token, Endpoint and Realm values, and update the environment variables of the web service with those values.

NOTES:

  • This integration requires a Metric Insights web container that has the OpenTelemetry libraries installed. Contact [email protected] to obtain a web Docker image with OpenTelemetry enabled for your version.
  • Currently, only the Metric Insights web service is instrumented for OpenTelemetry.
  • The example below uses Splunk Observability Cloud, but you can adapt the configuration for any OpenTelemetry-compatible backend.

Table of contents:

  1. Prepare Splunk Observability Cloud for Data Ingest
    1. Access Splunk Observability Cloud
    2. Create an Ingest Token
      1. Access Settings
      2. Create New Token
      3. Enter Token Details
      4. Proceed with Token Creation
      5. Set Expiration Date
    3. Copy Token Value
    4. Copy Real-Time Data Ingest Endpoint
      1. Access My Profile
      2. Copy Endpoint Value
  2. Configure Metric Insights Web Service Environment Variables for OpenTelemetry
    1. Simple Install
      1. Set Environment Variables
      2. Recreate the Web Container
    2. Orchestrated Environments
      1. Set Environment Variables
      2. Restart the Web Service
    3. Verifying Data Ingestion in Splunk Observability

1. Prepare Splunk Observability Cloud for Data Ingest

1.1. Access Splunk Observability Cloud

Access https://www.splunk.com/

  1. Access Observability
  2. Access Observability Cloud

NOTE: If you don’t already have one, create a Splunk Observability Cloud account (a free trial can be obtained via Splunk’s website). Once logged in to the Splunk Observability portal, ensure you have appropriate admin rights to create tokens.

1.2. Create an Ingest Token

1.2.1. Access Settings

  1. Access Settings

1.2.2. Create New Token

  1. Access Access Tokens
  2. [Create Token]

1.2.3. Enter Token Details

  1. Enter Token Name
  2. Activate the INGEST token checkbox
    • This grants permission to send metric and trace data
  3. Activate the Please accept to continue with your selection checkbox
  4. [Next]

1.2.4. Proceed with Token Creation

  1. Accept the defaults, [Next]

1.2.5. Set Expiration Date

  1. Set Token Expiration date
  2. [Create]

1.3. Copy Token Value

  1. Copy the Token value, it is required for the OTEL_EXPORTER_OTLP_HEADERS environment variable
    • The Splunk ingest token corresponds to the OpenTelemetry header for authentication

1.4. Copy Real-Time Data Ingest Endpoint

1.4.1. Access My Profile

  1. Open the User menu at the top-right of the page
  2. [My Profile]

1.4.2. Copy Endpoint Value

  1. Access Organizations
  2. Copy the Real-time Data Ingest Endpoint value, it is required for the environment variable OTEL_EXPORTER_OTLP_TRACES_ENDPOINT
    • Copy the Realm value as well.

With the Ingest Token and Realm/Endpoint in hand, you have the two pieces of information needed to configure Metric Insights:

  • Access Token – will be used in OTEL_EXPORTER_OTLP_HEADERS (as the Splunk token header).
  • Ingest Endpoint, Realm – will be used in OTEL_EXPORTER_OTLP_TRACES_ENDPOINT (the URL for sending trace data).

2. Configure Metric Insights Web Service Environment Variables for OpenTelemetry

NOTE: By default, Metric Insights application does not support OpenTelemetry. You will need a pre-configured web Docker image, which can be requested from [email protected]. Set the correct environment variables to enable the OpenTelemetry integration.

Configure the required environment variables of the Metric Insights web service to enable and direct the OpenTelemetry data to Splunk.

Parameter Name Required/Optional Description

OTEL_PHP_AUTOLOAD_ENABLED

 Required

Set to true to enable the OpenTelemetry integration.

OTEL_SERVICE_NAME

 Required

Logical name for your service (e.g., "metric-insights" or "mi"). 

  • This name will appear in Splunk Observability to identify the application.

OTEL_RESOURCE_ATTRIBUTES

 Required

Resource tags for your service, such as deployment environment or other identifiers. 

  • Example: "deployment.environment=prod,service.version=7.1.1"

OTEL_EXPORTER_OTLP_TRACES_HEADERS

 Required

Authentication header for the OTLP exporter. 

  • For Splunk Observability, this must be set to X-SF-TOKEN=<your_ingest_token>, where <your_ingest_token> is the token you have previously created.

OTEL_EXPORTER_OTLP_ENDPOINT

 Required

Endpoint URL for sending traces. 

NOTES:

  • For Splunk, this is https://ingest.<realm>.signalfx.com/v2/trace/otlp, where <realm> is your realm; e.g., us0, us1.

  • Make sure to include the /v2/trace/otlp path.

  • Data would default to the us0 realm if a realm is not specified.

OTEL_SDK_DISABLED 

 Optional

Set this to true to disable OpenTelemetry; e.g., if you need to turn off telemetry without removing all the other settings.

Example configuration:

OTEL_PHP_AUTOLOAD_ENABLED=true
OTEL_SERVICE_NAME="metric-insights"
OTEL_RESOURCE_ATTRIBUTES="deployment.environment=dev,service.version=7.1.1"
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://ingest.us1.signalfx.com/v2/trace/otlp
OTEL_EXPORTER_OTLP_TRACES_HEADERS="X-SF-TOKEN=ABC123XYZ0000000000"

Simple Install

2.1. Set Environment Variables

  1. On the Metric Insights host server, open the file /opt/mi/config/deployment/credentials/web.env
    • This file contains environment settings for the Metric Insights web container.
  2. Add the OpenTelemetry configuration variables as identified above.

2.2. Recreate the Web Container

Apply the changes by restarting the web  container. Run the following command on the Metric Insights host:

mi-control up -d --force-recreate web

The web service should start up again and begin sending telemetry data to the specified endpoint.

Orchestrated Environments

2.1. Set Environment Variables

Modify the environment variable settings for the Metric Insights web service in your orchestration configuration. Add the OTEL_* variables as described above.

NOTE: Ensure you handle the OTEL_EXPORTER_OTLP_TRACES_HEADERS value (the Splunk token) securely. For Kubernetes, you might create a Secret for the token and reference it in the pod spec, rather than including the raw token in the manifest.

  • For example, use a Kubernetes Secret with a key for the token and set OTEL_EXPORTER_OTLP_TRACES_HEADERS to X-SF-TOKEN=<secretValue> via an environment variable reference.

2.2. Restart the Web Service

Apply the updated configuration so that the Metric Insights web container is restarted with the new environment. Once the new configuration is applied, the Metric Insights web application will start up with OpenTelemetry instrumentation active and pointed at Splunk Observability Cloud.

3. Verifying Data Ingestion in Splunk Observability

After configuring and restarting, you should confirm that Metric Insights telemetry is being received in Splunk Observability Cloud:

  • Log in to Splunk Observability Cloud and navigate to the APM (Tracing) section or relevant dashboards.
    • Use the service name you configured (e.g., "metric-insights") to filter or search for traces and metrics.
  • You should begin to see data from Metric Insights appear, such as trace spans, service metrics, or other telemetry associated with the service name and environment you set.
    • It may take a few minutes for data to first appear.
  • If no data appears, double-check the configuration.
    • Ensure that:
    • Checking the web container logs can also provide clues (the OpenTelemetry agent may log initialization info or errors on startup).