Skip to content

otelconf: fix Prometheus reader converting dot-style labels to underscores#8696

Open
dmitryax wants to merge 1 commit intoopen-telemetry:mainfrom
dmitryax:fix-otelconf-prometheus-label-escaping
Open

otelconf: fix Prometheus reader converting dot-style labels to underscores#8696
dmitryax wants to merge 1 commit intoopen-telemetry:mainfrom
dmitryax:fix-otelconf-prometheus-label-escaping

Conversation

@dmitryax
Copy link
Copy Markdown
Member

@dmitryax dmitryax commented Mar 19, 2026

When both without_type_suffix and without_units are set (which is the default config for the OTel Collector), prometheusReaderOpts was selecting UnderscoreEscapingWithoutSuffixes as the translation strategy. This strategy sets LabelNamer{UTF8Allowed: false}, converting OTel dot-style label names (e.g. service.name) to underscore-style (service_name) in target_info and on all metric datapoints.

Fix by using NoTranslation instead, which preserves dot-style label names (ShouldEscape()=false) and suppresses metric name suffixes (ShouldAddSuffixes()=false).

The restored behavior is close to what we had before #8595.

Related issue: open-telemetry/opentelemetry-collector-contrib#47011

I manually tested it and it solves the issue above for the collector

@dmitryax dmitryax requested review from a team and pellared as code owners March 19, 2026 21:20
@github-actions github-actions bot requested a review from codeboten March 19, 2026 21:21
@dmitryax dmitryax changed the title otelconf: fix Prometheus reader converting dot-style labels to otelconf: fix Prometheus reader converting dot-style labels to underscores Mar 19, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 19, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.1%. Comparing base (529b99f) to head (a395b37).

Additional details and impacted files

Impacted file tree graph

@@          Coverage Diff          @@
##            main   #8696   +/-   ##
=====================================
  Coverage   82.1%   82.1%           
=====================================
  Files        183     183           
  Lines      13790   13790           
=====================================
  Hits       11332   11332           
  Misses      2054    2054           
  Partials     404     404           
Files with missing lines Coverage Δ
otelconf/v0.3.0/metric.go 89.7% <100.0%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

underscores

When both without_type_suffix and without_units are set (which is the
default config for the OTel Collector), prometheusReaderOpts was
selecting
UnderscoreEscapingWithoutSuffixes as the translation strategy. This
strategy
sets LabelNamer{UTF8Allowed: false}, converting OTel dot-style label
names
(e.g. service.name) to underscore-style (service_name) in target_info
and
on all metric datapoints.

Fix by using NoTranslation instead, which preserves dot-style label
names
(ShouldEscape()=false) and suppresses metric name suffixes
(ShouldAddSuffixes()=false).

Related issue:
open-telemetry/opentelemetry-collector-contrib#47011
@dmitryax dmitryax force-pushed the fix-otelconf-prometheus-label-escaping branch from 75d9783 to a395b37 Compare March 19, 2026 21:36
_, err = buf.ReadFrom(resp.Body)
require.NoError(t, err)
body := buf.String()

Copy link
Copy Markdown
Member

@XSAM XSAM Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we verify the body contains target_info and not contains the target.info? This prevents MetricNamer accidentally takes TranslationStrategyOption.NoTranslation seriously in the future.

@dmathieu
Copy link
Copy Markdown
Member

See collector upgrade slack thread: https://cloud-native.slack.com/archives/C02JJK840SH/p1773874400856229

I agree with @codeboten's comments on that thread. When setting up the prometheus exporter, we should follow the prometheus conventions, not enforce the OTel ones.

@MrAlias
Copy link
Copy Markdown
Contributor

MrAlias commented Mar 23, 2026

cc @dashpole

@dashpole
Copy link
Copy Markdown
Contributor

Is the collector able to use NoTranslation if it wants this behavior? Escaping to underscores is the correct default behavior according to the prometheus sdk exporter spec.

@dmitryax
Copy link
Copy Markdown
Member Author

dmitryax commented Mar 26, 2026

This is just an attempt to address a regression on the collector side appeared in 0.148.0 release when instead of getting the following resource for internal collector metrics

Resource attributes:
     -> service.name: Str(otelcontribcol)
     -> service.instance.id: Str(35ccf925-339a-47e6-a3f8-043013267950)
     -> server.port: Str(8888)
     -> url.scheme: Str(http)
     -> service.version: Str(0.147.0-dev)

we started getting

Resource attributes:
     -> service.name: Str(otel-collector)
     -> service.instance.id: Str(0.0.0.0:8888)
     -> server.port: Str(8888)
     -> url.scheme: Str(http)
     -> service_instance_id: Str(09da6493-2d67-4201-b962-8396ba5c5618)
     -> service_name: Str(otelcontribcol)
     -> service_version: Str(0.148.0-dev)

with the collector's default configuration.

According the @dmathieu this can be resolved by moving collector go sdk config to 1.0. If that better way, I'm happy to close this PR.

For now I've marked this issue as a collector release blocker

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants