feat: migrate built in metrics to OTEL by mutianf · Pull Request #1796 · googleapis/java-bigtable

mutianf · Jun 15, 2023

To use a custom otel instance:

/** Customer's application code */
SdkMeterProviderBuilder sdkMeterProvider = SdkMeterProvider.builder();

// resgister Builtin metrics
BuiltinViews.registerBuiltinViews("project-id", sdkMeterProvider);

OpenTelemetry openTelemetry = OpenTelemetrySdk.builder()
.setMeterProvider(sdkMeterProvider())
.setTracerProvider(..) // customer can set other providers
.build();

// pass in the openTelemetry instance
BigtableDataSettings settings = ..
   .setMetricsProvider(CustomOpenTelemetryMetricsProvider.create(openTelemetry));

Under the hood, BuiltinViews.registerBuiltinViews register the BigtableCloudMonitoringExporter:

registerBuiltinViews("project-id", sdkMeterProvider) {
    sdkMeterProvider
	 // register the metric reader
        .registerMetricReader(
            PeriodicMetricReader.create(
        // meter reader needs to be created with a exporter
                BigtableCloudMonitoringExporter.create(project, credentials)))
        .registerView(OPERATION_LATENCIES_SELECTOR, OPERATION_LATENCIES_VIEW)
}

blakeli0 · Aug 17, 2023

Note that we recently started doing dependency governance, you would need to create a ticket for every new dependency. Once this PR is ready, please reach out to @suztomo regarding the exact process for OpenTelemetry dependencies, see a pilot example b/266431647.

igorbernstein2 · Feb 16, 2024

+    // Builtin metrics will call getCredentialsProvider at which point it'll be a
+    // FixedCredentialProvider.
+    // So disabling in the test code it's fine.


I think you can do this now?

* feat: migrate to OTEL exporter (#1788) * feat: migrate exporter to OTEL * address comments * filter out only bigtable metrics * fix test * use the bom * update * update * update completeResultCode * add a comment * address comments * address comments * update pom * small fix * also check timestamp * address comment * updates * update * do not block on shutdown * chore: refactor factory class (#2081) * chore: refactor TracerFactory creation (#2102) * feat: migrate built in metrics to OTEL (#1796) * feat: migrate exporter to OTEL * address comments * filter out only bigtable metrics * fix test * use the bom * update * update * feat: migrate builtin metrics to OTEl * update completeResultCode * add a comment * udpate * fix tests * remove unrelated changes * fix tests * add documentation * fix test * merge exporter changes * address comments * rebase on otel * revert changes in stats * fix import * update * merge back the endpoint change * refactor constants and settings * refactor and fix tests * remove unused dependency * add some javadoc * address part of the comments * update test * test with nano * measure everything in nanos and publish with double histogram * address comments * fix test * add toString * merge from main * feat: migrate per connection error count metric to otel (#2133) * feat: migrate per connection error count metric to otel * update test * address comments * remove unnecessary check * clean up statsRecorder * remove dependency * address comments * feat: migrate per connection error exporter to otel (#2152) * chore: clean up stats package (#2163) * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * fix clirr and integration test * fix clirr and integration test * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * log more information on test failures * address comments * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * include version * fix flaky test * remove otel dependencies since they're added to shared dependencies * backport new tests since 2.37 * set feature flag when metrics is enabled * record metrics whenever it's registered * record metrics whenever it's registered * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * remove stale reference * clean up --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

feat: migrate exporter to OTEL

08cb1b8

product-auto-label Bot added size: xl Pull request size is extra large. api: bigtable Issues related to the googleapis/java-bigtable API. labels Jun 15, 2023

mutianf force-pushed the migrate12 branch from 5403ea8 to fafb056 Compare June 15, 2023 18:03

address comments

26a648a

igorbernstein2 mentioned this pull request Jun 26, 2023

build(deps): update dependency org.apache.maven.plugins:maven-shade-plugin to v3.5.0 #1798

Closed

1 task

blakeli0 reviewed Jun 27, 2023

View reviewed changes

Comment thread ...igtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinInstruments.java Outdated

blakeli0 reviewed Jun 27, 2023

View reviewed changes

Comment thread ...rc/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracerFactory.java Outdated

blakeli0 reviewed Jun 27, 2023

View reviewed changes

Comment thread ...e/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsAttributes.java Outdated

mutianf added 5 commits June 28, 2023 11:52

filter out only bigtable metrics

91fcd80

fix test

32ac2be

use the bom

af6cbb2

update

e87cf91

update

e2b7e58

mutianf force-pushed the migrate12 branch from fafb056 to b3e0f8e Compare July 6, 2023 16:43

feat: migrate builtin metrics to OTEl

6d127e3

mutianf force-pushed the migrate12 branch from b3e0f8e to 6d127e3 Compare July 6, 2023 19:08

mutianf added 2 commits July 12, 2023 13:06

update completeResultCode

e8be9c2

add a comment

cd04252

mutianf added 2 commits August 23, 2023 16:41

Merge branch 'main' into migrate12

31b6d5c

udpate

3d9b30d

mutianf force-pushed the migrate12 branch from 225e08b to 3d9b30d Compare August 23, 2023 21:20

mutianf added 4 commits August 24, 2023 16:56

fix tests

9fcd0f3

remove unrelated changes

0c80a4c

fix tests

4b66a0e

add documentation

9414ece

mutianf marked this pull request as ready for review September 1, 2023 18:20

mutianf requested review from a team September 1, 2023 18:20

mutianf requested a review from a team January 24, 2024 15:40

mutianf changed the base branch from main to otel January 24, 2024 15:45

mutianf added 4 commits January 24, 2024 11:04

revert changes in stats

9a435f7

fix import

a797c96

update

d534f7f

merge back the endpoint change

2dd6586

igorbernstein2 suggested changes Jan 29, 2024

View reviewed changes

mutianf added 2 commits January 30, 2024 14:27

refactor constants and settings

54a8d25

refactor and fix tests

96099b8

mutianf force-pushed the migrate12 branch from 3d8d830 to 96099b8 Compare January 30, 2024 22:44

mutianf added 3 commits January 31, 2024 09:14

remove unused dependency

803f5b5

add some javadoc

66ab16a

Merge branch 'otel' into migrate12

c5fd5ca

igorbernstein2 suggested changes Feb 6, 2024

View reviewed changes

mutianf added 2 commits February 6, 2024 14:45

address part of the comments

4f577fa

update test

0f21a31

mutianf commented Feb 6, 2024

View reviewed changes

Comment thread google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/it/BuiltinMetricsIT.java

mutianf commented Feb 6, 2024

View reviewed changes

Comment thread ...table/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracer.java

mutianf added 2 commits February 7, 2024 11:09

test with nano

56f6b71

measure everything in nanos and publish with double histogram

14f22a8

mutianf mentioned this pull request Feb 8, 2024

feat: client side metrics handlers googleapis/python-bigtable#924

Closed

Merge branch 'otel' into migrate12

5bfa166

igorbernstein2 suggested changes Feb 16, 2024

View reviewed changes

mutianf added 2 commits February 20, 2024 12:26

address comments

523724b

fix test

db6c496

igorbernstein2 approved these changes Feb 21, 2024

View reviewed changes

add toString

8f511cf

mutianf merged commit 08e1719 into googleapis:otel Feb 21, 2024

mutianf deleted the migrate12 branch February 21, 2024 15:56

Search code, repositories, users, issues, pull requests...

Uh oh!

Conversation

mutianf commented Jun 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

blakeli0 commented Aug 17, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

igorbernstein2 Feb 16, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mutianf commented Jun 15, 2023 •

edited

Loading