docs(workers-for-platforms): per-tenant observability with Workers Logs - #32760
docs(workers-for-platforms): per-tenant observability with Workers Logs#32760Ankcorn wants to merge 4 commits into
Conversation
Review✅ No issues found in commit Code ReviewThis code review is in beta and may not always be helpful — use your judgment. No code review issues found. ConventionsNo convention issues found. Style Guide ReviewNo style-guide issues found. CommandsOnly codeowners can run commands. Post a comment with the command to trigger it.
|
| ## Logs | ||
|
|
||
| Learn how to access logs with Workers for Platforms. | ||
| There are a few ways to access logs with Workers for Platforms. For most platforms, [Workers Logs](#workers-logs) is the best starting point: it is a fully managed logging platform, so you get storage, indexing, and a queryable API without building or operating a logging pipeline of your own. Reach for Logpush or Tail Workers when you need to forward logs to your own destination. |
There was a problem hiding this comment.
Here and elsewhere in the docs — I think the "Workers Logs" versus Logpush framing here is confusing, because you can export both logs and traces like this:
https://developers.cloudflare.com/workers/observability/exporting-opentelemetry-data/
But when we say "reach for logpush" — that sends people towards:
https://developers.cloudflare.com/workers/observability/logs/logpush/
Which is different
Can DM about — but are we solving this?
There was a problem hiding this comment.
Yes we are on the path of solving this. Logpush jobs and OTel Destinations will get folded into a single "Export" section in the Dash. Part of Birthday Week, can share docs if needed.
| Use [Tail Workers](/workers/observability/logs/tail-workers/) instead of Logpush if you want granular control over formatting before logs are sent to their destination to receive [diagnostics channel events](/workers/runtime-apis/nodejs/diagnostics-channel), or if you want logs delivered in real-time. | ||
|
|
||
| Adding a Tail Worker to your dispatch Worker collects logs for both the dispatch Worker and for any user Workers in the dispatch namespace. Logs are automatically collected for all new Workers added to a dispatch namespace. To enable logging for an individual user Worker rather than an entire dispatch namespace, add the [Tail Worker configuration](/workers/observability/logs/tail-workers/#configure-tail-workers) directly to the user Worker. | ||
| To collect logs from a user Worker, add the [Tail Worker configuration](/workers/observability/logs/tail-workers/#configure-tail-workers) directly to that user Worker. |
There was a problem hiding this comment.
Is this actually the path we recommend?
I would expect the path here to involve something like
- setting up logpush job
- showing you how you can filter out first-party dispatch Worker logs, to get only user workers
- showing you how you can filter only logs for a given user worker
Shipping off logs every invocation of a Tail Worker is not gonna work for most people's o11y pipelines, just can't handle the load
There was a problem hiding this comment.
No the recommended path is to use workers observability
The doc prioritises them
- use workers observability for fully managed pipeline and storage
- use logpush - let's catch up here :)
- use a tailworker - if you want granular control over formatting before logs are sent to their destination to receive diagnostics channel events, or if you want logs delivered in real-time.
There was a problem hiding this comment.
But above we say:
There are a few ways to access logs with Workers for Platforms. For most platforms, Workers Logs is the best starting point: it is a fully managed logging platform, so you get storage, indexing, and a queryable API without building or operating a logging pipeline of your own. Reach for Logpush or Tail Workers when you need to forward logs to your own destination.
Which makes it sound like in order to push logs to your own destination you need to use Tail Workers or do something other than using Workers Observability
I don't understand why I can't enable observability (logs) like I can for tracing and export to 3rd party destination — and instead need to use tail workers or use a different flavor of logpush
There was a problem hiding this comment.
ah gotcha - I see the confusion. The word "destination" is overloaded here. This has nothing to do with the destination feature we shipped last year with the traces launch. I will rephrase that
Use the standard Upload User Worker API path, documented GraphQL quantile fields, and prose instead of a two-item list.
Avoid "destination" — it collides with Observability destinations.
4 participants
Why
The Workers for Platforms observability page didn't explain how platforms give each end user an isolated view of their own Worker's logs and metrics. It also described Tail Worker behaviour that no longer matches the runtime.
What
observabilityblock in upload metadata, and how the platform controls enablement/sampling.$metadata.servicein thecloudflare-workersdataset, applied server-side so one tenant can't read another's telemetry.workersInvocationsAdaptivefiltered byscriptName(works without Workers Logs enabled).