Skip to content

FFL-3430 / FFL-3434: Protect Android and Kotlin flag evaluation telemetry - #4025

Open
vjfridge wants to merge 8 commits into
developfrom
vickie/ffl-3430/android-client-pii
Open

vjfridge wants to merge 8 commits into
developfrom
vickie/ffl-3430/android-client-pii

Conversation

@vjfridge

@vjfridge vjfridge commented Oct 21, 2026 •

Copy link
Copy Markdown

What does this PR do?

Protect Android flag evaluation telemetry by default and validate the Kotlin OpenFeature path in the same repository.

  • Parse only literal data.attributes.observeFullEvaluationData=true as opt-in; persist it with assignments, including empty configurations and offline cache restoration.
  • Capture flag, context, and consent atomically for success and error/default evaluations. Preserve consent through metadata, aggregation, shutdown flush, and retry-body construction.
  • Hash protected targeting keys using strict UTF-8 SHA-256 (sha256_ plus lowercase hex); preserve empty keys, omit missing/malformed keys without dropping counts, and omit evaluation context. Android's existing context omission in full-data mode remains unchanged.
  • Emit only the agreed eight stable error codes in both modes, use GENERAL for unknown failures, and separate consent modes in aggregation without retaining discarded context.
  • Preserve native consent metadata on OpenFeature structured results. Extend existing Android integration tests, with an option to consume the exact built SDK/provider release AARs.

Motivation

FFL-3430 and FFL-3434, following the October 21 client decisions in FFL-3427 and FFL-3437. Normal configuration refresh governs consent. This does not add new context bounds, queues, degradation tiers, or server-parity requirements.

Additional Notes

Review follow-up — d414bca614c1f255738c5522b1892bf50f93df98

  • Validate targeting-key Unicode before either hashing or returning a full-data key. Malformed keys are omitted in both modes without losing counts.
  • Compute the telemetry identity with the installed atomic configuration and reuse it for success/error evaluations. Bridge replay uses a per-client single-entry cache; native not-ready errors preserve that cache.
  • Merge current develop, preserve the separate native tracking hooks, apply both style suggestions, and regenerate the internal API surface.

Validation

  • 682 native Flags unit tests, 77 Kotlin provider unit tests, and 115 release integration tests passed: zero failures, errors, or skips.
  • All 115 integration tests also passed using the exact copied native/provider release AARs; the release instrumentation APK builds against those artifacts.
  • Release SDK/provider build passed with CI=true (warnings as errors). Changed library Detekt and Android lint passed; formatting passed in all three modules. Both libraries passed all five generated-file checks.
  • Negative controls: the new malformed-Unicode regression fails against the original AAR only in full-data mode and passes in both modes with the fix; the bridge-cache preservation regression failed before its fix and passes afterward.
  • Reproduction commands and coverage are in reliability/single-fit/flags/README.md.
  • The separate Foundation documentation checker could not run because this workspace does not have the FactoryKit runtime. Documentation was reviewed manually.

Android emulator validation
Verified 48/48 privacy instrumentation tests passed, with zero failures or skips, on d414bca614c1f255738c5522b1892bf50f93df98. All 16 expected direct SDK/OpenFeature cases were present on each API level.

Android API Passed Job
23 16/16 Emulator run
28 16/16 Emulator run
37 16/16 Emulator run

Static checks, unit/coverage checks, API compatibility, and all single-feature integration jobs also passed. The broader pipeline still has legacy/NDK instrumentation jobs running.

Current pipeline

The device suite uses real Android core, disk persistence, scheduled aggregation/upload and HTTP requests to a loopback server. It covers direct SDK and OpenFeature paths, consent refresh, stable errors, malformed/empty/valid Unicode keys in both modes, opt-out, cache restoration (including malformed keys in both modes), and a real 503 retry. CI builds and copies both release AARs, logs their hashes, and supplies them through flagsArtifactDirectory. Offline restart cases invoke the provider directly to isolate cached-state behavior from OpenFeature initialization policy. Shutdown cases use the feature stop callback and core test drain hook. This does not claim live backend acceptance or process-kill recovery.

Exact local artifacts — version 3.16.0-SNAPSHOT, built from the validated source tree:

  • Native release AAR: SHA-256 21a6826e18e252c0ab81a8ee12458a21b938368b73b366154ee6ae983f93508b.
  • Provider release AAR: SHA-256 02393e3612dedd1f2b12625be4a9541e6f22dc1cae29814df9c6b0e2308238ba.

React Native handoff
The optional internal UnparsedFlagWithConsent preserves UnparsedFlag ABI compatibility. The bridge must serialize the consent boolean, preserve it through the JS snapshot/cache, and reconstruct the consent-aware interface when replaying. Old bridge implementations remain protected; native Android owns validation and hashing.

Existing snapshot replay signatures remain intact, including the nullable telemetryTargetingKey overload for React Native #1510. The separate trackFlagEvaluationEvent hook now also has an overload accepting captured consent and a nullable telemetry identity. Use that overload for errors without assignments; the original four-argument hook derives consent from the supplied flag and safely defaults to protected mode when unavailable. Exposure attribution and subject opt-out continue to use the original context. Evaluation-only hooks emit no exposures or RUM annotations.

No React Native, JS, or iOS source is changed here. SDK-owner review, release cutoff, and published native/provider dependency versions remain release-owner decisions tracked by FFL-3437.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Make sure you discussed the feature or bugfix with the maintaining team in an Issue
  • Make sure each commit and the PR mention the Issue number (cf the CONTRIBUTING doc)

Capture consent with assignments and preserve it through cache, evaluation
and aggregation. Validate native and Kotlin provider paths using release AARs.

Ref: FFL-3430
Ref: FFL-3434

Environment: Datadog workspace
@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Pipelines  Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 90.25%
• Overall Coverage: 71.79% (-0.09%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: d414bca | Docs | View more details | Give us feedback!

vjfridge and others added 6 commits October 22, 2026 21:48
Exercise native and Kotlin provider release builds with real Android storage, scheduled uploads, HTTP retries, and consent refresh. Run the suite in the existing API 23, 28, and 37 emulator jobs.

Ref: FFL-3434

Environment: Datadog workspace
Use the artifact dependency mode for the emulator suite and record native/provider hashes in each CI job. Document the connected-device task.

Ref: FFL-3434

Environment: Datadog workspace
Escape unpaired surrogates before UTF-8 cache storage so restored evaluations omit malformed targeting keys without losing counts. Exercise the byte boundary and offline telemetry in regression tests.

Ref: FFL-3430

Environment: Datadog workspace

Co-Authored-By: Codex GPT-6 <noreply@localhost>
Use the sanitized aggregation error code directly, centralize consent field names, and clean up documentation and obsolete test state.

Ref: FFL-3430

Environment: Datadog workspace

Co-Authored-By: Codex GPT-6 <noreply@localhost>
Keep consent with the atomic evaluation snapshot and preserve strict parsing through generated cache models, obfuscated assignments, and lossless Unicode persistence. Cover consent and subject opt-out together.

Ref: FFL-3430

Environment: Datadog workspace

Co-Authored-By: Codex GPT-6 <noreply@localhost>
Keep nullable telemetry identity separate from the assignment subject while preserving snapshot consent, exposure attribution and opt-out.

Ref: FFL-3430

Environment: Datadog workspace

Co-Authored-By: Codex GPT-6 <noreply@localhost>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-23T18:26:14.013335Z d414bca New commits
🔒 Security Review ✅ Completed 2026-10-23T18:28:18.780121Z d414bca New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Consent propagation, cache restoration, telemetry sanitization, and snapshot replay appear consistent with the intended privacy behavior. Runtime behavior remains unverified because this review used static evidence only.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 7e5a7ea

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7e5a7eab41

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

private const val HEX_DIGITS = "0123456789abcdef"

fun targetingKey(value: String?, observeFullEvaluationData: Boolean): String? {
if (value == null || observeFullEvaluationData || value.isEmpty()) return value

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate malformed keys before honoring full-data consent

When observeFullEvaluationData=true and a targeting key contains an unpaired UTF-16 surrogate, this early return bypasses strict UTF-8 validation and forwards the malformed string to JSON/byte serialization, where it can be replacement-encoded instead of omitted. This contradicts the newly documented contract in features/dd-sdk-android-flags/README.md that malformed Unicode keys are omitted without losing the evaluation; validate the string before returning the raw full-data value.

Useful? React with 👍 / 👎.

Comment on lines +23 to +25
val digest = MessageDigest.getInstance("SHA-256")
digest.update(bytes)
"sha256_" + digest.digest().joinToString("") { byte ->

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Cache the protected targeting-key digest

With the default trackEvaluations=true, absent/false consent, and a nonempty targeting key, every flag resolution performs UTF-8 encoding, a crypto-provider lookup, SHA-256, and a 32-element joinToString on the caller thread. Public flag checks can run repeatedly on the UI thread, so recomputing the same subject digest for every evaluation adds avoidable CPU work and allocations; compute it once when the atomic assignment/context state is installed and reuse it during aggregation.

AGENTS.md reference: AGENTS.md:L142-L142

Useful? React with 👍 / 👎.

@maciejburda
maciejburda requested a review from hamorillo October 23, 2026 07:06
@0xnm

0xnm commented Oct 23, 2026

Copy link
Copy Markdown
Member

@hamorillo is out of office, assigning @ambushwork as next week IH.

@0xnm
0xnm requested a review from ambushwork October 23, 2026 07:34
@ambushwork

Copy link
Copy Markdown
Member

Waiting for codex comments and merge conflicts to be addressed before human reviewing.

private const val HEX_DIGITS = "0123456789abcdef"

fun targetingKey(value: String?, observeFullEvaluationData: Boolean): String? {
if (value == null || observeFullEvaluationData || value.isEmpty()) return value

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

value.isNullOrEmpty()

""
}

private fun escapeUnpairedSurrogates(json: String): String {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could be fun String.escapeUnpairedSurrogates() : String

Validate telemetry identities in both consent modes and reuse prepared keys.
Preserve captured bridge consent, merge develop, and extend device coverage.

Ref: FFL-3430
Environment: Datadog workspace
Co-Authored-By: Codex GPT-6 <noreply@localhost>
@domalessi domalessi self-assigned this Oct 23, 2026

@domalessi domalessi left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some editorial feedback inline. Approving so you're not blocked on me!


## React Native handoff

Native Android owns evaluation telemetry protection. The internal assignment snapshot can now

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Native Android owns evaluation telemetry protection. The internal assignment snapshot can now
Native Android owns evaluation telemetry protection. The internal assignment snapshot can

bridge overloads with retained consent, successful and default/error evaluations, unknown error
codes, distinct omitted and empty telemetry keys, and no exposure or RUM side effects. The bridge
fixtures use `doLog=true` to prove that evaluation-only hooks do not trigger exposure logging.
`FlagsPrivacyTest` uses `doLog=false` to ensure it cannot suppress evaluation telemetry. Existing

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`FlagsPrivacyTest` uses `doLog=false` to ensure it cannot suppress evaluation telemetry. Existing
`FlagsPrivacyTest` uses `doLog=false` to verify that it cannot suppress evaluation telemetry. Existing

Comment on lines +93 to +94
protected native release before shipping its integration. SDK-owner review, release cutoff and
published dependency versions remain release-owner decisions tracked by FFL-3437.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
protected native release before shipping its integration. SDK-owner review, release cutoff and
published dependency versions remain release-owner decisions tracked by FFL-3437.
protected native release before shipping its integration.

This is a public repo. The Jira key and the internal release-process sentence don't belong in the README.

Protected nonempty targeting keys are SHA-256 hashes of the exact UTF-8 text, prefixed with
`sha256_`. Empty keys remain empty; missing keys and malformed Unicode are omitted without losing
the evaluation in both consent modes. Valid Unicode is preserved exactly, without normalization.
Android continues to omit `context.evaluation` in both modes. This setting does

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Android continues to omit `context.evaluation` in both modes. This setting does
Android continues to omit `context.evaluation` in both modes. The consent setting does


The Datadog Feature Flags SDK for Android allows you to evaluate feature flags and experiments in your Android application and automatically send flag evaluation data to Datadog for monitoring and analysis.

## Evaluation telemetry privacy

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section sits above "Getting started", so readers hit privacy internals before they learn how to install and use the SDK. Consider moving it below the usage sections. Also, the "Internal bridge replay" subsection (line 24) is aimed at SDK maintainers and the React Native bridge, not app developers. Consider moving it to the reliability/single-fit/flags/README.md handoff section, where it's already covered.

Comment thread AGENTS.md
`instrumented/integration`, `reliability/core-it` and `reliability/with-backend` contain instrumented tests that run on a connected device or emulator.

`reliability/single-fit/*` contains E2E tests that run on the JVM without a device, e.g. `./gradlew :reliability:single-fit:rum:testReleaseUnitTest` (this is what CI runs).
`reliability/single-fit/*` contains E2E tests that run on the JVM without a device, e.g. `./gradlew :reliability:single-fit:rum:testReleaseUnitTest` (this is what CI runs). The Flags module also has an ART privacy suite: `./gradlew :reliability:single-fit:flags:connectedReleaseAndroidTest` requires a connected device or emulator. Its README documents the exact-AAR option.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`reliability/single-fit/*` contains E2E tests that run on the JVM without a device, e.g. `./gradlew :reliability:single-fit:rum:testReleaseUnitTest` (this is what CI runs). The Flags module also has an ART privacy suite: `./gradlew :reliability:single-fit:flags:connectedReleaseAndroidTest` requires a connected device or emulator. Its README documents the exact-AAR option.
`reliability/single-fit/*` contains E2E tests that run on the JVM without a device, e.g. `./gradlew :reliability:single-fit:rum:testReleaseUnitTest` (this is what CI runs). The Flags module also has a privacy suite that runs on a connected device or emulator: `./gradlew :reliability:single-fit:flags:connectedReleaseAndroidTest`. Its README documents the exact-AAR option.

not change evaluation inputs, assignment requests, exposure events, or RUM integration.

Evaluation telemetry error messages contain stable error codes in both consent modes, with
`GENERAL` for unknown failures. `trackEvaluations(false)` disables this track; assignment `doLog`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`GENERAL` for unknown failures. `trackEvaluations(false)` disables this track; assignment `doLog`
`GENERAL` for unknown failures. `trackEvaluations(false)` disables this track. Assignment `doLog`

### Internal bridge replay

Native Android owns telemetry validation and hashing. The installed configuration precomputes
its telemetry identity, so repeated flag evaluations reuse it; retained evaluations keep that

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
its telemetry identity, so repeated flag evaluations reuse it; retained evaluations keep that
its telemetry identity, so repeated flag evaluations reuse it. Retained evaluations keep that

time, including errors without a flag, and pass null for an absent telemetry identity or an
empty string for an explicitly empty identity. The original four-argument overload derives
consent from the supplied flag and defaults to protected mode when it is unavailable.
Neither overload emits exposures or RUM annotations; use the separate hooks for those tracks.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Neither overload emits exposures or RUM annotations; use the separate hooks for those tracks.
Neither overload emits exposures or RUM annotations. Use the separate hooks for those tracks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants