Skip to content

ci: cover native Ruby UUIDs and synchronous capture in SDK harness - #268

Draft
dustinbyrne wants to merge 1 commit into
mainfrom
ci/sdk-harness-coverage-2026-09-07
Draft

ci: cover native Ruby UUIDs and synchronous capture in SDK harness#268
dustinbyrne wants to merge 1 commit into
mainfrom
ci/sdk-harness-coverage-2026-09-07

Conversation

@dustinbyrne

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Exercise Ruby's own UUID generation and synchronous capture transport in the SDK harness, and include the UTC timestamp case from harness 1.0.0 in recurring CI.

  • Observe SDK-generated UUIDs through the existing before_send hook; preserve explicitly supplied UUIDs.
  • Run separate async and sync_mode profiles with distinct reports. Keep real snapshot flag evaluation and the native omitted GeoIP default.
  • Keep assertion failures advisory while requiring both complete 47-case reports.

This changes only the adapter, its tests/documentation and CI. SDK public APIs/defaults and Rails are unchanged; no package release is needed.

💚 How did you test it?

  • bundle exec rspec: 696 examples, 0 failures, including 19 adapter examples.
  • bundle exec rubocop: 85 files, no offenses.
  • bundle exec rake public_api:check: passed.
  • Harness 1.0.0 against a locally built Ruby gem on Ruby 3.4.7:
Profile Selected Passed Failed
async 47 46 1
sync 47 45 2

Both profiles expose feature_flags.request_payload.disable_geoip_omitted_defaults_to_false: the SDK omits the field instead of sending literal false. Sync additionally fails capture.batch_format.multiple_events_batched_together because it sends each event immediately. Both profiles pass the UTC and native UUID definitions.

The full suites run without filtering. Sync's batch-retry identity definition passes on a retried single-event request; neither that result nor its multi-event batching failure certifies queue batching. V1, dedicated AI and non-gzip codecs remain unsupported. Existing diagnostic-state limitations are documented in the adapter README.

The inventory check accepts the real reports with these failures and rejects missing/zero-test reports. Docker was unavailable locally; container validation will come from this PR's CI.

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

🤖 Agent context

Autonomy: Human-driven (agent-assisted).

Implemented using a Pi coding agent with shell/file tools and GitHub CLI, followed by a fresh read-only agent review. Coverage uses existing public SDK calls and passive observation; wire assertions remain owned by the unchanged harness. Human review is still required.

@dustinbyrne dustinbyrne self-assigned this Sep 7, 2026
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

posthog-ruby-sync Compliance Report

Date: 2026-09-07T20:16:01.918235+00:00
Duration: 94250ms

⚠️ Some Tests Failed

45/47 tests passed, 2 failed


Capture Tests

⚠️ 29/30 tests passed, 1 failed

View Details
Test Status Duration
Format Validation.Event Has Required Fields 8ms
Format Validation.Event Has Uuid 10ms
Format Validation.Event Has Lib Properties 5ms
Format Validation.Distinct Id Is String 9ms
Format Validation.Token Is Present 6ms
Format Validation.Custom Properties Preserved 8ms
Format Validation.Event Has Timestamp 7ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc 6ms
Retry Behavior.Retries On 503 5279ms
Retry Behavior.Does Not Retry On 400 2008ms
Retry Behavior.Does Not Retry On 401 2011ms
Retry Behavior.Respects Retry After Header 8017ms
Retry Behavior.Implements Backoff 15382ms
Retry Behavior.Retries On 500 5157ms
Retry Behavior.Retries On 502 5161ms
Retry Behavior.Retries On 504 5116ms
Retry Behavior.Max Retries Respected 15624ms
Deduplication.Generates Unique Uuids 21ms
Deduplication.Preserves Uuid On Retry 5113ms
Deduplication.Preserves Uuid And Timestamp On Retry 10314ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 5163ms
Deduplication.No Duplicate Events In Batch 21ms
Deduplication.Different Events Have Different Uuids 9ms
Compression.Sends Gzip When Enabled 5ms
Batch Format.Uses Proper Batch Structure 7ms
Batch Format.Flush With No Events Sends Nothing 3ms
Batch Format.Multiple Events Batched Together 16ms
Error Handling.Does Not Retry On 403 2006ms
Error Handling.Does Not Retry On 413 2009ms
Error Handling.Retries On 408 5144ms

Failures

batch_format.multiple_events_batched_together

Expected 1 requests, got 5

Feature_Flags Tests

⚠️ 16/17 tests passed, 1 failed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 9ms
Request Payload.Flags Request Uses V2 Query Param 7ms
Request Payload.Flags Request Hits Flags Path Not Decide 10ms
Request Payload.Flags Request Omits Authorization Header 10ms
Request Payload.Token In Flags Body Matches Init 6ms
Request Payload.Groups Round Trip 6ms
Request Payload.Groups Default To Empty Object 6ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 7ms
Request Payload.Disable Geoip Omitted Defaults To False 6ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 6ms
Request Lifecycle.No Flags Request On Init Alone 2ms
Request Lifecycle.No Flags Request On Normal Capture 7ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 11ms
Request Lifecycle.Mock Response Value Is Returned To Caller 7ms
Retry Behavior.Retries Flags On 502 109ms
Retry Behavior.Retries Flags On 504 111ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 8ms

Failures

request_payload.disable_geoip_omitted_defaults_to_false

Field 'geoip_disable' not found in /flags request body at path 'geoip_disable'. Available keys: ['distinct_id', 'groups', 'person_properties', 'group_properties', 'flag_keys_to_evaluate', 'token']

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

posthog-ruby-async Compliance Report

Date: 2026-09-07T20:16:14.576961+00:00
Duration: 98528ms

⚠️ Some Tests Failed

46/47 tests passed, 1 failed


Capture Tests

30/30 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields 107ms
Format Validation.Event Has Uuid 107ms
Format Validation.Event Has Lib Properties 108ms
Format Validation.Distinct Id Is String 106ms
Format Validation.Token Is Present 107ms
Format Validation.Custom Properties Preserved 106ms
Format Validation.Event Has Timestamp 106ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc 10ms
Retry Behavior.Retries On 503 5308ms
Retry Behavior.Does Not Retry On 400 2109ms
Retry Behavior.Does Not Retry On 401 2109ms
Retry Behavior.Respects Retry After Header 8115ms
Retry Behavior.Implements Backoff 15622ms
Retry Behavior.Retries On 500 5212ms
Retry Behavior.Retries On 502 5213ms
Retry Behavior.Retries On 504 5212ms
Retry Behavior.Max Retries Respected 15722ms
Deduplication.Generates Unique Uuids 111ms
Deduplication.Preserves Uuid On Retry 5210ms
Deduplication.Preserves Uuid And Timestamp On Retry 10418ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 5216ms
Deduplication.No Duplicate Events In Batch 113ms
Deduplication.Different Events Have Different Uuids 107ms
Compression.Sends Gzip When Enabled 106ms
Batch Format.Uses Proper Batch Structure 106ms
Batch Format.Flush With No Events Sends Nothing 4ms
Batch Format.Multiple Events Batched Together 112ms
Error Handling.Does Not Retry On 403 2108ms
Error Handling.Does Not Retry On 413 2109ms
Error Handling.Retries On 408 5212ms

Feature_Flags Tests

⚠️ 16/17 tests passed, 1 failed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 108ms
Request Payload.Flags Request Uses V2 Query Param 107ms
Request Payload.Flags Request Hits Flags Path Not Decide 107ms
Request Payload.Flags Request Omits Authorization Header 108ms
Request Payload.Token In Flags Body Matches Init 108ms
Request Payload.Groups Round Trip 109ms
Request Payload.Groups Default To Empty Object 108ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 106ms
Request Payload.Disable Geoip Omitted Defaults To False 106ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 107ms
Request Lifecycle.No Flags Request On Init Alone 3ms
Request Lifecycle.No Flags Request On Normal Capture 105ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 109ms
Request Lifecycle.Mock Response Value Is Returned To Caller 106ms
Retry Behavior.Retries Flags On 502 209ms
Retry Behavior.Retries Flags On 504 259ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 108ms

Failures

request_payload.disable_geoip_omitted_defaults_to_false

Field 'geoip_disable' not found in /flags request body at path 'geoip_disable'. Available keys: ['distinct_id', 'groups', 'person_properties', 'group_properties', 'flag_keys_to_evaluate', 'token']

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant