feature(live): webhook / external push for chat channel messages#3445
Open
ArnavBallinCode wants to merge 11 commits intofossasia:devfrom
Open
feature(live): webhook / external push for chat channel messages#3445ArnavBallinCode wants to merge 11 commits intofossasia:devfrom
ArnavBallinCode wants to merge 11 commits intofossasia:devfrom
Conversation
- New Celery task (send_chat_webhook) for HMAC-SHA256 signed HTTP POST - Dispatch webhook after group_send in ChatModule.send() and .react() - Challenge verification on webhook URL registration in room config patch - Webhook config stored in module_config JSONField (no migration needed) - HTTPS-only URL validation, fire-and-forget delivery
- Relax HTTPS requirement when DEBUG=True for local development - Add tools/test_webhook_receiver.py for manual testing
Contributor
There was a problem hiding this comment.
Sorry @ArnavBallinCode, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
Show webhook config inputs under the Chat toggle in the room sidebar addons panel. Fields are only visible when chat is enabled.
- Strip webhook_hmac_secret from frontend room configs - Validate secret whenever webhook_url is set, not just on URL change - Use urlparse for URL validation, block private networks in prod - Use sync_to_async instead of database_sync_to_async for HTTP - Use event_data sender instead of str(sender.id) to avoid 'None' - Skip webhook for unsupported message types (files, deleted, call) - Include reaction emoji, action, and reactions snapshot in meta
Look up CentralAuth gu_id (SocialAccount.uid) and gu_name (User.wikimedia_username) at webhook dispatch time. For video room users created via JWT token auth, trace back to the main Django user by matching token_id to sha256(email)[:7] of MediaWiki-linked users.
|
on the accepting side, we're live: https://chatstream-moderate.toolforge.org/ . Let me know if you like access for testing. Github updated with documentation. My AI assistant suggested you double check Is there a 'channel language' that you could add? Might be helpful. I added some other fields that I thought might be useful, and updated the specs to this PR as much as possible. |
- Fix sender_id for reactions: always use acting user (self.consumer.user.id) instead of event_data['sender'] which is the original message author - Reuse shared encode_email() from eventyay_common.utils instead of duplicating the implementation - Stricter SSRF: use not ip.is_global instead of is_private to also catch loopback, link-local, and reserved ranges via DNS resolution - Fix misleading error message: 'http or https' instead of 'HTTPS only' for the scheme validation that also accepts http - Guard event.py webhook_hmac_secret pop against missing/non-dict config - Sanitize webhook URL in log output by stripping query string and fragment - Remove unused sys import from test_webhook_receiver.py
Avoid querying the DB on every webhook dispatch. Results are cached in Django's cache (Redis) with a 1-hour TTL. CentralAuth renames are rare so stale data is acceptable, as noted by @lgelauff.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3443
Summary
Adds per-room webhook configuration so chat messages and emoji reactions are POSTed to external endpoints with HMAC-SHA256 authentication. This enables external moderation tools (e.g. chatstream-moderate) to receive and process live chat events in real time.
Webhook Payload Schema
{ "message_id": "int — Eventyay event_id", "channel": "uuid — Eventyay chat channel identifier", "timestamp": "ISO 8601 — message creation time", "screen_name": "string — chat display name", "sender_id": "string — Eventyay internal user ID (opaque, anonymized)", "centralauth_id": "int | null — Wikimedia CentralAuth gu_id (persistent, globally unique)", "centralauth_username": "string | null — Wikimedia CentralAuth gu_name (global username)", "message": "string — message body text", "message_type": "\"text\" | \"emoji\"", "profile_img": "null (reserved)", "user_language": "string | null — user's locale (BCP 47)", "meta": "object — type-specific extras (see below)" }metaby message typemessage_typemetafieldstext{}emoji{ "target_message_id", "reaction", "action": "add"|"remove", "reactions": {} }CentralAuth field mapping
centralauth_idSocialAccount.uid(allauth, provider=mediawiki) →intgu_id— persistent, globally unique numeric IDcentralauth_usernameUser.wikimedia_username(CharField, set at OAuth login)gu_name— global username (can be renamed)sender_idIdentity resolution for video room users: Video room users are created separately via JWT token auth. Their
token_idissha256(main_user.email)[:7], which allows tracing back to the main Django user'sSocialAccountto resolve CentralAuth fields. Bothcentralauth_idandcentralauth_usernamewill benullfor users who did not log in via Wikimedia OAuth.OAuth endpoint: All three OAuth endpoints hit
meta.wikimedia.org/w/rest.php/oauth2/(authorize, access_token, profile). The OIDCsubclaim from meta.wikimedia.org is the CentralAuth global user ID (gu_id), not a local wiki ID.Authentication
X-Eventyay-Signature: sha256=<hex_digest>HMAC-SHA256(raw_request_body, shared_secret)GET ?challenge=<token>→ respond{"challenge": "<token>"}Security
urlparse, private network blocking viasocket.getaddrinfo+ipaddress.ip_address. Private networks are only allowed inDEBUGmode.webhook_hmac_secretis stripped fromget_room_config()so it's never sent to WebSocket clients.textandemojitypes trigger webhooks;files,deleted,callare skipped.Screenshots