Add guide for migrating Celery task queues to Standalone Activities - #5135
Add guide for migrating Celery task queues to Standalone Activities#5135brianmacdonald-temporal wants to merge 7 commits into
Conversation
Add a Python guide that walks through converting a Celery task into a Temporal Standalone Activity: running an Activity-only Worker, replacing .delay() and .get() calls, migrating max_retries to a Retry Policy, and using list_activities() in place of Flower. Link it from the Guides sidebar and add a card to the Guides landing page grid under a new Migration tag. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📖 Docs PR preview links
|
There was a problem hiding this comment.
Pull request overview
Adds a Python migration guide for converting Celery tasks to Temporal Standalone Activities.
Changes:
- Documents task conversion, invocation, retries, and monitoring.
- Adds the guide to sidebar navigation.
- Adds a migration card and filter tag to the Guides grid.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 10 comments.
| File | Description |
|---|---|
docs/guides/celery-to-standalone-activity.mdx |
Adds the migration guide and Python examples. |
sidebars.js |
Adds the guide to navigation. |
src/components/GuidesGrid/guides-data.json |
Adds the guide’s landing-page card. |
Suppressed comments (7)
docs/guides/celery-to-standalone-activity.mdx:369
maximum_attempts=5does not mirror Celery'smax_retries=5: Temporal includes the initial attempt, whereas Celery counts five retries after it. Update this explanation and the policy above to six total attempts; also account for the Celery task's fixed 10-second retry delay.
Here, `maximum_attempts=5` mirrors Celery's `max_retries`, and `maximum_interval` caps the backoff between attempts. The `non_retryable_error_types` list names errors that should fail immediately without retrying — the equivalent of *not* calling `self.retry()` for a permanent failure. To raise such an error from the Activity, use `ApplicationError` with `non_retryable=True` in `my_activity.py`:
docs/guides/celery-to-standalone-activity.mdx:103
- Both occurrences must use the proper product name “Temporal Service” (
readme/STYLE.md:24-31), not “Temporal service.” The table at line 37 needs the same correction.
Confirm the printed version is at least 1.7.0. With the tools installed, you can start a local Temporal service.
## Step 3 — Starting the Temporal Development Server
In Celery, work flows through a broker such as Redis. In Temporal, work flows through the Temporal service, which also stores each Activity's durable state. In this step, you will start a local development server that stands in for that service.
docs/guides/celery-to-standalone-activity.mdx:91
- Use the relative destination
/cli/setup-clifor this internal link. Absolutedocs.temporal.iolinks fail the CI-scopedTemporal.RelativeLinksrule; other occurrences remain at lines 25, 431, 437, 445, 447, and 448.
If you aren't using Homebrew, download the binary for your platform from the [Temporal CLI install guide](https://docs.temporal.io/cli/setup-cli) and add it to your `PATH`.
docs/guides/celery-to-standalone-activity.mdx:431
- Use the relative destination
/list-filterfor this internal link. Absolutedocs.temporal.iolinks fail the CI-scopedTemporal.RelativeLinksrule; other occurrences remain at lines 25, 91, 437, 445, 447, and 448.
The `query` uses the same [List Filter](https://docs.temporal.io/list-filter) syntax as Workflow visibility, so you can filter by attributes such as `ActivityType` and `Status` — for example, `"ActivityType = 'send_welcome_email' AND Status = 'Running'"`. These calls return only Standalone Activities; Activities running inside Workflows are excluded. The Temporal CLI offers the same views with `temporal activity list` and `temporal activity count`.
docs/guides/celery-to-standalone-activity.mdx:437
- Use the relative destination
/develop/pythonfor this internal link. Absolutedocs.temporal.iolinks fail the CI-scopedTemporal.RelativeLinksrule; other occurrences remain at lines 25, 91, 431, 445, 447, and 448.
If your Celery app uses [Canvas](https://docs.celeryq.dev/en/stable/userguide/canvas.html) primitives — chaining tasks so one result feeds the next (`chain`), fanning work out in parallel (`group`), or running a callback after a group finishes (`chord`) — that coordination logic needs somewhere to live durably. A Standalone Activity cannot call another Activity or guarantee progress across several steps. For those pipelines, wrap your Activities in a Temporal **Workflow**, where sequencing is ordinary `await` statements and parallelism is `asyncio.gather`. See the [Temporal Python documentation](https://docs.temporal.io/develop/python) for building Workflows.
docs/guides/celery-to-standalone-activity.mdx:448
- These three internal links must use relative destinations (
/develop/python/activities/standalone-activities,/develop/python/activities/standalone-activities-quickstart, and/develop/python/activities/timeouts). Absolutedocs.temporal.iolinks fail the CI-scopedTemporal.RelativeLinksrule; lines 25, 91, 431, and 437 have the same issue.
Because Standalone Activities are in Public Preview, review the [Standalone Activities feature guide](https://docs.temporal.io/develop/python/activities/standalone-activities) for the latest API details before relying on them in production. Useful next topics include:
- The [Standalone Activities Quickstart](https://docs.temporal.io/develop/python/activities/standalone-activities-quickstart) for the runnable reference sample.
- [Activity timeouts](https://docs.temporal.io/develop/python/activities/timeouts) for tuning `start_to_close` and related limits.
docs/guides/celery-to-standalone-activity.mdx:321
- Temporal core identifiers use
Id, so change these references to “Activity Id” and “Run Id” (readme/STYLE.md:50-58). The generic “user ID” at line 276 must likewise be written as “user identifier.”
The pattern mirrors Celery precisely. `start_activity` corresponds to `.delay()` and returns a handle immediately, the way Celery returns an `AsyncResult`. Calling `handle.result()` corresponds to `AsyncResult.get()`. If you need to reconnect to an Activity from a different process — for example, a web request started it and a later request checks on it — recreate the handle from the Activity's ID and run ID (the run ID is available on the handle returned by `start_activity`):
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
MasonEgger
left a comment
There was a problem hiding this comment.
One other thing that might be worth mentioning is using start delay on a SAA to delay the invocation for a set amount of time. This is basically the equivalent of if someone wants to have a workflow that immediately fires a timer and then does an activity. I will also gladly accept if you call that scope Creek for the purpose of this article. It's just a neat little tool.
✅ Docs build passed |
Changed in response to technical feedback.
What does this PR do?
Adds a Python guide, Migrate a Celery task queue to a Temporal Standalone Activity, that walks through converting a Celery task into a Standalone Activity:
delay(),AsyncResult.get(),max_retries, Flower) onto their Temporal equivalents.delay()and.get()withclient.start_activity()andclient.execute_activity()max_retries/self.retry()to a Retry Policyclient.list_activities()andclient.count_activities()in place of FlowerAlso links the page from the Guides sidebar and adds a card to the Guides landing page grid under a new
Migrationtag.Notes to reviewers
Opened as a draft: the guide documents Standalone Activities, which are
publicPreviewinsrc/constants/featureReleaseTypes.js. Happy to mark it ready once the docs team is clear to merge preview coverage for this feature.yarn buildpasses; the page and its landing page card both appear in the build output.Two follow-ups intentionally left out of this PR:
docs/guides/loyalty-points-revised.mdxis a separate untracked draft in my working tree and is not included here. It's currently an unlinked orphan and needs its own PR.author: n/ain its frontmatter, which no other guide uses. Let me know if you'd like it dropped.🤖 Generated with Claude Code
┆Attachments: EDU-6982 Add guide for migrating Celery task queues to Standalone Activities