Skip to content

[Suggestion] Temporal.io durable batch audio transcription workflow with retries and fan-out (Python) #322

Description

@deepgram-robot

What to build

A Temporal workflow that orchestrates batch audio transcription with durable execution guarantees. The workflow accepts a list of audio files, fans out to parallel transcription activities with configurable concurrency, applies Audio Intelligence analysis, aggregates results, and handles retries automatically — surviving process restarts and infrastructure failures.

Why this matters

Enterprise audio processing pipelines need durable execution: if a transcription job fails halfway through 10,000 files, it should resume from where it stopped, not restart. Temporal is the leading workflow orchestration platform for this pattern, used by Netflix, Uber, and Stripe. Developers building production transcription pipelines need a reference implementation that combines Temporal's durability guarantees with Deepgram's transcription and intelligence APIs. No existing Deepgram example covers workflow orchestration.

Suggested scope

  • Language: Python
  • Deepgram APIs: STT pre-recorded, Audio Intelligence (sentiment, topics, summary)
  • Framework: Temporal Python SDK (temporalio)
  • Key patterns: Workflow definition, parallel activities with concurrency control, retry policies, progress reporting
  • Complexity: Moderate to advanced

Acceptance criteria

  • Runnable with minimal setup (clone, add API key, start Temporal dev server, run)
  • README explains the pattern clearly
  • Uses current SDK version
  • Workflow survives worker restart and resumes from last checkpoint
  • Supports configurable parallelism for concurrent transcription
  • Includes retry policies with exponential backoff
  • Reports progress during batch processing

Raised by the DX intelligence system.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions