MemCode recall plugin for returning LiveKit voice callers

[vivekgupta-memcode]

Hi LiveKit Agents maintainers. I’m Vivek Gupta, Founder & CEO of MemCode (https://memcode.in). LiveKit’s realtime participant model and plugin ecosystem seem like a good place to let a returning caller carry a few approved preferences across calls.

I’d propose a narrow Python plugin or recipe: resolve the authenticated caller at session setup, fetch a small relevant memory set before the first response, and write only user-confirmed facts after a completed turn. It would keep audio, raw transcripts, and transient room state out of long-term storage. We could measure the added setup latency and make recall optional so realtime behavior stays predictable.

Would you prefer a community plugin, an example agent, or a first-party memory integration? I can contribute the implementation and a short guide if you indicate the right path.

cc @chenghao-mou for your thoughts on this integration.

[Rehansanjay]

On the three-way question: of the three, "community plugin" is the one that does not really exist for this.

CONTRIBUTING.md scopes that invitation specifically to providers — "if there is a TTS/STT/LLM provider you use that isn't on our plugins list". The livekit-plugins/ tree matches that: model and media providers, plus a few framework pieces (silero, turn-detector, krisp, the langchain adapter). None of them is a data or storage backend, so there is no plugin interface a memory package would implement. For a new capability the same file says to open an issue first and agree the scope, which is what this is.

Where the code would actually go, if it helps to be concrete:

• Agent.on_enter() (voice/agent.py:336) runs when the agent is entered, before the first reply, which is the hook for resolving the caller and fetching the recall set.
• Agent.on_user_turn_completed(turn_ctx, new_message) (voice/agent.py:344) is called after the user stops speaking and before the LLM is asked. Its own docstring calls it the opportunity to update the chat context or edit the new message, so both the per-turn read and the write-back of confirmed facts fit there.
• Agent.update_chat_ctx() (voice/agent.py:250) covers mutating the context outside a turn.

That makes this an Agent subclass with a service behind it rather than a plugin, which already has a precedent in the repo: examples/frontdesk/ is an agent plus an external stateful service in calendar_api.py, with scenarios.yaml, simulation.py and test_agent.py alongside it. Same shape as what you are describing, including a way to demonstrate it.

One note on the latency point. on_enter runs ahead of the first response, so a blocking fetch there lands directly on time-to-first-word rather than being absorbed anywhere. Kicking it off in on_enter and awaiting it in the first on_user_turn_completed keeps it off the greeting, at the cost of the first turn not having recall.