Skip to content

fix(win32): drop modifiers from IME-committed text to avoid CSI-u garbling#2

Open
nanasess wants to merge 1 commit into
Codavo:mainfrom
nanasess:fix/win32-ime-commit-csiu
Open

fix(win32): drop modifiers from IME-committed text to avoid CSI-u garbling#2
nanasess wants to merge 1 commit into
Codavo:mainfrom
nanasess:fix/win32-ime-commit-csiu

Conversation

@nanasess

Copy link
Copy Markdown

Problem

With a Japanese IME (reproduced with CorvusSKK), text committed from a conversion arrives in the WSL shell as escape-sequence garbage instead of the actual characters:

typed expected actual
ESC[30456;5u
今日 今日 ESC[20170;5uESC[26085;5u

Direct kana input (no conversion) and pasting are unaffected.

Cause

SKK-style IMEs confirm a conversion with Ctrl+J, so Ctrl is still physically down at the moment the committed characters are delivered via WM_CHAR.

handleTextInput (src/apprt/win32/App.zig) reads the live modifier state with getModifiers() and puts it on KeyEvent.mods unchanged. The core's legacy encoder (src/input/key_encode.zig) encodes any Ctrl + single codepoint as a fixterms CSI <codepoint>;<mods>u sequence — ;5u being Ctrl — so the committed codepoint is emitted as CSI-u instead of raw UTF-8.

This is consistent with the observed behaviour: confirming with Enter (Ctrl not held) never reproduces the problem.

Fix

Non-ASCII output from WM_CHAR/WM_SYSCHAR can only come from one of two places:

  1. Composition — an IME commit (Ctrl held) or a dead key / AltGr (Ctrl+RightAlt). The modifiers were consumed to produce the character, not to trigger a shortcut.
  2. An intentional Alt + <non-ASCII key> shortcut, which should still be encoded as meta / ESC-prefixed.

So for codepoint >= 0x80:

  • Alt held without Ctrl → keep the modifiers (case 2 preserved).
  • every other case, i.e. Ctrl held (SKK commit, AltGr) → clear the modifiers so the core emits the raw UTF-8 text.

Control characters (< 0x20) are already filtered out earlier in the function, and the existing AltGr consumed_mods handling right above is left intact for the ASCII path.

Testing

Built for Windows and run against WSL:

  • CorvusSKK conversion + Ctrl+J commit → correct characters in zsh, no CSI-u.
  • Enter commit, direct kana input, paste → unchanged.
  • Alt + <non-ASCII> and ordinary Ctrl shortcuts (Ctrl+C, Ctrl+D, ...) → unchanged.

This has been running as a patch in my fork (nanasess#8) in daily use for a couple of weeks without regressions before opening this PR.

Notes / scope

This is a deliberately minimal fix on the WM_CHAR path. A more thorough approach would handle WM_IME_CHAR / WM_IME_COMPOSITION explicitly (including preedit rendering), which is a much larger change — happy to go that route instead if you'd prefer.

AI disclosure

Per AI_POLICY.md: this change was developed with Claude Code assisting in the investigation, and the code comment and this PR description were drafted with its help and edited by me. I diagnosed and verified the behaviour on real hardware, and I understand and stand behind the change.

…bling

Japanese IMEs such as CorvusSKK confirm a conversion with Ctrl+J, so Ctrl
is still physically held at the moment the committed characters arrive via
WM_CHAR. handleTextInput passed those live modifiers straight through to
the core, and the legacy encoder turns any "Ctrl + single codepoint" into a
fixterms "CSI <codepoint>;<mods>u" sequence (see input/key_encode.zig).
Committed text therefore reached the shell as garbage, e.g. 相 became
ESC[30456;5u and 今日 became ESC[20170;5uESC[26085;5u.

Non-ASCII WM_CHAR/WM_SYSCHAR output is either composition (Ctrl held for an
IME commit, or AltGr = Ctrl+RightAlt) or an intentional "Alt + <non-ASCII>"
shortcut. Only the former should have its modifiers dropped, so keep Alt
when it is held without Ctrl (preserving the meta/ESC-prefix encoding) and
clear the modifiers for every Ctrl-held case so the raw UTF-8 is emitted.
Control characters (< 0x20) are already filtered earlier in the function.

Direct kana input and paste do not go through this encoding path and are
unaffected. Confirming with Enter (no Ctrl held) never reproduced the bug,
which matches the diagnosis.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant