fix(win32): drop modifiers from IME-committed text to avoid CSI-u garbling#2
Open
nanasess wants to merge 1 commit into
Open
fix(win32): drop modifiers from IME-committed text to avoid CSI-u garbling#2nanasess wants to merge 1 commit into
nanasess wants to merge 1 commit into
Conversation
…bling Japanese IMEs such as CorvusSKK confirm a conversion with Ctrl+J, so Ctrl is still physically held at the moment the committed characters arrive via WM_CHAR. handleTextInput passed those live modifiers straight through to the core, and the legacy encoder turns any "Ctrl + single codepoint" into a fixterms "CSI <codepoint>;<mods>u" sequence (see input/key_encode.zig). Committed text therefore reached the shell as garbage, e.g. 相 became ESC[30456;5u and 今日 became ESC[20170;5uESC[26085;5u. Non-ASCII WM_CHAR/WM_SYSCHAR output is either composition (Ctrl held for an IME commit, or AltGr = Ctrl+RightAlt) or an intentional "Alt + <non-ASCII>" shortcut. Only the former should have its modifiers dropped, so keep Alt when it is held without Ctrl (preserving the meta/ESC-prefix encoding) and clear the modifiers for every Ctrl-held case so the raw UTF-8 is emitted. Control characters (< 0x20) are already filtered earlier in the function. Direct kana input and paste do not go through this encoding path and are unaffected. Confirming with Enter (no Ctrl held) never reproduced the bug, which matches the diagnosis.
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
With a Japanese IME (reproduced with CorvusSKK), text committed from a conversion arrives in the WSL shell as escape-sequence garbage instead of the actual characters:
相ESC[30456;5u今日ESC[20170;5uESC[26085;5uDirect kana input (no conversion) and pasting are unaffected.
Cause
SKK-style IMEs confirm a conversion with Ctrl+J, so Ctrl is still physically down at the moment the committed characters are delivered via
WM_CHAR.handleTextInput(src/apprt/win32/App.zig) reads the live modifier state withgetModifiers()and puts it onKeyEvent.modsunchanged. The core's legacy encoder (src/input/key_encode.zig) encodes any Ctrl + single codepoint as a fixtermsCSI <codepoint>;<mods>usequence —;5ubeing Ctrl — so the committed codepoint is emitted as CSI-u instead of raw UTF-8.This is consistent with the observed behaviour: confirming with Enter (Ctrl not held) never reproduces the problem.
Fix
Non-ASCII output from
WM_CHAR/WM_SYSCHARcan only come from one of two places:Ctrl+RightAlt). The modifiers were consumed to produce the character, not to trigger a shortcut.Alt + <non-ASCII key>shortcut, which should still be encoded as meta / ESC-prefixed.So for
codepoint >= 0x80:Altheld withoutCtrl→ keep the modifiers (case 2 preserved).Ctrlheld (SKK commit, AltGr) → clear the modifiers so the core emits the raw UTF-8 text.Control characters (
< 0x20) are already filtered out earlier in the function, and the existing AltGrconsumed_modshandling right above is left intact for the ASCII path.Testing
Built for Windows and run against WSL:
Ctrl+Jcommit → correct characters inzsh, no CSI-u.Alt + <non-ASCII>and ordinary Ctrl shortcuts (Ctrl+C,Ctrl+D, ...) → unchanged.This has been running as a patch in my fork (nanasess#8) in daily use for a couple of weeks without regressions before opening this PR.
Notes / scope
This is a deliberately minimal fix on the
WM_CHARpath. A more thorough approach would handleWM_IME_CHAR/WM_IME_COMPOSITIONexplicitly (including preedit rendering), which is a much larger change — happy to go that route instead if you'd prefer.AI disclosure
Per AI_POLICY.md: this change was developed with Claude Code assisting in the investigation, and the code comment and this PR description were drafted with its help and edited by me. I diagnosed and verified the behaviour on real hardware, and I understand and stand behind the change.