feat: interview language picker, switchable mid-interview - #108
Open
alpha5611331 wants to merge 20 commits into
Open
feat: interview language picker, switchable mid-interview#108alpha5611331 wants to merge 20 commits into
alpha5611331 wants to merge 20 commits into
Conversation
It had one member and was read by nothing. Six now, mirrored across the two processes the way SuggestionMode is, plus the display metadata the picker needs: endonym first, since someone whose interview is in the wrong language recognises "Deutsch" before "German". Six because that is what AssemblyAI's universal-streaming-multilingual model transcribes. Offering one the ASR cannot hear would not degrade gracefully, it would answer a question that was never asked. configStore.getConfig resolves the value on the way out rather than on the way in. The disk holds whatever some build wrote, and every consumer reads through getConfig, so that is the one place an unknown code can be stopped before it reaches the ASR URL and three request bodies. Refs #24 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Live, action and summarize, off one field on the shared LLMRequest base so the three cannot drift. Optional on the wire because the backend defaults it, which keeps the client working against a deployment that predates it. Each service already reads the config store as it builds its request, so this is also what makes the setting changeable mid-interview for free: the next suggestion follows without anything being reconnected. The exported report goes with them. A Spanish interview summarised in English is a document the candidate cannot hand to anyone who was in it. Refs #24 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…and switch them mid-session ?language= on the streaming URL, and English sends no parameter at all rather than language=en - the backend treats an absent language as English and builds the URL it has always built, so a session that never touches the picker produces exactly the traffic it produced before. The language is a connection parameter, so changing it mid-interview means tearing both sockets down and re-opening them. The URL is therefore rebuilt per attempt rather than captured, and two guards keep that from leaving two sockets on one channel, one of them orphaned and still relaying audio into a dead session: - onclose ignores a close from a socket that is no longer this.ws. That is the tail of a replacement, not a disconnect. - setLanguage sets a switching flag so the ordinary backoff reconnect does not fire for the close it caused itself; it reconnects immediately instead of waiting out WS_RETRY_BASE_DELAY_MS. The in-flight utterance is still reported as disconnected, because a switch orphans it exactly the way a dropped connection does. Refs #24 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sits with Audio and Model, because it is an input as much as an output: it picks the speech model before it picks the answer's language. Unlike those two it stays live while the assistant runs. An interview that switches language is the case this control exists for and not one the candidate can prepare for by restarting, so it locks only through the transient Starting and Stopping states. The switch is not instant and the button says so. Suggestions follow at once; the ASR reconnects, so the trigger spins and the menu warns that the sentence being spoken may be cut short - a two-second hole in the transcript is alarming if it arrives unannounced mid-question. The setting is persisted before the reconnect and never rolled back on failure: reverting it would leave the user with no route to the language they picked, while leaving it set means stop-and-start recovers. The trigger carries the code next to the icon. A globe alone is only useful to someone who already knows what it is set to, which is the one question this control has to answer at a glance. Refs #24 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The failure this guards is silent: an English speech model does not report a language it cannot handle, it returns confident English words for speech that was never English. So what matters is that a chosen language survives the round trip, and that one this build does not know dies at getConfig rather than on the ASR URL. Refs #24 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
|
…over a failed one Three defects in the switch path, all of them silent. channelDisconnected was left to onclose, which the new "ignore a close from a socket that is no longer this.ws" guard correctly swallows: `new WebSocket` assigns this.ws synchronously, so the old socket's close always arrives after its replacement exists. The partial in flight would have stayed open and gated live suggestions for the rest of the session. setLanguage reports it directly now. A failed switch left the channel silent until the assistant was stopped and started. Five failed attempts is a network or provider problem, not a permanent one, so it hands the channel back to the ordinary backoff loop and the toast says it is still retrying rather than telling the user to restart. The no-op check keyed on `active`, which start() sets only after its first connect returns. In that window a socket exists on the old language and would never have been reconnected. Keyed on the socket instead. Also: allSettled across the two channels, since Promise.all leaves the second channel's rejection unhandled, and a swallow of the stop-mid-switch case, which is a shutdown rather than a failure. Refs #24 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The default-device pick ran in the render body and called updateConfig, which writes to the Zustand store synchronously before its IPC call resolves. That re-enters React mid-commit, and the failure mode is worse than the warning: updateConfig rolls the optimistic value back when the write fails, so the condition that triggered the pick is true again on the very next render. A store that cannot be written produced one attempt per frame, each one an unhandled rejection, for as long as the panel stayed mounted. The rejection is handled now, and a latch keeps the pick to once per mount, so clearing the selection or unplugging the chosen device mid-session does not silently reassign it underneath the user. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An empty device list means two different things - enumerateDevices() has not answered yet, and this machine has none - and the control panel treated both as the second. useMediaDevices starts empty, so every launch put a destructive badge on the mic button for the first frames, and a Start pressed quickly was refused with a message naming a device that was there all along. An unset audioInputDeviceName was a third state read the same way, so a fresh install was told its microphone `""` could not be found while AudioGroup was in the middle of choosing one. useMediaDevices now reports `ready` alongside the list, and the two conditions are separated: a machine with no audio input at all gets a message that says so, rather than one naming a device the user never chose. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every control on the bar is disabled while runningState is Stopping, Stop included. stopAssistant set Idle only on its success path, so any throw on the way out froze the app mid-teardown with no reachable control and no way back short of restarting it - during an interview, which is the only time it runs. Idle is now set in a finally. The teardown itself used Promise.all over four services, which rejects on the first failure and abandons the other three: one service that refused to close left the rest running. allSettled closes all four whatever happens to any of them. A partial failure is now a toast rather than a throw, because the session is over either way and the store's `error` field is rendered nowhere. doStart no longer calls stopAssistant after a failed start. startAssistant's own catch has already torn both services down and returned to Idle, so the second call only walked the button through three seconds of "Stopping" for a session that never started - and it sat in the catch block, so its own failure became an unhandled rejection. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The recompute effect listed the whole appState object in its dependencies alongside the runningState field it actually cares about. appState is a fresh object on every push from main, which during an interview is every ASR partial and every streamed suggestion token - several times a second. Each run does three getBoundingClientRect() calls, so this forced a synchronous layout on the renderer for every token that arrived, while nothing being measured had changed size at all. Every other input to the layout has an effect of its own. Two timers are cleared on unmount while here: the deferred first measurement, which could otherwise measure a tree being torn down by a navigation, and the login redirect. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both suggestion panels carried their own copy, and neither honoured its own bound: three characters were reserved for the separator and thirteen were spliced in, so every truncated question came back ten characters over the limit the caller asked for. Harmless on one line, wrong for anything that reuses the helper to size something. Now one helper in lib/suggestions.ts, counted against the budget, splitting what is left so the tail survives - a question's actual ask is usually at the end of it, which is the whole reason this truncates the middle rather than the end. Degenerate budgets return a string that still fits rather than a longer one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Eleven inputs across login, signup and the reset wizard had a label with no htmlFor, so clicking the label did nothing and a screen reader announced the field with no name at all. Remember-me was the one control already wired correctly, which is what made the omission easy to miss. Fields now carry autoComplete as well, so a password manager can fill them. The password reveal button was an icon with no accessible name - announced as "button", on a field whose value is deliberately unreadable - and said nothing about which of its two states it was in. It also sat in the tab order between the password field and submit, where it is a stop nobody wants. Form errors used a hardcoded text-red-600 in nine places, inconsistent with the text-destructive token used everywhere else and low-contrast against the dark theme's card. None carried role=alert, so submitting a login and being told "Incorrect email or password" announced nothing: the message is the only thing that changes on screen. Radix Select triggers are buttons, so htmlFor does not reach them; the LLM provider, model and microphone pickers are associated by aria-labelledby. The microphone picker's config write was also unhandled. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Profile and Context are capped at 128,000 characters by maxLength, which truncates a paste silently. A CV or a job description pasted over the limit arrives shorter than the one the user copied and nothing on screen says so - and these two fields are the whole basis of every suggestion the app writes. A counter appears at 90% of the budget and turns destructive at the cap, where it says what happened rather than counting. Hidden below the threshold because a counter over an empty box is noise and most sessions never approach it. The three fields are also label-associated, and the values are trimmed on the way out: the Save button is gated on the trimmed name being non-empty, so a name of pure whitespace could never be saved, but a name with a trailing space could - and it is the string the prompts address the candidate by. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three things that read as tidy code until they are not: that runningState has to reach a terminal value on every path, because Starting and Stopping disable the control that would recover it; that the four teardown calls are settled rather than raced, because one refusing to close must not leave the other three running; and that an empty device list is two different states, only one of which is a missing microphone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The export button is live whenever the assistant is idle, which includes every launch before the first session. Pressed then, it sent an empty transcript to the summarize endpoint - a billed model call whose only possible output is invented - and wrote the result into a document the candidate is told is a record of their interview. Guarded in the service, which is what stops the request, and again in the panel, which is what produces the sentence the user reads: an error raised out of an ipcMain handler reaches the renderer wrapped in Electron's "Error invoking remote method" prefix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The transcript toggle, Clear, Export, the three buttons on the export toast, the microphone button and the payment page's back arrow were all icon-only with no accessible name. A Radix tooltip supplies a description, not a name, so each was announced as "button" with a description appended. The microphone button carries one more thing. Its warning state is a badge drawn over the corner - colour and position and nothing else - so it is folded into the button's name, and the badge itself is hidden from the tree rather than announced as a stray "!". That warning is the only condition this control reports, and it was reachable only by looking at it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The backend loop retried every second for as long as the app stayed open, with no ceiling. A laptop left overnight on a dropped connection made tens of thousands of failing requests, and at the end of a real outage every installed client came back at that same rate at the same moment. Both loops now back off geometrically to a 30 second ceiling and reset the moment a check succeeds, so a single blip does not leave the app checking slowly for the rest of the session and recovery is still noticed inside the window the reconnect notice is waiting on. The signed-out branch of the client loop keeps its one-second cadence deliberately: it makes no request, so it costs a timer wake-up rather than traffic, and it is what decides how soon after a sign-in the credits and role reach the UI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nothing stopped the renderer navigating away or opening a window, and the panels render Markdown that came from a language model - remark-gfm autolinks bare URLs, so an anchor here is not necessarily one a person wrote. Two routes followed from that, both silent. A target="_blank" anchor asks Electron for a new window, and with no setWindowOpenHandler installed the default is to make one: a chromeless BrowserWindow, no address bar, showing a page the user did not choose. And an anchor with no target navigates the frame it is in, which is the app - preload runs on whatever document loads next, so a remote page would inherit window.electronAPI and with it the session token through config.get() and the candidate's CV through account.get(). Every new window is now denied and a web URL handed to the real browser instead, which is what the user expected from a link anyway; will-navigate pins the window to the app's own document. external:open shares the same openExternally(), which allows http, https and mailto only. It called shell.openExternal on any string, and openExternal delegates to the OS protocol handler - file: launches whatever the path points at, and a registered custom scheme runs whatever claimed it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
createWindow() runs again when the single-instance lock recovers a destroyed window, and web-contents-created is an app-level event, so the second call stacked a duplicate will-navigate listener onto every web contents from then on. Harmless in effect - both listeners call preventDefault - but the comment claimed the window is created only once per process, and it is not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #24.
Languagehad one member and was read by nothing. This makes it a real setting: a picker on the control bar that decides which speech model transcribes the call, what language suggestions are written in, and the language of the exported report - changeable mid-interview, not only before Start.Pairs with PowerInterviewAI/backend#58. Neither side needs the other to ship: the backend defaults the field, so this client works against a deployment that predates it (suggestions stay English, and so does the transcription).
What changed
src/main/types/language.ts,src/renderer/types/language.tsSuggestionModeis, plus display metadatasrc/main/store/config.store.tslanguageis typed and resolved on readsrc/main/services/suggestion-{live,action}.service.ts,tools.service.tslanguageon all three request bodiessrc/renderer/services/live-transcription.service.ts?language=on the ASR sockets, andsetLanguage()to switch themsrc/renderer/hooks/use-interview-language.tssrc/renderer/components/custom/control-panel/language-group.tsxThe setting has two halves that move at different speeds
Suggestions are free. Each service reads the config store as it builds its request, so the next suggestion already follows a change - nothing to reconnect, nothing to wait for.
The ASR is not. AssemblyAI takes the language as a connection parameter, so switching means tearing both channels down and re-opening them: a second or two of gap, and whatever utterance was mid-flight is orphaned.
useInterviewLanguageis where those two are reconciled, and the UI reports the difference rather than papering over it - the trigger spins while the sockets come back, and the menu says what will happen before the user commits, because a two-second hole in the transcript is alarming if it arrives unannounced mid-question.Two guards on the reconnect
Both protect against the same failure: two sockets on one channel, one of them orphaned and still relaying audio into a dead session. The existing
onclosepath schedules a backoff reconnect on any unexpected close, and a deliberate close for a language switch looks exactly like one.onclosenow ignores a close from a socket that is no longerthis.ws. That is the tail of a replacement, not a disconnect. (This one is a latent fix on the pre-existing reconnect path too.)setLanguagesets aswitchingflag so the backoff reconnect does not fire for the close it caused itself, and reconnects immediately instead of waiting outWS_RETRY_BASE_DELAY_MS.connectWebSocketrebuilds the URL per attempt rather than capturing it, which is what lets a reconnect pick up the new language at all.The first guard has two consequences worth calling out, both caught on review:
setLanguagereportschannelDisconnecteditself rather than leaving it toonclose.new WebSocketassignsthis.wssynchronously, so the old socket's close always arrives after its replacement exists and is correctly ignored - which would have swallowed the report too, leaving the orphaned partial gating live suggestions for the rest of the session.this.ws, not onactive, whichstart()sets only after its first connect returns. In that window a socket already exists on the old language and anactivecheck would skip it.A switch that cannot reconnect hands the channel back to the ordinary backoff loop rather than leaving it silent until the assistant is restarted - five failed attempts is a network or provider problem, not a permanent one - and the toast says it is still retrying. The two channels are switched with
allSettled, sincePromise.allleaves the second one's rejection unhandled.Decisions worth arguing with
English sends no parameter at all, not
language=en. The backend treats an absent language as English and builds the AssemblyAI URL it has always built, so a session that never touches the picker produces byte-identical traffic to what shipped before this existed.The setting is persisted before the reconnect and never rolled back on failure. A failed reconnect that reverted the setting would leave the user with no route to the language they picked; leaving it set means stopping and starting the assistant recovers, and the toast says so.
Resolution happens on the way out of the config store, not on the way in. The disk holds whatever some build wrote - a code a later release dropped, or one an older release never knew - and every consumer reads through
getConfig(), so that is the single point where an unknown code can be stopped before it reaches the ASR URL and three request bodies.Six languages, because that is what
universal-streaming-multilingualtranscribes. Offering one the ASR cannot hear would not degrade gracefully, it would answer a question that was never asked.Endonym first in the menu, English name second. Someone whose interview has just switched language finds "Deutsch" faster than "German"; the English column is there for the reverse lookup. The trigger shows the code (
EN,ES) next to the icon - a globe alone is only useful to someone who already knows what it is set to, which is the one question this control has to answer at a glance.Explicitly not in scope
Localising the app's own chrome - buttons, labels, dialogs, toasts. Separate feature, separate cost (~45 components plus every main-process error string), and not what makes the product unusable in Spanish today: an English button on a Spanish interview is an inconvenience, an English transcript of Spanish speech is a wrong answer read out loud.
Testing
pnpm lint, bothtscconfigs,pnpm buildandpnpm test:mainall clean, CI green -test/language.test.mjsadds 11 checks covering the default, the set, the unknown-code fallback and the store round-trip.Not covered: the reconnect itself.
AudioWsStreamneeds a realWebSocket,AudioContextandgetDisplayMedia, none of which the dependency-free main-process harness reaches. The mid-session switch is worth exercising by hand - change language during a running session and confirm the transcript resumes on the new language, exactly one socket per channel survives, and the card that was in flight is not left pending.