feat(agents): support declarative prompt-voice agents (managed model) - #9364
feat(agents): support declarative prompt-voice agents (managed model)#9364v1212 wants to merge 20 commits into
Conversation
Add azd support for a new 'prompt-voice' agent kind that creates a managed speech-to-speech (voice) agent on Azure AI Foundry. - yaml: new prompt-voice kind + VoiceAgent authoring struct/parsing - map: translate authoring kind prompt-voice -> data-plane kind voice, defaulting the audio pipeline (PCM16@24k, server_vad, whisper-1, DragonHD default voice) and v1 implicit managed model_type - agent_api: VoiceAgentDefinition wire structs + CreateVoiceAgent with Foundry-Features: VoiceAgents=V1Preview preview header - project: voice-aware agent_definition read/write + isolated deployVoiceAgent deploy path (container path unchanged) - init: --kind/--voice flags, voice manifest synthesis, prompt option Scope: prompt-voice + managed model only. BYOM, hosted-voice, tools, avatar, and cascaded models are follow-ups. Draft: needs further end-to-end session testing and optimization.
|
Azure Pipelines: Successfully started running 1 pipeline(s). 21 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
- map_voice_test.go: cover CreateVoiceAgentAPIRequest defaults/overrides, managed enforcement, BYOM rejection, missing-model error, and the isOpenAIVoice/buildVoiceConfig voice-type selection - parse_voice_test.go: cover prompt-voice manifest parsing and ValidateAgentDefinition (ok / missing model.id / self_deployed rejected) - azure.ai.agent.json: add prompt-voice to the kind enum and document the voice service properties (modelType/model/instructions/voice/store) - cspell.yaml: allow BYOM
The azure.ai.projects synthesis copy must stay byte-identical to the azure.ai.agents copy (TestAgentsSynthesisCopyMatches). Mirror the prompt-voice comment update made in the agents synthesizer.
There was a problem hiding this comment.
Pull request overview
Adds managed declarative prompt-voice agents across initialization, configuration, API mapping, and deployment.
Changes:
- Adds voice-agent YAML/schema models and validation.
- Maps voice manifests to Foundry’s preview API contract.
- Adds voice-specific initialization and deployment paths with tests.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
schemas/azure.ai.agent.json |
Adds voice-agent schema fields. |
internal/synthesis/synthesizer.go |
Marks voice agents as non-container. |
internal/project/service_target_agent.go |
Adds voice deployment handling. |
internal/project/agent_definition.go |
Adds inline voice configuration conversion. |
internal/pkg/agents/agent_yaml/yaml.go |
Defines voice authoring models. |
internal/pkg/agents/agent_yaml/parse.go |
Parses and validates voice manifests. |
internal/pkg/agents/agent_yaml/parse_voice_test.go |
Tests voice parsing and validation. |
internal/pkg/agents/agent_yaml/map.go |
Maps voice manifests to API requests. |
internal/pkg/agents/agent_yaml/map_voice_test.go |
Tests voice request mapping. |
internal/pkg/agents/agent_api/operations.go |
Adds the preview create operation. |
internal/pkg/agents/agent_api/models.go |
Defines voice API wire models. |
internal/cmd/init.go |
Adds voice initialization flags and flow. |
internal/cmd/init_from_templates_helpers.go |
Adds the interactive voice option. |
cspell.yaml |
Adds voice-related terminology. |
Resolve four correctness gaps in the declarative prompt-voice flow surfaced in review, keeping every existing hosted/container/workflow path unchanged: - init: validate --kind (and its --image incompatibility) before either the image or prompt-voice synthesis fast path, so `--kind prompt-voice --image` is rejected instead of silently creating a hosted image agent. - init: skipACR now also covers prompt-voice (managed, no container), while a new isHostedAgent decision drives hosted-region filtering. selectFoundryProject gains a distinct filterHostedRegions parameter so a voice agent skips ACR without being constrained to hosted-agent regions. - deploy: resolve an explicit AGENT_DEFINITION_PATH override before the voice/container dispatch (resolveVoiceAgentForDeploy), so an override wins for voice just as it does for the container path. - deploy contract: make Endpoints() and next-step isDeployed voice-aware. Voice agents record only NAME + base ENDPOINT (no agent-version / per-protocol endpoints), so both consumers now treat the base endpoint as the deployment marker instead of reporting a created voice agent as undeployed. Adds unit tests for the skipACR/isHostedAgent split, the override-precedence dispatch, and the voice deployed-marker fallback.
|
Azure Pipelines: Successfully started running 2 pipeline(s). 20 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.
Note
This error may be related to your runner configuration. You can now configure runners for Copilot code review separately from Copilot cloud agent by creating a copilot-code-review.yml file with your setup steps. Read the docs for details.
jongio
left a comment
There was a problem hiding this comment.
Ran through the voice path end to end against the current head. Build is clean and the new tests pass locally. Three things I'd like to sort out before this ships, plus two nits.
Medium
- The new deployed-agent detection keys off "VERSION empty and ENDPOINT set" rather than the service kind, in both
Endpoints()andnextstep.isDeployed(). Inline comments on both. - Voice create is an unconditional POST to
/voice_agentswith no version model. What's the intended behavior for a secondazd deploy? Inline comment onoperations.go. - Delete has no voice path.
AgentClient.DeleteAgentbuilds%s/agents/%s, andDeleteAction.cleanupEnvVarsclears the threeAGENT_<KEY>_*vars regardless. Soazd ai agent deleteagainst a prompt-voice service hits/agents/<name>, gets a 404, andclassifyDeleteErrorturns that intoCodeAgentNotFound. The user is told the agent doesn't exist while it's still live in the project, and there's no way to tear it down from azd. If delete is intentionally out of scope for this PR, could you add it to the follow-ups list in the description so it doesn't get lost?
Low
- Orphaned comment fragment in
init.go. Inline comment. isOpenAIVoiceclassifies by name shape. Inline comment.
One question on AZURE_VOICE_OVERRIDDEN_HOST: the comment says it exists to bypass the public Foundry APIM while the voice route rolls out. Worth filing an issue to remove it once the route is live, otherwise it tends to stick around forever.
Also, the description still says Draft but the PR is open and review is requested. Probably just needs updating.
- Endpoints()/isDeployed(): gate voice base-endpoint fallback on the service's actual prompt-voice kind instead of the env-var shape, so a partially-failed hosted deploy still surfaces CodeMissingAgentEnvVars - add nextstep isVoiceService helper (mirrors project kind gate; the two stay in separate packages to avoid a project->nextstep import cycle) - CreateVoiceAgent: document create-only redeploy semantics - isOpenAIVoice: classify via explicit OpenAI voice set + Azure Neural locale-prefix pattern instead of a bare '-' check - init.go: drop orphaned comment fragment - tests: cover hosted lingering-endpoint gate and voice name classification
|
@jongio thanks for the review — pushed 438466c addressing the inline threads (replied on each; left them unresolved for you to confirm). For the remaining points from the review body:
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.
Suppressed comments (8)
cli/azd/extensions/azure.ai.agents/internal/cmd/init_foundry_project_setup.go:148
- [azd-code-reviewer] The interactive existing-project branch has the same coupling: prompt-voice sets
skipACR=true, which now turns on hosted-region filtering and hides otherwise valid voice projects. Use the separate hosted-agent decision here as well.
skipACR, // filterHostedRegions: this path is code/container only (non-voice)
cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:440
- [azd-code-reviewer] This voice check ignores
AGENT_DEFINITION_PATH, although deployment now lets that override determine voice/container dispatch. A voice override on a hosted service deploys successfully but endpoint discovery still treats it as hosted and fails on the missing VERSION; next-step state has the same mismatch. Persist or resolve a common effective-kind marker in all three consumers.
if _, isVoice, err := VoiceAgentFromResolvedService(serviceConfig, p.projectPath); err != nil {
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1646
- [azd-code-reviewer] This resolved name is not pinned to
flags.agentName.runInitFromManifestlater reachesdownloadAgentYaml, which callsresolveInitAgentNameagain, so the interactive voice flow asks for the agent name twice. Pin the result before entering the manifest flow and add a regression test.
resolvedName, err := resolveInitAgentName(ctx, azdClient, flags, "voice-agent")
cli/azd/extensions/azure.ai.agents/internal/cmd/init_foundry_project_setup.go:74
- [azd-code-reviewer] This still equates skipping ACR with requiring hosted-agent regions. Prompt-voice reaches
configureFoundryProjectwithskipACR=true, so a voice init using--project-idincorrectly filters out projects in regions that do not support hosted agents. Pass a separatefilterHostedRegionsdecision through this helper.
This issue also appears on line 148 of the same file.
skipACR, // filterHostedRegions: this path is code/container only (non-voice)
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1288
- [azd-code-reviewer] The prompt-voice fast path silently ignores
--model-deployment, even though the shared flag help says it takes precedence over--model. Because this PR supports managed models only, reject this combination instead of creating the defaultgpt-realtimeagent.
if flags.image != "" {
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:2216
- [azd-code-reviewer] Recording
isVoiceAgentdoes not preserve the documented two-question scaffold flow. A synthesized voice manifest has no model resources, so this function continues intoconfigureFoundryProject, whose interactive branch prompts for project choice, subscription, and location during init rather than deferring them toazd provisionas the PR description states.
if kind, err := agentManifestKind(agentManifest); err == nil {
a.isVoiceAgent = kind == agent_yaml.AgentKindPromptVoice
cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_api/operations.go:188
- [azd-code-reviewer] The linked REST contract does not have the create-only behavior documented here:
POST /voice_agentscreates an agent or a new version of an existing one, andPOST /voice_agents/{agent_name}provides update semantics. Treating every second deploy as an unavoidable failure makesazd deploynon-repeatable despite API support; implement the existing-agent path rather than deferring it.
// Redeploy semantics: the voice data-plane exposes create-only POST /voice_agents
// with no version/upsert model (unlike hosted agents, which mint a new
// agent-version per deploy). A second `azd deploy` of the same voice service
// therefore re-POSTs with the same name and the service rejects it with a
// non-success status, which this method surfaces as a deploy error rather than
// silently overwriting the existing agent. Idempotent redeploy/update is tracked
// as a follow-up (see the PR "Follow-ups" section); until the service adds an
// update route, redeploy requires deleting the existing voice agent first.
cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_api/operations.go:212
- [azd-code-reviewer]
CreateVoiceAgenthas no operation-level test, leaving the required preview header, optional overridden-host header, route, and accepted response statuses unverified. Add a fake-transport test alongsideoperations_test.gothat asserts these request details and response parsing.
// Voice agents are a preview feature; the service rejects the request with
// 403 preview_feature_required unless this opt-in header is present.
req.Raw().Header.Set("Foundry-Features", voiceAgentsPreviewFeature)
if overriddenHost != "" {
req.Raw().Header.Set("x-ms-overridden-host", overriddenHost)
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated 1 comment.
Suppressed comments (3)
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:3018
- [azd-code-reviewer] This early return silently drops container-specific flags when the voice kind comes from a manifest. For example,
--manifest voice.yaml --image ...succeeds even though--imageis unused;--deploy-mode code, runtime, entry-point, dependency-resolution, protocol, and model-deployment flags are similarly ignored. Validate incompatible options from the parsed kind before dispatching so synthesized and user-provided voice manifests behave consistently.
if agentDef.Kind == agent_yaml.AgentKindPromptVoice {
return a.addVoiceAgentToProject(ctx, targetDir, agentManifest)
cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/state.go:456
- [azd-code-reviewer] Voice classification is used only to choose the deployment marker and is then discarded. The resulting
ServiceStatestill looks invokable, so the generic post-init/doctor resolvers can recommendazd ai agent run,show, andinvokefor prompt-voice agents even though those commands target the non-voice flows and are explicitly out of scope. Preserve a voice/capability marker in state and suppress unsupported suggestions, with resolver tests for voice services.
IsDeployed: isDeployed(ctx, src, envName, svc.Name, isVoiceService(project.Path, svc), errs),
cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:65
- [azd-code-reviewer] Requiring the
idproperty does not rejectmodel: { id: "" }, although both manifest validation and deployment reject an empty model ID. AddminLength: 1so editor/schema validation matches the runtime requirement described below.
"id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }
isOpenAIVoice classifies names case-insensitively, but buildVoiceConfig emitted the original casing, so `--voice Shimmer` produced "Shimmer" while OpenAI wire IDs are lowercase. Trim and lowercase OpenAI voice names; Azure Neural names stay case-sensitive (whitespace-trimmed only).
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.
Suppressed comments (4)
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:2219
- [azd-code-reviewer] A voice manifest supplied with
--manifeststill accepts--imagebecause the earlier incompatibility check only runs when--kindis set. Once this block marks the manifest as voice,isHostedAgent()remains true from the image flag, so project selection incorrectly applies hosted-region filtering, andaddVoiceAgentToProjectlater ignores the image. Reject--imagehere when the parsed manifest is prompt-voice.
if kind, err := agentManifestKind(agentManifest); err == nil {
a.isVoiceAgent = kind == agent_yaml.AgentKindPromptVoice
}
cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:65
- [azd-code-reviewer] Requiring the
idproperty still allowsmodel: { id: "" }, whileCreateVoiceAgentAPIRequestrejects an empty ID. Editor/schema validation therefore accepts a manifest that deploy always rejects. Add a non-empty constraint here to keep the schema aligned with runtime validation.
"id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145
- [azd-code-reviewer] This changes
AZD_AGENT_SKIP_ACRto be set for managed voice agents, butcli/azd/docs/environment-variables.md:161still says it is automatically set only for code-deploy scenarios. Update that environment-variable reference so it reflects the new prompt-voice behavior.
return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent
cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1773
- [azd-code-reviewer] This new override is absent from the environment-variable reference. It is also read from the active azd environment map rather than
os.Getenv, so exporting it in the shell has no effect. Document the requiredazd envusage and default behavior, or add an explicit shell-environment fallback if process-level use is intended.
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"
jongio
left a comment
There was a problem hiding this comment.
Re-approving against 7f5667f. All five commits since 470ea64 check out against the code.
The two AGENT_DEFINITION_PATH fixes close the last classification gap. ensureDeployContext assigns the raw env value to p.agentDefinitionPath without resolving it, so Endpoints and nextstep.isVoiceService falling back to os.Getenv verbatim gives all three paths identical semantics, relative paths included. TestEndpoints_VoiceAgentDefinitionPathOverride covers the fresh-process case that was previously reporting missing-VERSION.
The schema if/then matches runtime. CreateVoiceAgentAPIRequest returns model.id is required for a prompt-voice agent, so requiring model when kind: prompt-voice is inline is the same rule expressed for editors. It can't false-positive on $ref or on-disk manifests, since those don't carry an inline kind and the if won't fire.
Voice casing normalization lines up with isOpenAIVoice, which already lowercases and trims before the known-set lookup, so only the emitted name changed. Azure Neural names stay case-sensitive and only get trimmed.
The new CreateVoiceAgent tests cover the wire contract that had none: collection path, Foundry-Features preview header, x-ms-overridden-host, and the non-success path.
The interactive 'Create a prompt voice agent' path resolved the agent name but did not pin flags.agentName, so the inner resolveInitAgentName call in runInitFromManifest prompted for the name a second time. Pin the resolved name after resolution, matching resolveAgentNameFromManifestPointer.
…ice-agent-managed-model # Conflicts: # cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.
Suppressed comments (5)
cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:848
- [azd-code-reviewer] The shared kind lookup can identify a voice agent from the service directory's
agent.yaml, but this fallback only parses inline/config properties. For the on-disk shape covered byTestKind_ManifestFallbackand the new endpoint test,IsPromptVoicereturns true here, thenVoiceAgentFromResolvedServicereturnsfound=false, and deploy falls through to the container path instead of deploying the voice definition. Parse the same on-disk manifest source when the service entry has no definition.
)
cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:954
- [azd-code-reviewer] Inline voice definitions bypass
validateAgentServiceDefinition, unlike hosted definitions and file-based voice definitions. A manually authored service with an empty/invalidnametherefore reachescreateAgentAPIRequest, which silently deploys it asunspecified-agent-nameinstead of returning the manifest validation error. Validate the reconstructed voice agent before returning it.
func resolveVoiceAgentForDeploy(
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1327
- [azd-code-reviewer] Supplying
--kind prompt-voicetogether with--manifestskips this branch, so--kindbecomes a no-op: a hosted manifest still initializes a hosted agent, and the new--model/--voicevalues are not applied. This is especially surprising because the validation message explicitly offers--manifestas an alternative to--agent-name. Either reject this flag combination or verify that the loaded manifest is prompt-voice and apply the requested overrides.
if flags.kind != "" && flags.manifestPointer == "" {
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:3285
- [azd-code-reviewer] The generic post-init resolver treats this managed voice service as locally runnable. When
--project-idor a reused project already supplies the Foundry endpoint,ResolveAfterInittakes its ready-state branch and printsazd ai agent runplusinvoke --local; prompt-voice has no local source/runtime, and invoke is explicitly out of scope. Make the next-step state/resolver voice-aware or use voice-specific guidance limited to provision/deploy.
fmt.Printf(
cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:65
- [azd-code-reviewer] Requiring the
idproperty still allowsmodel: { id: "" }, while both manifest validation and deploy reject an empty model ID. AddminLength: 1so editor/schema validation matches the runtime requirement stated below.
"description": "Voice agent (kind: prompt-voice) system prompt for the assistant."
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated 1 comment.
Suppressed comments (3)
cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:58
- [azd-code-reviewer] The schema still accepts
model: { id: "" }, while both manifest validation and deployment reject an emptymodel.id. Add a non-empty constraint so editor validation matches runtime validation as the new conditional promises.
"id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }
cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788
- [azd-code-reviewer] This adds a consumed extension environment variable without adding it to the canonical environment-variable reference. Document
AZURE_VOICE_OVERRIDDEN_HOSTunder theazure.ai.agentsdebug/internal variables, including its expected host format and unset behavior, so this temporary routing escape hatch is supportable.
// voiceOverriddenHostEnvKey optionally routes the /voice_agents call directly to
// a regional data-plane host (bypassing the public Foundry APIM, whose voice
// route may not yet be rolled out). When unset, default endpoint routing is used.
//
//nolint:gosec // env var key name, not a credential
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145
- [azd-code-reviewer] The canonical environment-variable reference still says
AZD_AGENT_SKIP_ACRis set only for code-deploy scenarios, but this branch now sets it for prompt-voice agents too. Update that entry so documented provisioning behavior matches this new path.
// This happens when:
// - Code deploy mode is selected (ZIP upload, no container build)
// - Pre-built image is provided via --image flag (user manages their own registry)
// - The manifest is a prompt-voice agent (managed, no container image)
func (a *InitAction) skipACR() bool {
return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent
Previously --kind prompt-voice was silently ignored when --manifest was also supplied, so a hosted manifest would create a hosted service despite the user explicitly selecting the voice kind. Reject the combination early, matching the existing --kind/--image validation.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.
Suppressed comments (3)
cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/map.go:530
- [azd-code-reviewer] A whitespace-only
model.idpasses both manifest validation and this check, then is sent as the managed model name. Normalize the ID withstrings.TrimSpace, reject the normalized empty value, and use the normalized value in the request; keep the schema andValidateAgentDefinitionchecks aligned so hand-authored manifests fail before the service call.
if voiceAgent.Model == nil || voiceAgent.Model.Id == "" {
return nil, fmt.Errorf("model.id is required for a prompt-voice agent")
cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788
- [azd-code-reviewer] This introduces a supported environment override, but
cli/azd/docs/environment-variables.mddoes not list it. That file is the repository's source of truth for environment variables; document whether this must be an azd environment value or shell variable, the expected host format, and that it is a temporary/internal routing override.
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"
cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1802
- [azd-code-reviewer] The tests cover dispatch and the HTTP client separately, but never exercise this new deploy orchestration. Add a provider-level test that verifies the successful call writes both
AGENT_<KEY>_NAMEandAGENT_<KEY>_ENDPOINTand returns the endpoint artifact, plus failure cases for a missing project endpoint and environment persistence errors.
func (p *AgentServiceTargetProvider) deployVoiceAgent(
ctx context.Context,
serviceConfig *azdext.ServiceConfig,
va agent_yaml.VoiceAgent,
azdEnv map[string]string,
progress azdext.ProgressReporter,
) (*azdext.ServiceDeployResult, error) {
jongio
left a comment
There was a problem hiding this comment.
Re-approving against bea4d9c. Verified the three commits since 7f5667f.
4db597e pins flags.agentName after the interactive resolve, matching what resolveAgentNameFromManifestPointer already does, so runInitFromManifest short-circuits instead of prompting a second time. The mutation can't leak, since the only thing running after that switch is ejectInfraAfterInit, which never reads the name.
bea4d9c rejects --kind prompt-voice combined with --manifest before either synthesis fast path, mirroring the existing --image guard. That closes the silent-adopt hole Copilot flagged.
The merge of main didn't drift anything. Net diff against the merge base is still 27 files, +2123/-29, byte-identical to the pre-merge PR, so conflict resolution didn't alter this branch's own contribution. All three agentkind.IsPromptVoice call sites (deploy dispatch, Endpoints, next-step reader) still share one lookup with the same AGENT_DEFINITION_PATH precedence. Main's new setServiceEnvironment call in addToProject correctly doesn't apply to addVoiceAgentToProject, since EnvironmentVariables only exists on ContainerAgent and a voice agent has no way to declare env.
Build and vet are clean and the agents extension tests pass. The two internal/synthesis bicep-stale failures reproduce identically on main at 493e6a7, so they aren't from this branch.
One leftover error hint, inline.
When 'azd ai agent init' for a prompt-voice (managed) agent runs inside an existing azd project, add it as a new azure.ai.agent service to the current azure.yaml (src/<name> layout), matching hosted and other agents, instead of scaffolding a separate nested <name>/ project. Applies to both the interactive voice menu path and the '--kind prompt-voice' fast path. A brand-new (empty) init still creates the <name>/ project folder.
Since --kind prompt-voice combined with --manifest is now rejected, drop the '(or provide --manifest ...)' remediation that would walk the user into a dead end. Addresses PR review feedback.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.
Suppressed comments (5)
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1286
- [azd-code-reviewer] The prompt-voice compatibility check covers
--imageand--manifest, but container/code flags still pass through.--kind prompt-voice --deploy-mode containeris silently ignored; with--deploy-mode code,validateCodeDeployFlagsmay require--runtime/--entry-point, after which all three values are still ignored because aVoiceAgentbypassespromptDeployMode. Reject deployment-mode-specific flags when--kind prompt-voiceis selected so accepted CLI input is never discarded.
if flags.image != "" {
return exterrors.Validation(
exterrors.CodeInvalidParameter,
"--kind prompt-voice cannot be combined with --image",
"a voice agent is managed and has no container image; drop --image",
cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788
- [azd-code-reviewer] This new deployment override is absent from
cli/azd/docs/environment-variables.md, which is the repository's source of truth for extension environment variables. Document its expected host format, default routing behavior, and temporary/internal support status so users do not have to infer how to configure it from source.
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"
cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:969
- [azd-code-reviewer] This kind probe now intercepts an invalid
AGENT_DEFINITION_PATHbefore the existing container loader can classify it. For example, malformed YAML in a hosted override returns a raw YAML error here instead of the existingCodeInvalidAgentManifesterror and recovery suggestion. Parse the explicit override throughvoiceAgentFromDefinitionFilefirst, then useagentkindonly when resolving the service entry.
isVoice, err := agentkind.IsPromptVoice(svc, projectRoot, agentDefinitionPath)
if err != nil {
return agent_yaml.VoiceAgent{}, false, err
cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:58
- [azd-code-reviewer] Requiring the
idproperty still permitsmodel: { id: "" }, while both manifest validation and deployment reject an empty model ID. Add a minimum length so editor/schema validation matches the runtime requirement stated below.
"id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145
- [azd-code-reviewer] Extending
skipACRto voice agents makes the currentAZD_AGENT_SKIP_ACRentry incli/azd/docs/environment-variables.md:161inaccurate because it says the variable is set automatically only for code-deploy scenarios. Update that entry to include managed prompt-voice initialization.
return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent
jongio
left a comment
There was a problem hiding this comment.
Re-approving against 03f8a6c. Verified both commits since bea4d9c.
7b30f47 lines up the prompt-voice target directory across the interactive and non-interactive paths. Inside an existing project both now hold targetDir at "." and append the service to the current azure.yaml, while a fresh init still creates the <name>/ folder and the cd hint. I walked all four combinations (interactive and flag-driven, crossed with existing and empty project) and they agree. Reordering the manifestInCwd branch keeps the old flags.src = "." behavior, and because --kind and --manifest are rejected together at line 1288, the synthesized voice manifest always lands in a temp dir, so manifestInCwd can't shadow the new voice branch.
03f8a6c drops the --manifest remediation from the prompt-voice hint, which matches that validation. Leaving the --image hint at line 1314 alone also tracks, since --image has no equivalent incompatibility check and still accepts --manifest.
Build, vet, and gofmt are clean here, and the cmd package tests pass.
Non-blocking, for a follow-up: the targetDir and folderDisplay decision now lives in three places: the non-interactive manifest branch, the template branch, and the interactive voice branch. That duplication is the same drift 7b30f47 is fixing, and the template branch still nests into <name>/ inside an existing project. The rest of this package factors decisions like this into a helper and unit tests it, the way synthesizeVoiceManifestFile and resolveInitAgentName are covered. Worth pulling this one out too so the next kind can't drift?
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.
Suppressed comments (4)
cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:58
- [azd-code-reviewer]
required: ["id"]still acceptsid: "", whileCreateVoiceAgentAPIRequestrejects an empty ID. Editors therefore report this configuration as valid only for deploy to fail. Add a minimum length to keep schema and runtime validation aligned.
"id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }
cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788
- [azd-code-reviewer] This adds a new extension environment setting, but it is absent from the
azure.ai.agentstable incli/azd/docs/environment-variables.md. Users cannot discover that this expects a regional host in the active azd environment or that leaving it unset uses the default APIM route. Document it alongside the other extension-specific variables.
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145
- [azd-code-reviewer]
skipACR()now persistsAZD_AGENT_SKIP_ACR=truefor prompt-voice, butcli/azd/docs/environment-variables.md:161still describes automatic use only for code deploy. Update that entry so the documented behavior matches generated voice projects.
return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1651
- [azd-code-reviewer] The added tests cover the synthesis helper and state predicates, but neither the interactive voice branch nor the
--kind prompt-voicecommand path is driven through service creation. The conflict validation, target-directory handling, and inlineAddServiceshape can regress while these tests still pass. Add an action-level test that exercises each entry path and asserts the generated voice service properties.
case initModeVoice:
Summary
Closes #9336
Adds azd support for a new declarative
prompt-voiceagent kind that creates a managed speech-to-speech (voice) agent on Azure AI Foundry, end-to-end throughazd init→azure.yaml→azd deploy.Scope (intentionally narrow):
prompt-voice+ managed model only — i.e. scaffold (init) and deploy only. BYOM, hosted-voice, tools, avatar, and cascaded models are deliberately left as follow-up PRs. azd-native textinvoke,list, and a Portal Playground link are out of scope for this PR (see Follow-ups).What's included
agent_yaml/yaml.go,parse.go): newprompt-voiceauthoring kind +VoiceAgentstruct/parsing/validation.agent_yaml/map.go): translate authoring kindprompt-voice→ data-plane service kindvoice; default the audio pipeline (PCM16 @ 24 kHz,server_vad,whisper-1, DragonHD default voice); v1 = implicitmodel_type: managed.agent_api/models.go,operations.go):VoiceAgentDefinitionwire structs +CreateVoiceAgentwith the required preview headerFoundry-Features: VoiceAgents=V1Preview.project/agent_definition.go,service_target_agent.go): voice-aware definition read/write + an isolateddeployVoiceAgentdeploy path — the existing hosted/container path is byte-for-byte unchanged.cmd/init.go,init_from_templates_helpers.go):--kind/--model/--voiceflags, voice manifest synthesis, and a new interactive prompt option.azure.ai.agent.jsongains theprompt-voicekind + voice service properties; unit tests cover the map translation, voice-type selection, and manifest parse/validation.Compatibility
All changes are additive — new
casebranches with untoucheddefaults andomitemptyfields. Existingazd init/ invoke / deploy / list flows and the hosted/workflow code paths are unchanged.Testing
Automated
go build ./...,go vet ./..., andcspellclean.map_voice_test.go,parse_voice_test.go,init_test.go,nextstep/state_test.go,voice_deploy_dispatch_test.go) plus the existing suite pass.Live validation
CreateVoiceAgentcode appears in the project list and its storeddefinitionis byte-identical to the service's own managed reference agent (kind:voice/model_type:managed/ audio pipeline / voice config). Connect reachessession.created.How to test end-to-end
Build the branch binary and install the branch build of the
azure.ai.agentsextension first:1) Interactive (recommended)
azd ai agent initfollows the same prompt flow as other agent kinds (agent name, then the standard Foundry-project selection).--model(defaultgpt-realtime) and--voiceare flags, not prompts — pass them on the same command to override, e.g.azd ai agent init --model gpt-realtime --voice alloy.azd provisionandazd deployadd no voice-specific interaction — they behave exactly like the existing hosted/code experience.infra/is required. Wheninfra/is absent, the Foundry provider synthesizes the embedded ARM template in-memory and resolves the required parameters via the interactive prompts.--infrais optional (only needed to eject IaC to disk).azure.yamlservice:host: azure.ai.agent,kind: prompt-voice,modelType: managed,model: { id: gpt-realtime }(avoice:field appears only when--voicewas passed).2) Non-interactive (CI / scripted)
--no-promptcannot prompt for the three required provision parameters, so eject IaC once and set them explicitly:3) Runtime check (client)
azd-native text
invokeis out of scope for this PR (managed voice agents are Voice Live realtime over WebSocket; the/voice_agentsdata plane has no HTTP text-invoke endpoint). Validate runtime instead with a voice client connecting to the deployed agent — a session reachessession.createdand supports voice/text turns. A read-only control-planeGET /voice_agents/<name>(with headerFoundry-Features: VoiceAgents=V1Preview) is a quick sanity check: expectstate=enabled,definition.model_type=managed.Known gaps
session.audio.output.voicestring coercion at the Voice Live layer, and a demo-clientsession.updateframing issue) — being investigated; not specific to this change.Follow-ups
self_deployed) model typeinvoke,list, and Portal Playground link for voice agentsazd ai agent deletefor voice services — the current delete path targets/agents/<name>and returnsCodeAgentNotFound(404) for a voice agent, which lives under/voice_agents/<name>. Voice-aware teardown (and an idempotent redeploy/update path over the create-only/voice_agentsroute) is a follow-up.AZURE_VOICE_OVERRIDDEN_HOST/x-ms-overridden-hostonce the public Foundry APIM voice route is generally rolled out; will be filed as a tracking issue.