Skip to content

feat(agents): support declarative prompt-voice agents (managed model) - #9364

Open
v1212 wants to merge 20 commits into
Azure:mainfrom
v1212:users/wujia/prompt-voice-agent-managed-model
Open

feat(agents): support declarative prompt-voice agents (managed model)#9364
v1212 wants to merge 20 commits into
Azure:mainfrom
v1212:users/wujia/prompt-voice-agent-managed-model

Conversation

@v1212

@v1212 v1212 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #9336

Adds azd support for a new declarative prompt-voice agent kind that creates a managed speech-to-speech (voice) agent on Azure AI Foundry, end-to-end through azd initazure.yamlazd deploy.

Scope (intentionally narrow): prompt-voice + managed model only — i.e. scaffold (init) and deploy only. BYOM, hosted-voice, tools, avatar, and cascaded models are deliberately left as follow-up PRs. azd-native text invoke, list, and a Portal Playground link are out of scope for this PR (see Follow-ups).

What's included

  • yaml (agent_yaml/yaml.go, parse.go): new prompt-voice authoring kind + VoiceAgent struct/parsing/validation.
  • map (agent_yaml/map.go): translate authoring kind prompt-voice → data-plane service kind voice; default the audio pipeline (PCM16 @ 24 kHz, server_vad, whisper-1, DragonHD default voice); v1 = implicit model_type: managed.
  • agent_api (agent_api/models.go, operations.go): VoiceAgentDefinition wire structs + CreateVoiceAgent with the required preview header Foundry-Features: VoiceAgents=V1Preview.
  • project (project/agent_definition.go, service_target_agent.go): voice-aware definition read/write + an isolated deployVoiceAgent deploy path — the existing hosted/container path is byte-for-byte unchanged.
  • init (cmd/init.go, init_from_templates_helpers.go): --kind / --model / --voice flags, voice manifest synthesis, and a new interactive prompt option.
  • schema + tests: azure.ai.agent.json gains the prompt-voice kind + voice service properties; unit tests cover the map translation, voice-type selection, and manifest parse/validation.

Compatibility

All changes are additive — new case branches with untouched defaults and omitempty fields. Existing azd init / invoke / deploy / list flows and the hosted/workflow code paths are unchanged.

Testing

Automated

  • go build ./..., go vet ./..., and cspell clean.
  • New unit tests (map_voice_test.go, parse_voice_test.go, init_test.go, nextstep/state_test.go, voice_deploy_dispatch_test.go) plus the existing suite pass.

Live validation

  • Against a real Foundry project: an agent created via the real CreateVoiceAgent code appears in the project list and its stored definition is byte-identical to the service's own managed reference agent (kind:voice / model_type:managed / audio pipeline / voice config). Connect reaches session.created.

How to test end-to-end

Build the branch binary and install the branch build of the azure.ai.agents extension first:

cd cli/azd && go build -o azd-branch.exe ./     # then: AZD=./azd-branch.exe
# install the branch-built azure.ai.agents extension from local source (overrides the released build)

1) Interactive (recommended)

mkdir voice-test && cd voice-test
"$AZD" ai agent init      # choose "Create a prompt voice agent", then follow the standard init prompts
"$AZD" provision          # standard azd provision (subscription / location / resource group / Foundry project)
"$AZD" deploy             # POST /voice_agents -> stderr "Voice agent '<name>' created successfully!"
"$AZD" env get-values | grep AGENT_    # expect AGENT_<KEY>_NAME and AGENT_<KEY>_ENDPOINT
  • The interactive azd ai agent init follows the same prompt flow as other agent kinds (agent name, then the standard Foundry-project selection). --model (default gpt-realtime) and --voice are flags, not prompts — pass them on the same command to override, e.g. azd ai agent init --model gpt-realtime --voice alloy.
  • azd provision and azd deploy add no voice-specific interaction — they behave exactly like the existing hosted/code experience.
  • No on-disk infra/ is required. When infra/ is absent, the Foundry provider synthesizes the embedded ARM template in-memory and resolves the required parameters via the interactive prompts. --infra is optional (only needed to eject IaC to disk).
  • Resulting azure.yaml service: host: azure.ai.agent, kind: prompt-voice, modelType: managed, model: { id: gpt-realtime } (a voice: field appears only when --voice was passed).

2) Non-interactive (CI / scripted)

--no-prompt cannot prompt for the three required provision parameters, so eject IaC once and set them explicitly:

export AZURE_SUBSCRIPTION_ID=<sub> AZURE_LOCATION=eastus2 AZURE_TENANT_ID=<tenant>

"$AZD" init --minimal --environment voice-test --no-prompt
"$AZD" ai agent init --kind prompt-voice --agent-name my-voice-agent \
       --model gpt-realtime --voice alloy --no-prompt
"$AZD" ai agent init --infra --no-prompt                              # eject embedded Bicep to ./infra
"$AZD" env config set infra.parameters.location eastus2
"$AZD" env config set infra.parameters.resourceGroupName rg-voice-test
"$AZD" env config set infra.parameters.foundryProjectName fp-voice-test
"$AZD" provision --no-prompt
"$AZD" deploy --no-prompt
"$AZD" env get-values | grep AGENT_

3) Runtime check (client)

azd-native text invoke is out of scope for this PR (managed voice agents are Voice Live realtime over WebSocket; the /voice_agents data plane has no HTTP text-invoke endpoint). Validate runtime instead with a voice client connecting to the deployed agent — a session reaches session.created and supports voice/text turns. A read-only control-plane GET /voice_agents/<name> (with header Foundry-Features: VoiceAgents=V1Preview) is a quick sanity check: expect state=enabled, definition.model_type=managed.

Known gaps

  • End-to-end voice session runtime still surfaces errors that reproduce equally on the service's own reference managed agent (e.g. session.audio.output.voice string coercion at the Voice Live layer, and a demo-client session.update framing issue) — being investigated; not specific to this change.

Follow-ups

  • BYOM (self_deployed) model type
  • hosted-voice, tools, avatar, cascaded models
  • azd-native text invoke, list, and Portal Playground link for voice agents
  • azd ai agent delete for voice services — the current delete path targets /agents/<name> and returns CodeAgentNotFound (404) for a voice agent, which lives under /voice_agents/<name>. Voice-aware teardown (and an idempotent redeploy/update path over the create-only /voice_agents route) is a follow-up.
  • Remove AZURE_VOICE_OVERRIDDEN_HOST / x-ms-overridden-host once the public Foundry APIM voice route is generally rolled out; will be filed as a tracking issue.

Add azd support for a new 'prompt-voice' agent kind that creates a
managed speech-to-speech (voice) agent on Azure AI Foundry.

- yaml: new prompt-voice kind + VoiceAgent authoring struct/parsing
- map: translate authoring kind prompt-voice -> data-plane kind voice,
  defaulting the audio pipeline (PCM16@24k, server_vad, whisper-1,
  DragonHD default voice) and v1 implicit managed model_type
- agent_api: VoiceAgentDefinition wire structs + CreateVoiceAgent with
  Foundry-Features: VoiceAgents=V1Preview preview header
- project: voice-aware agent_definition read/write + isolated
  deployVoiceAgent deploy path (container path unchanged)
- init: --kind/--voice flags, voice manifest synthesis, prompt option

Scope: prompt-voice + managed model only. BYOM, hosted-voice, tools,
avatar, and cascaded models are follow-ups. Draft: needs further
end-to-end session testing and optimization.
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
21 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@github-actions github-actions Bot added the ext-agents azure.ai.agents extension label Jul 30, 2026
- map_voice_test.go: cover CreateVoiceAgentAPIRequest defaults/overrides,
  managed enforcement, BYOM rejection, missing-model error, and the
  isOpenAIVoice/buildVoiceConfig voice-type selection
- parse_voice_test.go: cover prompt-voice manifest parsing and
  ValidateAgentDefinition (ok / missing model.id / self_deployed rejected)
- azure.ai.agent.json: add prompt-voice to the kind enum and document the
  voice service properties (modelType/model/instructions/voice/store)
- cspell.yaml: allow BYOM
@v1212
v1212 requested a balanced review from Copilot July 30, 2026 09:16
The azure.ai.projects synthesis copy must stay byte-identical to the
azure.ai.agents copy (TestAgentsSynthesisCopyMatches). Mirror the
prompt-voice comment update made in the agents synthesizer.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds managed declarative prompt-voice agents across initialization, configuration, API mapping, and deployment.

Changes:

  • Adds voice-agent YAML/schema models and validation.
  • Maps voice manifests to Foundry’s preview API contract.
  • Adds voice-specific initialization and deployment paths with tests.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
schemas/azure.ai.agent.json Adds voice-agent schema fields.
internal/synthesis/synthesizer.go Marks voice agents as non-container.
internal/project/service_target_agent.go Adds voice deployment handling.
internal/project/agent_definition.go Adds inline voice configuration conversion.
internal/pkg/agents/agent_yaml/yaml.go Defines voice authoring models.
internal/pkg/agents/agent_yaml/parse.go Parses and validates voice manifests.
internal/pkg/agents/agent_yaml/parse_voice_test.go Tests voice parsing and validation.
internal/pkg/agents/agent_yaml/map.go Maps voice manifests to API requests.
internal/pkg/agents/agent_yaml/map_voice_test.go Tests voice request mapping.
internal/pkg/agents/agent_api/operations.go Adds the preview create operation.
internal/pkg/agents/agent_api/models.go Defines voice API wire models.
internal/cmd/init.go Adds voice initialization flags and flow.
internal/cmd/init_from_templates_helpers.go Adds the interactive voice option.
cspell.yaml Adds voice-related terminology.

Comment thread cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go Outdated
Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/init.go
Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/init.go Outdated
@github-actions github-actions Bot added the ext-projects azure.ai.projects extension label Jul 30, 2026
Resolve four correctness gaps in the declarative prompt-voice flow surfaced
in review, keeping every existing hosted/container/workflow path unchanged:

- init: validate --kind (and its --image incompatibility) before either the
  image or prompt-voice synthesis fast path, so `--kind prompt-voice --image`
  is rejected instead of silently creating a hosted image agent.
- init: skipACR now also covers prompt-voice (managed, no container), while a
  new isHostedAgent decision drives hosted-region filtering. selectFoundryProject
  gains a distinct filterHostedRegions parameter so a voice agent skips ACR
  without being constrained to hosted-agent regions.
- deploy: resolve an explicit AGENT_DEFINITION_PATH override before the
  voice/container dispatch (resolveVoiceAgentForDeploy), so an override wins for
  voice just as it does for the container path.
- deploy contract: make Endpoints() and next-step isDeployed voice-aware. Voice
  agents record only NAME + base ENDPOINT (no agent-version / per-protocol
  endpoints), so both consumers now treat the base endpoint as the deployment
  marker instead of reporting a created voice agent as undeployed.

Adds unit tests for the skipACR/isHostedAgent split, the override-precedence
dispatch, and the voice deployed-marker fallback.
@v1212
v1212 marked this pull request as ready for review August 3, 2026 09:13
Copilot AI review requested due to automatic review settings August 3, 2026 09:13
@v1212
v1212 requested review from huimiu and hund030 as code owners August 3, 2026 09:13
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).
20 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Note

This error may be related to your runner configuration. You can now configure runners for Copilot code review separately from Copilot cloud agent by creating a copilot-code-review.yml file with your setup steps. Read the docs for details.

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran through the voice path end to end against the current head. Build is clean and the new tests pass locally. Three things I'd like to sort out before this ships, plus two nits.

Medium

  1. The new deployed-agent detection keys off "VERSION empty and ENDPOINT set" rather than the service kind, in both Endpoints() and nextstep.isDeployed(). Inline comments on both.
  2. Voice create is an unconditional POST to /voice_agents with no version model. What's the intended behavior for a second azd deploy? Inline comment on operations.go.
  3. Delete has no voice path. AgentClient.DeleteAgent builds %s/agents/%s, and DeleteAction.cleanupEnvVars clears the three AGENT_<KEY>_* vars regardless. So azd ai agent delete against a prompt-voice service hits /agents/<name>, gets a 404, and classifyDeleteError turns that into CodeAgentNotFound. The user is told the agent doesn't exist while it's still live in the project, and there's no way to tear it down from azd. If delete is intentionally out of scope for this PR, could you add it to the follow-ups list in the description so it doesn't get lost?

Low

  1. Orphaned comment fragment in init.go. Inline comment.
  2. isOpenAIVoice classifies by name shape. Inline comment.

One question on AZURE_VOICE_OVERRIDDEN_HOST: the comment says it exists to bypass the public Foundry APIM while the voice route rolls out. Worth filing an issue to remove it once the route is live, otherwise it tends to stick around forever.

Also, the description still says Draft but the PR is open and review is requested. Probably just needs updating.

Comment thread cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go Outdated
Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/state.go
Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/init.go Outdated
- Endpoints()/isDeployed(): gate voice base-endpoint fallback on the
  service's actual prompt-voice kind instead of the env-var shape, so a
  partially-failed hosted deploy still surfaces CodeMissingAgentEnvVars
- add nextstep isVoiceService helper (mirrors project kind gate; the two
  stay in separate packages to avoid a project->nextstep import cycle)
- CreateVoiceAgent: document create-only redeploy semantics
- isOpenAIVoice: classify via explicit OpenAI voice set + Azure Neural
  locale-prefix pattern instead of a bare '-' check
- init.go: drop orphaned comment fragment
- tests: cover hosted lingering-endpoint gate and voice name classification
Copilot AI review requested due to automatic review settings August 3, 2026 12:43
@v1212

v1212 commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

@jongio thanks for the review — pushed 438466c addressing the inline threads (replied on each; left them unresolved for you to confirm). For the remaining points from the review body:

  • azd ai agent delete on a voice service (404 CodeAgentNotFound because it targets /agents/<name> not /voice_agents/<name>): out of scope for this init+deploy PR — added a Follow-ups entry for voice-aware teardown (plus the idempotent redeploy/update path over the create-only /voice_agents route).
  • Nit: > Draft text — removed from the description (PR is out of draft).
  • Nit: AZURE_VOICE_OVERRIDDEN_HOST / x-ms-overridden-host cleanup — added a Follow-ups entry; will file a tracking issue to remove it once the public Foundry APIM voice route is rolled out.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

Suppressed comments (8)

cli/azd/extensions/azure.ai.agents/internal/cmd/init_foundry_project_setup.go:148

  • [azd-code-reviewer] The interactive existing-project branch has the same coupling: prompt-voice sets skipACR=true, which now turns on hosted-region filtering and hides otherwise valid voice projects. Use the separate hosted-agent decision here as well.
				skipACR, // filterHostedRegions: this path is code/container only (non-voice)

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:440

  • [azd-code-reviewer] This voice check ignores AGENT_DEFINITION_PATH, although deployment now lets that override determine voice/container dispatch. A voice override on a hosted service deploys successfully but endpoint discovery still treats it as hosted and fails on the missing VERSION; next-step state has the same mismatch. Persist or resolve a common effective-kind marker in all three consumers.
	if _, isVoice, err := VoiceAgentFromResolvedService(serviceConfig, p.projectPath); err != nil {

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1646

  • [azd-code-reviewer] This resolved name is not pinned to flags.agentName. runInitFromManifest later reaches downloadAgentYaml, which calls resolveInitAgentName again, so the interactive voice flow asks for the agent name twice. Pin the result before entering the manifest flow and add a regression test.
					resolvedName, err := resolveInitAgentName(ctx, azdClient, flags, "voice-agent")

cli/azd/extensions/azure.ai.agents/internal/cmd/init_foundry_project_setup.go:74

  • [azd-code-reviewer] This still equates skipping ACR with requiring hosted-agent regions. Prompt-voice reaches configureFoundryProject with skipACR=true, so a voice init using --project-id incorrectly filters out projects in regions that do not support hosted agents. Pass a separate filterHostedRegions decision through this helper.

This issue also appears on line 148 of the same file.

			skipACR, // filterHostedRegions: this path is code/container only (non-voice)

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1288

  • [azd-code-reviewer] The prompt-voice fast path silently ignores --model-deployment, even though the shared flag help says it takes precedence over --model. Because this PR supports managed models only, reject this combination instead of creating the default gpt-realtime agent.
				if flags.image != "" {

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:2216

  • [azd-code-reviewer] Recording isVoiceAgent does not preserve the documented two-question scaffold flow. A synthesized voice manifest has no model resources, so this function continues into configureFoundryProject, whose interactive branch prompts for project choice, subscription, and location during init rather than deferring them to azd provision as the PR description states.
	if kind, err := agentManifestKind(agentManifest); err == nil {
		a.isVoiceAgent = kind == agent_yaml.AgentKindPromptVoice

cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_api/operations.go:188

  • [azd-code-reviewer] The linked REST contract does not have the create-only behavior documented here: POST /voice_agents creates an agent or a new version of an existing one, and POST /voice_agents/{agent_name} provides update semantics. Treating every second deploy as an unavoidable failure makes azd deploy non-repeatable despite API support; implement the existing-agent path rather than deferring it.
// Redeploy semantics: the voice data-plane exposes create-only POST /voice_agents
// with no version/upsert model (unlike hosted agents, which mint a new
// agent-version per deploy). A second `azd deploy` of the same voice service
// therefore re-POSTs with the same name and the service rejects it with a
// non-success status, which this method surfaces as a deploy error rather than
// silently overwriting the existing agent. Idempotent redeploy/update is tracked
// as a follow-up (see the PR "Follow-ups" section); until the service adds an
// update route, redeploy requires deleting the existing voice agent first.

cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_api/operations.go:212

  • [azd-code-reviewer] CreateVoiceAgent has no operation-level test, leaving the required preview header, optional overridden-host header, route, and accepted response statuses unverified. Add a fake-transport test alongside operations_test.go that asserts these request details and response parsing.
	// Voice agents are a preview feature; the service rejects the request with
	// 403 preview_feature_required unless this opt-in header is present.
	req.Raw().Header.Set("Foundry-Features", voiceAgentsPreviewFeature)

	if overriddenHost != "" {
		req.Raw().Header.Set("x-ms-overridden-host", overriddenHost)

Comment thread cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/map.go Outdated
Copilot AI review requested due to automatic review settings August 3, 2026 15:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:3018

  • [azd-code-reviewer] This early return silently drops container-specific flags when the voice kind comes from a manifest. For example, --manifest voice.yaml --image ... succeeds even though --image is unused; --deploy-mode code, runtime, entry-point, dependency-resolution, protocol, and model-deployment flags are similarly ignored. Validate incompatible options from the parsed kind before dispatching so synthesized and user-provided voice manifests behave consistently.
	if agentDef.Kind == agent_yaml.AgentKindPromptVoice {
		return a.addVoiceAgentToProject(ctx, targetDir, agentManifest)

cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/state.go:456

  • [azd-code-reviewer] Voice classification is used only to choose the deployment marker and is then discarded. The resulting ServiceState still looks invokable, so the generic post-init/doctor resolvers can recommend azd ai agent run, show, and invoke for prompt-voice agents even though those commands target the non-voice flows and are explicitly out of scope. Preserve a voice/capability marker in state and suppress unsupported suggestions, with resolver tests for voice services.
			IsDeployed:    isDeployed(ctx, src, envName, svc.Name, isVoiceService(project.Path, svc), errs),

cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:65

  • [azd-code-reviewer] Requiring the id property does not reject model: { id: "" }, although both manifest validation and deployment reject an empty model ID. Add minLength: 1 so editor/schema validation matches the runtime requirement described below.
        "id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }

isOpenAIVoice classifies names case-insensitively, but buildVoiceConfig
emitted the original casing, so `--voice Shimmer` produced "Shimmer"
while OpenAI wire IDs are lowercase. Trim and lowercase OpenAI voice
names; Azure Neural names stay case-sensitive (whitespace-trimmed only).
Copilot AI review requested due to automatic review settings August 3, 2026 15:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.

Suppressed comments (4)

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:2219

  • [azd-code-reviewer] A voice manifest supplied with --manifest still accepts --image because the earlier incompatibility check only runs when --kind is set. Once this block marks the manifest as voice, isHostedAgent() remains true from the image flag, so project selection incorrectly applies hosted-region filtering, and addVoiceAgentToProject later ignores the image. Reject --image here when the parsed manifest is prompt-voice.
	if kind, err := agentManifestKind(agentManifest); err == nil {
		a.isVoiceAgent = kind == agent_yaml.AgentKindPromptVoice
	}

cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:65

  • [azd-code-reviewer] Requiring the id property still allows model: { id: "" }, while CreateVoiceAgentAPIRequest rejects an empty ID. Editor/schema validation therefore accepts a manifest that deploy always rejects. Add a non-empty constraint here to keep the schema aligned with runtime validation.
        "id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145

  • [azd-code-reviewer] This changes AZD_AGENT_SKIP_ACR to be set for managed voice agents, but cli/azd/docs/environment-variables.md:161 still says it is automatically set only for code-deploy scenarios. Update that environment-variable reference so it reflects the new prompt-voice behavior.
	return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1773

  • [azd-code-reviewer] This new override is absent from the environment-variable reference. It is also read from the active azd environment map rather than os.Getenv, so exporting it in the shell has no effect. Document the required azd env usage and default behavior, or add an explicit shell-environment fallback if process-level use is intended.
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving against 7f5667f. All five commits since 470ea64 check out against the code.

The two AGENT_DEFINITION_PATH fixes close the last classification gap. ensureDeployContext assigns the raw env value to p.agentDefinitionPath without resolving it, so Endpoints and nextstep.isVoiceService falling back to os.Getenv verbatim gives all three paths identical semantics, relative paths included. TestEndpoints_VoiceAgentDefinitionPathOverride covers the fresh-process case that was previously reporting missing-VERSION.

The schema if/then matches runtime. CreateVoiceAgentAPIRequest returns model.id is required for a prompt-voice agent, so requiring model when kind: prompt-voice is inline is the same rule expressed for editors. It can't false-positive on $ref or on-disk manifests, since those don't carry an inline kind and the if won't fire.

Voice casing normalization lines up with isOpenAIVoice, which already lowercases and trims before the known-set lookup, so only the emitted name changed. Azure Neural names stay case-sensitive and only get trimmed.

The new CreateVoiceAgent tests cover the wire contract that had none: collection path, Foundry-Features preview header, x-ms-overridden-host, and the non-success path.

The interactive 'Create a prompt voice agent' path resolved the agent
name but did not pin flags.agentName, so the inner resolveInitAgentName
call in runInitFromManifest prompted for the name a second time. Pin the
resolved name after resolution, matching resolveAgentNameFromManifestPointer.
Copilot AI review requested due to automatic review settings August 4, 2026 05:20
…ice-agent-managed-model

# Conflicts:
#	cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.

Suppressed comments (5)

cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:848

  • [azd-code-reviewer] The shared kind lookup can identify a voice agent from the service directory's agent.yaml, but this fallback only parses inline/config properties. For the on-disk shape covered by TestKind_ManifestFallback and the new endpoint test, IsPromptVoice returns true here, then VoiceAgentFromResolvedService returns found=false, and deploy falls through to the container path instead of deploying the voice definition. Parse the same on-disk manifest source when the service entry has no definition.
		)

cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:954

  • [azd-code-reviewer] Inline voice definitions bypass validateAgentServiceDefinition, unlike hosted definitions and file-based voice definitions. A manually authored service with an empty/invalid name therefore reaches createAgentAPIRequest, which silently deploys it as unspecified-agent-name instead of returning the manifest validation error. Validate the reconstructed voice agent before returning it.
func resolveVoiceAgentForDeploy(

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1327

  • [azd-code-reviewer] Supplying --kind prompt-voice together with --manifest skips this branch, so --kind becomes a no-op: a hosted manifest still initializes a hosted agent, and the new --model/--voice values are not applied. This is especially surprising because the validation message explicitly offers --manifest as an alternative to --agent-name. Either reject this flag combination or verify that the loaded manifest is prompt-voice and apply the requested overrides.
			if flags.kind != "" && flags.manifestPointer == "" {

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:3285

  • [azd-code-reviewer] The generic post-init resolver treats this managed voice service as locally runnable. When --project-id or a reused project already supplies the Foundry endpoint, ResolveAfterInit takes its ready-state branch and prints azd ai agent run plus invoke --local; prompt-voice has no local source/runtime, and invoke is explicitly out of scope. Make the next-step state/resolver voice-aware or use voice-specific guidance limited to provision/deploy.
	fmt.Printf(

cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:65

  • [azd-code-reviewer] Requiring the id property still allows model: { id: "" }, while both manifest validation and deploy reject an empty model ID. Add minLength: 1 so editor/schema validation matches the runtime requirement stated below.
      "description": "Voice agent (kind: prompt-voice) system prompt for the assistant."

Copilot AI review requested due to automatic review settings August 4, 2026 05:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:58

  • [azd-code-reviewer] The schema still accepts model: { id: "" }, while both manifest validation and deployment reject an empty model.id. Add a non-empty constraint so editor validation matches runtime validation as the new conditional promises.
        "id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788

  • [azd-code-reviewer] This adds a consumed extension environment variable without adding it to the canonical environment-variable reference. Document AZURE_VOICE_OVERRIDDEN_HOST under the azure.ai.agents debug/internal variables, including its expected host format and unset behavior, so this temporary routing escape hatch is supportable.
// voiceOverriddenHostEnvKey optionally routes the /voice_agents call directly to
// a regional data-plane host (bypassing the public Foundry APIM, whose voice
// route may not yet be rolled out). When unset, default endpoint routing is used.
//
//nolint:gosec // env var key name, not a credential
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145

  • [azd-code-reviewer] The canonical environment-variable reference still says AZD_AGENT_SKIP_ACR is set only for code-deploy scenarios, but this branch now sets it for prompt-voice agents too. Update that entry so documented provisioning behavior matches this new path.
// This happens when:
// - Code deploy mode is selected (ZIP upload, no container build)
// - Pre-built image is provided via --image flag (user manages their own registry)
// - The manifest is a prompt-voice agent (managed, no container image)
func (a *InitAction) skipACR() bool {
	return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent

Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/init.go
Previously --kind prompt-voice was silently ignored when --manifest was
also supplied, so a hosted manifest would create a hosted service despite
the user explicitly selecting the voice kind. Reject the combination
early, matching the existing --kind/--image validation.
Copilot AI review requested due to automatic review settings August 4, 2026 05:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.

Suppressed comments (3)

cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/map.go:530

  • [azd-code-reviewer] A whitespace-only model.id passes both manifest validation and this check, then is sent as the managed model name. Normalize the ID with strings.TrimSpace, reject the normalized empty value, and use the normalized value in the request; keep the schema and ValidateAgentDefinition checks aligned so hand-authored manifests fail before the service call.
	if voiceAgent.Model == nil || voiceAgent.Model.Id == "" {
		return nil, fmt.Errorf("model.id is required for a prompt-voice agent")

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788

  • [azd-code-reviewer] This introduces a supported environment override, but cli/azd/docs/environment-variables.md does not list it. That file is the repository's source of truth for environment variables; document whether this must be an azd environment value or shell variable, the expected host format, and that it is a temporary/internal routing override.
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1802

  • [azd-code-reviewer] The tests cover dispatch and the HTTP client separately, but never exercise this new deploy orchestration. Add a provider-level test that verifies the successful call writes both AGENT_<KEY>_NAME and AGENT_<KEY>_ENDPOINT and returns the endpoint artifact, plus failure cases for a missing project endpoint and environment persistence errors.
func (p *AgentServiceTargetProvider) deployVoiceAgent(
	ctx context.Context,
	serviceConfig *azdext.ServiceConfig,
	va agent_yaml.VoiceAgent,
	azdEnv map[string]string,
	progress azdext.ProgressReporter,
) (*azdext.ServiceDeployResult, error) {

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving against bea4d9c. Verified the three commits since 7f5667f.

4db597e pins flags.agentName after the interactive resolve, matching what resolveAgentNameFromManifestPointer already does, so runInitFromManifest short-circuits instead of prompting a second time. The mutation can't leak, since the only thing running after that switch is ejectInfraAfterInit, which never reads the name.

bea4d9c rejects --kind prompt-voice combined with --manifest before either synthesis fast path, mirroring the existing --image guard. That closes the silent-adopt hole Copilot flagged.

The merge of main didn't drift anything. Net diff against the merge base is still 27 files, +2123/-29, byte-identical to the pre-merge PR, so conflict resolution didn't alter this branch's own contribution. All three agentkind.IsPromptVoice call sites (deploy dispatch, Endpoints, next-step reader) still share one lookup with the same AGENT_DEFINITION_PATH precedence. Main's new setServiceEnvironment call in addToProject correctly doesn't apply to addVoiceAgentToProject, since EnvironmentVariables only exists on ContainerAgent and a voice agent has no way to declare env.

Build and vet are clean and the agents extension tests pass. The two internal/synthesis bicep-stale failures reproduce identically on main at 493e6a7, so they aren't from this branch.

One leftover error hint, inline.

Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/init.go Outdated
When 'azd ai agent init' for a prompt-voice (managed) agent runs inside an
existing azd project, add it as a new azure.ai.agent service to the current
azure.yaml (src/<name> layout), matching hosted and other agents, instead of
scaffolding a separate nested <name>/ project. Applies to both the interactive
voice menu path and the '--kind prompt-voice' fast path. A brand-new (empty)
init still creates the <name>/ project folder.
Copilot AI review requested due to automatic review settings August 4, 2026 07:33
Since --kind prompt-voice combined with --manifest is now rejected, drop the
'(or provide --manifest ...)' remediation that would walk the user into a dead
end. Addresses PR review feedback.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.

Suppressed comments (5)

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1286

  • [azd-code-reviewer] The prompt-voice compatibility check covers --image and --manifest, but container/code flags still pass through. --kind prompt-voice --deploy-mode container is silently ignored; with --deploy-mode code, validateCodeDeployFlags may require --runtime/--entry-point, after which all three values are still ignored because a VoiceAgent bypasses promptDeployMode. Reject deployment-mode-specific flags when --kind prompt-voice is selected so accepted CLI input is never discarded.
				if flags.image != "" {
					return exterrors.Validation(
						exterrors.CodeInvalidParameter,
						"--kind prompt-voice cannot be combined with --image",
						"a voice agent is managed and has no container image; drop --image",

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788

  • [azd-code-reviewer] This new deployment override is absent from cli/azd/docs/environment-variables.md, which is the repository's source of truth for extension environment variables. Document its expected host format, default routing behavior, and temporary/internal support status so users do not have to infer how to configure it from source.
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"

cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:969

  • [azd-code-reviewer] This kind probe now intercepts an invalid AGENT_DEFINITION_PATH before the existing container loader can classify it. For example, malformed YAML in a hosted override returns a raw YAML error here instead of the existing CodeInvalidAgentManifest error and recovery suggestion. Parse the explicit override through voiceAgentFromDefinitionFile first, then use agentkind only when resolving the service entry.
	isVoice, err := agentkind.IsPromptVoice(svc, projectRoot, agentDefinitionPath)
	if err != nil {
		return agent_yaml.VoiceAgent{}, false, err

cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:58

  • [azd-code-reviewer] Requiring the id property still permits model: { id: "" }, while both manifest validation and deployment reject an empty model ID. Add a minimum length so editor/schema validation matches the runtime requirement stated below.
        "id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145

  • [azd-code-reviewer] Extending skipACR to voice agents makes the current AZD_AGENT_SKIP_ACR entry in cli/azd/docs/environment-variables.md:161 inaccurate because it says the variable is set automatically only for code-deploy scenarios. Update that entry to include managed prompt-voice initialization.
	return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent

Copilot AI review requested due to automatic review settings August 4, 2026 07:45

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving against 03f8a6c. Verified both commits since bea4d9c.

7b30f47 lines up the prompt-voice target directory across the interactive and non-interactive paths. Inside an existing project both now hold targetDir at "." and append the service to the current azure.yaml, while a fresh init still creates the <name>/ folder and the cd hint. I walked all four combinations (interactive and flag-driven, crossed with existing and empty project) and they agree. Reordering the manifestInCwd branch keeps the old flags.src = "." behavior, and because --kind and --manifest are rejected together at line 1288, the synthesized voice manifest always lands in a temp dir, so manifestInCwd can't shadow the new voice branch.

03f8a6c drops the --manifest remediation from the prompt-voice hint, which matches that validation. Leaving the --image hint at line 1314 alone also tracks, since --image has no equivalent incompatibility check and still accepts --manifest.

Build, vet, and gofmt are clean here, and the cmd package tests pass.

Non-blocking, for a follow-up: the targetDir and folderDisplay decision now lives in three places: the non-interactive manifest branch, the template branch, and the interactive voice branch. That duplication is the same drift 7b30f47 is fixing, and the template branch still nests into <name>/ inside an existing project. The rest of this package factors decisions like this into a helper and unit tests it, the way synthesizeVoiceManifestFile and resolveInitAgentName are covered. Worth pulling this one out too so the next kind can't drift?

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.

Suppressed comments (4)

cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:58

  • [azd-code-reviewer] required: ["id"] still accepts id: "", while CreateVoiceAgentAPIRequest rejects an empty ID. Editors therefore report this configuration as valid only for deploy to fail. Add a minimum length to keep schema and runtime validation aligned.
        "id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788

  • [azd-code-reviewer] This adds a new extension environment setting, but it is absent from the azure.ai.agents table in cli/azd/docs/environment-variables.md. Users cannot discover that this expects a regional host in the active azd environment or that leaving it unset uses the default APIM route. Document it alongside the other extension-specific variables.
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145

  • [azd-code-reviewer] skipACR() now persists AZD_AGENT_SKIP_ACR=true for prompt-voice, but cli/azd/docs/environment-variables.md:161 still describes automatic use only for code deploy. Update that entry so the documented behavior matches generated voice projects.
	return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1651

  • [azd-code-reviewer] The added tests cover the synthesis helper and state predicates, but neither the interactive voice branch nor the --kind prompt-voice command path is driven through service creation. The conflict validation, target-directory handling, and inline AddService shape can regress while these tests still pass. Add an action-level test that exercises each entry path and asserts the generated voice service properties.
				case initModeVoice:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ext-agents azure.ai.agents extension ext-projects azure.ai.projects extension

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Voice-based prompt agents

4 participants