fix(agents): fail fast on unready Foundry dependencies

[hund030]

Summary

Fixes #8587.

Previously, agent deployment succeeded even when required Foundry resources were unavailable, producing an agent that failed at invocation time. This change validates declared dependencies before creating the agent version and returns actionable provisioning, deployment, configuration, or migration guidance when they are not ready.

Changes

• Agent deployment: recursively validate enabled uses: dependencies for projects, connections, toolboxes, skills, and agents.
• Readiness state: publish project-scoped markers for connections, toolboxes, skills, and agents, and clear mutable markers during updates and deletion.
• Legacy manifests: reject unavailable bundled toolboxes with guidance to migrate them to azure.ai.toolbox services instead of silently dropping them.

Design

Core uses: edges continue to control deployment ordering; the agent target adds semantic readiness validation immediately before mutation. Provision-time dependencies (azure.ai.project and azure.ai.connection) direct users to azd provision, while deploy-time dependencies direct users to azd deploy <service> or azd deploy --all.

Readiness markers are scoped to FOUNDRY_PROJECT_ENDPOINT so state from another Foundry project cannot satisfy validation. Existing endpoint markers remain compatible when their URL proves they belong to the active project.

Manual Validation

• Built and installed the branch extension, then deployed an agent referencing an undeployed azure.ai.toolbox service against an existing Foundry project.
• Deployment stopped before creating the agent with foundry_dependency_not_ready, named the missing TOOLBOX_REVIEW_TOOLS_MCP_ENDPOINT, and suggested deploying review-tools before retrying the agent.

[github-actions]

📋 Prioritization Note

Thanks for the contribution! The linked issue isn't in the current milestone yet.
Thank you for logging this issue; our team is reviewing it. If you need urgent prioritization, tag @RickWinter and @kristenwomack to let us know.

[azure-pipelines]

Azure Pipelines:
Successfully started running 4 pipeline(s).
18 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Adds fail-fast validation for unavailable Foundry dependencies before agent deployment.

Changes:

• Recursively validates enabled uses: dependencies.
• Publishes project-scoped readiness markers.
• Rejects unavailable legacy bundled toolboxes with migration guidance.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
azure.ai.toolboxes/.../envkey.go	Adds toolbox project marker keys.
azure.ai.toolboxes/.../envkey_test.go	Tests marker key generation.
azure.ai.toolboxes/.../toolbox_env.go	Publishes scoped toolbox readiness state.
azure.ai.toolboxes/.../toolbox_env_test.go	Tests endpoint scope extraction.
azure.ai.skills/.../envkey.go	Defines skill readiness keys.
azure.ai.skills/.../envkey_test.go	Tests skill keys.
azure.ai.skills/.../skill_delete.go	Clears skill readiness markers.
azure.ai.skills/.../skill_delete_test.go	Adds cleanup coverage.
azure.ai.skills/.../service_target.go	Publishes skill deployment markers.
azure.ai.skills/.../service_target_test.go	Tests marker publication.
azure.ai.projects/.../foundry_provisioning_provider.go	Scopes connection outputs to projects.
azure.ai.agents/.../service_target_agent.go	Runs validation and publishes agent markers.
azure.ai.agents/.../service_target_agent_test.go	Tests condition lookup and agent markers.
azure.ai.agents/.../foundry_dependencies.go	Implements dependency readiness validation.
azure.ai.agents/.../foundry_dependencies_test.go	Covers dependency and remediation scenarios.
azure.ai.agents/.../envkey.go	Defines shared readiness keys.
azure.ai.agents/.../envkey_test.go	Tests readiness keys.
azure.ai.agents/.../codes.go	Adds the dependency-not-ready error code.

[Copilot]

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (5)

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:420

• The condition lookup precedence is opposite to azd core. environment.Environment.Getenv checks the azd .env first (cli/azd/pkg/environment/environment.go:188-198), but this checks the process environment first. If both define a condition differently, core can skip a dependency while this validator requires it, or this validator can skip readiness for a dependency core deployed. Read dependencyEnv first and only then fall back to the process environment.

	if value, ok := os.LookupEnv(name); ok {
		return value
	}
	return p.dependencyEnv[name]

cli/azd/extensions/azure.ai.skills/internal/cmd/skill_delete_test.go:29

• This test calls the stub directly, so it passes even if deleteAction.Run never invokes marker cleanup, invokes it before a failed delete, or passes the wrong skill name. Exercise the delete action through an injectable resolver/client (or extract a testable delete core) and assert cleanup occurs only after API success.

	// The API path is covered by client tests; this seam asserts deletion owns
	// marker cleanup without requiring a live Foundry endpoint.
	require.NoError(t, clearSkillMarkersFunc(t.Context(), "my-skill"))
	require.True(t, called)

cli/azd/extensions/azure.ai.agents/internal/project/foundry_dependencies.go:178

• The remediation collapse produces dead-end suggestions for configuration failures: a disabled runtime toolbox and a toolbox service missing from uses both receive an azd deploy suggestion, which cannot change the condition or graph wiring and will repeat the same error. The migration branch also suppresses azd provision when migration and provision failures are aggregated. Preserve per-failure remediation so these cases instruct users to enable/remove the dependency, add the uses edge, and run every required command.

	suggestion := "run 'azd deploy --all', then retry the agent deployment"
	if requiresMigration {
		suggestion = "migrate bundled toolboxes to azure.ai.toolbox services, run 'azd deploy --all', " +
			"then retry the agent deployment"
	} else if requiresProvision && !requiresDeploy {

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:2718

• This adds AGENT_*_PROJECT_ENDPOINT, but whole-agent deletion still clears only NAME, VERSION, and ENDPOINT (internal/cmd/delete.go:212-240). The stale scope marker can later override the legacy endpoint fallback and reject a valid redeployment from an older compatible publisher. Add this key to deletion cleanup and cover it in the delete tests.

		azdext.SetEnvRequest{EnvName: p.env.Name, Key: envkey.AgentProjectEndpoint(serviceConfig.Name), Value: projectEndpoint},

cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_env.go:58

• The empty-value path is used after whole-toolbox deletion, but it clears only the MCP endpoint and leaves the new project-scope marker behind. Clear the commit marker first, then clear the project marker so deleted readiness state cannot leak into a later compatible deployment.

		if value == "" {
			return setValue(commitKey, "")

[Copilot]

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_env.go:65

• [azd-code-reviewer] This fallback scopes an endpoint to the active project even when the URL does not prove that relationship. endpoint: accepts any non-empty expanded string, so a noncanonical or unrelated MCP URL without /toolboxes/ gets a matching project marker and then passes the new readiness check. Reject endpoints whose project cannot be derived, or otherwise verify them against FOUNDRY_PROJECT_ENDPOINT before publishing the commit marker.

		if projectEndpoint == "" {
			projectResp, err := c.Environment().GetValue(ctx, &azdext.GetEnvRequest{
				EnvName: envName, Key: "FOUNDRY_PROJECT_ENDPOINT",

cli/azd/extensions/azure.ai.skills/internal/cmd/skill_delete.go:68

• [azd-code-reviewer] The deleted skill may come from --project-endpoint, but cleanup always blanks markers in the active azd environment. Deleting my-skill from project B while the active environment tracks the same-named skill in project A therefore removes A's valid readiness state. Pass skillCtx.endpoint into cleanup and only clear markers whose stored project endpoint matches the deletion target.

	if err := deleteSkillAndClearMarkers(ctx, skillCtx.client, a.flags.name); err != nil {

cli/azd/extensions/azure.ai.agents/internal/project/foundry_dependencies_test.go:214

• [azd-code-reviewer] This added line exceeds the repository's 125-character Go limit (cli/azd/AGENTS.md:97-108), so the lll preflight check will fail. Split the URL across concatenated strings.

		envkey.ToolboxMCPEndpoint("legacy-tools"): "https://account.services.ai.azure.com/api/projects/old/toolboxes/legacy-tools/mcp",

[jongio]

A few things worth a look before this merges.

• foundry_dependencies.go:243 - the skill readiness check reads a marker that's written from a different resolution chain than FOUNDRY_PROJECT_ENDPOINT, so a successful skill deploy can produce a marker the agent deploy then rejects.
• toolbox_env_test.go - five existing tests and a helper were removed, including coverage for code this PR doesn't touch.
• skill_delete_test.go:59 - exercises the stub rather than any production code.
• service_target_agent.go:409 - the condition truthiness list is duplicated from azd core with nothing keeping the two in sync.
• delete.go:172 - the version marker gets cleared, but AGENT_<KEY>_ENDPOINT keeps pointing at the deleted version.

Details inline.

[Copilot]

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.

Suppressed comments (3)

cli/azd/extensions/azure.ai.agents/internal/cmd/delete.go:261

• [azd-code-reviewer] Legacy agent markers have no AGENT_*_PROJECT_ENDPOINT; readiness intentionally falls back to AGENT_*_ENDPOINT when that URL identifies the active project. This cleanup instead returns on the empty project marker, so deleting the marked legacy version leaves its version and endpoint values behind and later validation can still treat the deleted version as ready. Apply the same base-endpoint fallback before checking the project scope.

	projectKey := envkey.AgentProjectEndpoint(serviceName)
	projectResp, err := azdClient.Environment().GetValue(ctx, &azdext.GetEnvRequest{
		EnvName: envResp.Environment.Name,
		Key:     projectKey,
	})
	if err != nil || !sameAgentProjectEndpoint(projectResp.Value, deletedProjectEndpoint) {
		return

cli/azd/extensions/azure.ai.agents/internal/project/foundry_dependencies.go:308

• [azd-code-reviewer] This compares the provisioned connection list with the azure.yaml service key, but provisioning emits the configured Foundry connection name. collectConnections preserves that configured name and only defaults it to svc.Name when empty (internal/cmd/resource_services.go:527-535), so an aliased connection service is provisioned successfully but is always reported as unready here. Resolve the dependency's effective connection name before checking AZURE_AI_PROJECT_CONNECTION_NAMES.

	for name := range strings.SplitSeq(env["AZURE_AI_PROJECT_CONNECTION_NAMES"], ",") {
		if strings.TrimSpace(name) == service.GetName() {
			return ""

cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_delete.go:193

• [azd-code-reviewer] Cleanup only runs for a cascading last-version deletion. If the marker points to version 1, the user publishes version 2 as default, then deletes now-non-default version 1, this leaves the MCP marker pointing at a deleted URL. Since dependency validation checks only that the marker is nonempty and project-scoped, a dependent agent can still deploy and fail at invocation. On every version deletion, clear the markers when the stored endpoint references the deleted version, while preserving markers for other versions.

	if cascaded {
		if err := setToolboxEndpointEnvFunc(ctx, name, "", client.Endpoint()); err != nil {

[jongio]

Two things in 325d867, both about how the new project guard behaves when the project marker isn't set.

delete.go:260 bails when AGENT_<KEY>_PROJECT_ENDPOINT is empty, so version deletes on agents deployed before that marker existed now leave AGENT_<KEY>_VERSION behind. That's the stale-readiness case this PR is closing.

delete.go:225 still clears markers on the full delete path without checking the project.

[Copilot]

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated 1 comment.

Suppressed comments (4)

cli/azd/extensions/azure.ai.agents/internal/project/foundry_dependencies.go:274

• [azd-code-reviewer] Empty endpoints compare equal here. If a stale readiness value exists while both its project marker and FOUNDRY_PROJECT_ENDPOINT are empty, toolbox/skill/agent validation can report the dependency ready and continue to mutation. Require a non-empty normalized endpoint before taking the direct-equality path.

	if strings.EqualFold(
		strings.TrimRight(strings.TrimSpace(a), "/"),
		strings.TrimRight(strings.TrimSpace(b), "/"),
	) {

cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_env.go:105

• [azd-code-reviewer] Using the first /toolboxes/ segment misparses a valid project named toolboxes: /api/projects/toolboxes/toolboxes/tools/... is truncated to /api/projects, so the published scope never matches the active project. Use the final resource-collection segment and add this project-name case to the test.

	index := strings.Index(parsed.Path, segment)

cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_delete.go:195

• [azd-code-reviewer] Marker cleanup only runs when deleting the last version. A concrete stale-marker path is: deploy v1 (publishes a v1 MCP URL), publish v2 (does not rewrite that URL), then delete non-default v1. The marker still points to the deleted version, and the new readiness check accepts it because it is non-empty and project-scoped. Also clear the markers when the stored MCP endpoint references the version being deleted.

	if cascaded {
		if err := setToolboxEndpointEnvFunc(ctx, name, "", client.Endpoint()); err != nil {
			return err
		}

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:425

• [azd-code-reviewer] GetServiceConfigValue reads the raw azure.yaml map and does not resolve service $ref files (internal/grpcserver/project_service.go:616-625). A dependency whose condition is defined in its referenced file is therefore treated as enabled here, even though core resolves that condition and skips it, causing a false readiness failure. Read the condition from the resolved service configuration or make the lookup follow the same include semantics as core.

	resp, err := p.azdClient.Project().GetServiceConfigValue(ctx, &azdext.GetServiceConfigValueRequest{

[jongio]

Re-checked against 7887e93. Both items from my last review are fixed and verified.

The full delete path now passes the resolved project endpoint into cleanupEnvVars and gates on it, so an agent that lives in a different project keeps its markers. The version path and the whole-agent path both route through agentMarkersBelongToProject, so the two guards can't drift apart.

The legacy fallback is the part I wanted to confirm. When AGENT_<KEY>_PROJECT_ENDPOINT is unset, the helper falls back to AGENT_<KEY>_ENDPOINT, which service_target_agent.go:2718 writes as <project endpoint>/agents/<name>/versions/<version>. agentProjectIdentity pulls the host and project name out of that, so pre-existing environments get cleaned up without needing a redeploy first.

Ran TestDeleteMarkerCleanup against the branch locally: all five subtests pass, including the two new ones. go build ./... and go vet come back clean.

One residual worth knowing about, not a blocker. If both scope markers are empty the guard fails closed and cleanupEnvVars skips everything, so AGENT_<KEY>_NAME can survive a delete. The reachable path is a legacy agent where a version delete clears AGENT_<KEY>_ENDPOINT first, leaving nothing for the fallback to read. validateFoundryAgentDependency needs both NAME and VERSION, so this can't report ready by mistake. It just leaves a stale name that azd agent doctor will warn about.

[Copilot]

Pull request overview

Copilot reviewed 32 out of 32 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_env.go:121

• [azd-code-reviewer] This strict full-URL comparison preserves readiness markers when the same Foundry project is addressed through supported endpoint aliases, such as /api/projects/p versus /projects/p. The agent and skill cleanup paths compare hostname plus project identity; use equivalent matching here so deleting through an alias cannot leave a stale toolbox marker that later passes dependency validation.

	return deletionProject != "" && strings.EqualFold(markerProject, deletionProject)

cli/azd/extensions/azure.ai.agents/internal/cmd/delete.go:360

• [azd-code-reviewer] These write failures are only logged even though NAME/VERSION now serve as semantic readiness markers. If deletion succeeds while these clears fail, the deleted agent still passes validateFoundryAgentDependency; retrying deletion can return not-found without repairing the environment. Invalidate the commit marker before the delete and propagate cleanup failures instead of reporting successful deletion with stale readiness state.

		if _, err := azdClient.Environment().SetValue(ctx, &azdext.SetEnvRequest{
			EnvName: envName,
			Key:     key,
			Value:   "",
		}); err != nil {

[jongio]

Re-checked against 9799185. The Terraform gap is closed and verified end to end.

Both outputs.tf.tmpl copies now emit AZURE_AI_PROJECT_CONNECTIONS_PROJECT_ENDPOINT using the same expression as FOUNDRY_PROJECT_ENDPOINT, and the two files are byte identical, so validateFoundryConnectionDependency (foundry_dependencies.go:303) gets a marker that satisfies sameProjectEndpoint after azd provision on the Terraform path. Assertions cover both the embedded template (synthesizer_test.go in the agents and projects extensions) and the rendered outputs.tf (init_infra_test.go:895).

I also checked the Bicep side, since the same key is required there. Bicep eject does not stamp infra.provider, so the microsoft.foundry provider still runs provisioning and withTenantOutput (foundry_provisioning_provider.go:557) derives the key from FOUNDRY_PROJECT_ENDPOINT on every ARM and brownfield output path (809, 837, 906, 971, 1036). No template change is needed there.

One non-blocking note: the scoped output duplicates the FOUNDRY_PROJECT_ENDPOINT expression instead of sharing a local, and the tests only assert the output exists, not that the two values agree. If the account or path shape changes in one and not the other, connection validation fails after a clean provision with nothing pointing at the cause. main.tf already has a locals block if you want to pull it up later.

CI is green. Re-approving.

[Copilot]

Pull request overview

Copilot reviewed 32 out of 32 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

cli/azd/extensions/azure.ai.agents/internal/cmd/delete.go:175

• [azd-code-reviewer] Cleanup is reached only after Azure reports a successful delete. If the agent/version was already removed externally and the API returns 404, the command exits before clearing AGENT_*_VERSION; dependency validation then continues treating the missing agent as ready. The toolbox and skill deletion paths added in this PR explicitly attempt scope-safe cleanup on not-found. Apply the same behavior to both agent and version deletion paths.

		a.clearDeletedVersionMarker(ctx, azdClient, info.ServiceName, a.flags.version, endpoint)

cli/azd/extensions/azure.ai.agents/internal/project/foundry_dependencies_test.go:226

• [azd-code-reviewer] This added line exceeds the repository's enforced 125-character Go line limit (cli/azd/.golangci.yaml:26-28), so golangci-lint will reject it. Split the URL literal across lines.

		envkey.ToolboxMCPEndpoint("legacy-tools"): "https://account.services.ai.azure.com/api/projects/old/toolboxes/legacy-tools/mcp",

[Copilot]

Pull request overview

Copilot reviewed 32 out of 32 changed files in this pull request and generated no new comments.

Suppressed comments (2)

cli/azd/extensions/azure.ai.agents/internal/cmd/delete.go:201

• [azd-code-reviewer] Whole-agent cleanup is only reached after a successful delete. If the agent was removed out of band and the API returns 404, its NAME/VERSION markers remain and dependent deployments incorrectly treat it as ready. Handle StatusNotFound by running this project-scoped cleanup before returning the not-found error, matching the toolbox and skill delete paths.

	// Best-effort: clear readiness and endpoint state after a successful delete.
	a.cleanupEnvVars(ctx, azdClient, info.ServiceName, endpoint)

cli/azd/extensions/azure.ai.agents/internal/cmd/delete.go:175

• [azd-code-reviewer] A 404 from DeleteAgentVersion returns before this cleanup, so a marker for that already-absent version remains and can still satisfy dependency validation. Clear the scoped marker on StatusNotFound as well; clearDeletedVersionMarker already verifies both the version and project before mutating state.

This issue also appears on line 200 of the same file.

		a.clearDeletedVersionMarker(ctx, azdClient, info.ServiceName, a.flags.version, endpoint)

[Copilot]

Pull request overview

Copilot reviewed 33 out of 33 changed files in this pull request and generated no new comments.

Suppressed comments (4)

cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_env.go:118

• [azd-code-reviewer] Cleanup compares raw project URLs, while the agent and skill implementations compare Foundry project identity. The toolbox endpoint validator accepts noncanonical paths with a warning, so a toolbox created under /api/projects/p and deleted after the same project is configured as /projects/p is deleted remotely but retains its readiness markers. Those stale markers then pass sameProjectEndpoint during agent validation. Compare hostname and project name here, as the sibling cleanup implementations do.

func shouldClearToolboxMarkers(_ string, markerEndpoint, deletionProject string) bool {
	deletionProject = strings.TrimRight(strings.TrimSpace(deletionProject), "/")
	resourceProject := toolboxProjectEndpoint(markerEndpoint)
	return deletionProject != "" && strings.EqualFold(resourceProject, deletionProject)

cli/azd/extensions/azure.ai.agents/internal/cmd/delete.go:175

• [azd-code-reviewer] Marker cleanup only runs after a successful delete. If the agent or version was already removed out of band, the 404 returns through classifyDeleteError before this call (and before whole-agent cleanup), leaving AGENT_*_VERSION and the project marker intact. validateFoundryAgentDependency can then accept a nonexistent agent as ready. Treat 404 as already deleted and run the same project-scoped cleanup, matching the new toolbox and skill deletion behavior.

		a.clearDeletedVersionMarker(ctx, azdClient, info.ServiceName, a.flags.version, endpoint)

cli/azd/extensions/azure.ai.agents/internal/project/foundry_dependencies.go:55

• [azd-code-reviewer] This only validates the agent's direct uses: entries, despite the PR contract requiring recursive validation. A targeted azd deploy <agent> selects only that service (pkg/project/importer.go:114-132), so the producer is not redeployed to validate its own dependencies. For example, if agent A uses agent B and B's toolbox was deleted after B was deployed, B's marker lets A pass even though the transitive toolbox is unavailable. Walk enabled service-to-service uses: recursively with a visited set and validate each reachable Foundry dependency.

	for _, dependencyName := range agent.GetUses() {

cli/azd/extensions/azure.ai.agents/internal/project/foundry_dependencies.go:299

• [azd-code-reviewer] This provisioning output remains stale when azd ai connection delete removes one connection: that command does not update AZURE_AI_PROJECT_CONNECTION_NAMES. A later agent deployment therefore still treats the deleted connection as ready and misses the failure this PR is intended to catch. Update the connection deletion path to remove the scoped name (including the already-absent case), or validate the connection against the service instead of trusting the stale list.

	for name := range strings.SplitSeq(env["AZURE_AI_PROJECT_CONNECTION_NAMES"], ",") {
		if strings.TrimSpace(name) == service.GetName() {
			found = true
			break
		}

[jongio]

I found four remaining issues. Three can report missing resources as ready or disagree with the deployment graph, so they block the readiness contract; the fourth makes an idempotent delete silently succeed.

_{🤖 agent jongio}

[jongio]

[azd-code-reviewer] NumberValue no longer retains the YAML spelling, so formatting it maps both condition: 1.0 and condition: 1e0 to 1. Core evaluates its string condition with only literal 1 truthy, so this validator can require a dependency that the deployment graph skips. Preserve the raw condition scalar through the Project API, or expose core's enabled result, and add parity tests for 1.0 and 1e0.

[jongio]

[azd-code-reviewer] This legacy bypass cannot distinguish a pre-marker deployment from a skill that was never deployed or was deleted: both marker keys are empty and this returns ready. That defeats fail-fast validation for azd deploy <agent>. Validate the skill remotely, or fail closed and require a one-time redeploy or migration; empty markers need independent proof of readiness.

[jongio]

[azd-code-reviewer] markerProject is written for explicit reuse but ignored here. A BYO endpoint has no /toolboxes/, so parsing markerEndpoint yields empty and deletion skips cleanup even when markerProject matches deletionProject. The stale endpoint and project marker then keep the deleted toolbox ready. Compare markerProject first and only fall back to deriving scope from the endpoint for legacy markers; add a BYO reuse and delete test.

[jongio]

[azd-code-reviewer] The 404 path reports already_deleted, but emitDeleteResult has no default-output case for that outcome. JSON reports it, while normal output succeeds silently. Add an already_deleted rendering case and a default-output test; retain exit code 0 if deletion is intentionally idempotent.