ci: verify examples and Copilot integration tests

[Michelle Clayton (michelle-clayton-work)]

Summary

Add scheduled and manually dispatched CI that runs every example supported by the configured credentials, then runs dedicated GitHub Copilot provider integration tests.

The first provider covered is GitHub Copilot. The workflow uses the short-lived ${{ github.token }} with copilot-requests: write, so it does not require a repository secret, API key, Foundry project, or Azure setup. Examples for providers without configured credentials skip with their existing reasons.

Copilot integration tests

Port an initial safe subset of GitHubCopilotAgentTests.cs from the .NET Agent Framework:

• nonstreaming response
• streaming response
• function-tool invocation

The tests live in provider/copilotprovider/agent_e2e_test.go. Ordinary test runs skip them. GitHub Actions runs them when GITHUB_TOKEN is available, and developers can opt in locally with RUN_COPILOT_INTEGRATION_TESTS=true.

Shell, URL, and MCP integration cases are left for follow-up because they add external side effects or dependencies.

Example verification

Run the existing cmd/verifyexamples command without a name filter. In the current Copilot-only configuration, the live workflow ran 24 available examples and skipped 83 unavailable examples with explicit reasons.

Two fixed-input number-guessing examples now use their existing deterministic output checks instead of requiring Foundry semantic grading.

Authentication and safety

• The checkout does not persist Git credentials.
• Copilot receives the job-scoped GitHub Actions token only in the two live test steps.
• The integration tests disable config discovery, file hooks, git operations, session storage, and skills.
• The function-tool test exposes only its in-memory weather tool.
• The workflow uploads example and integration-test logs even when either phase fails.

Validation

• go test -count=1 ./...
• go vet ./cmd/verifyexamples ./provider/copilotprovider
• YAML parsing and git diff --check
• Live workflow run: https://github.com/microsoft/agent-framework-go/actions/runs/33197392893

The live run passed all 24 available examples and all three Copilot integration tests.

Upstream reference

• https://github.com/microsoft/agent-framework/blob/main/dotnet/tests/Microsoft.Agents.AI.GitHub.Copilot.IntegrationTests/GitHubCopilotAgentTests.cs

[Copilot]

Pull request overview

Adds a scheduled GitHub Actions workflow that runs a minimal GitHub Copilot-backed example end-to-end via the existing cmd/verifyexamples harness, to continuously validate the Copilot provider integration without external secrets or Azure/Foundry setup.

Changes:

• Add a new Copilot “function tool canary” example that exposes a fixed tool result and prompts Copilot to include it in the final response.
• Register the new example in cmd/verifyexamples with output assertions to validate the run.
• Add a new scheduled/manual GitHub Actions workflow to execute the example and upload logs/reports as artifacts.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
examples/02-agents/providers/github-copilot/function_tool/main.go	New minimal Copilot-backed agent example with a single fixed function tool and locked-down session config.
examples/02-agents/providers/github-copilot/function_tool/main_test.go	Unit test to verify the function tool returns the expected canary string.
cmd/verifyexamples/examples.go	Adds the Copilot canary example to the 02-agents verifyexamples set with must-contain / must-not-contain assertions.
.github/workflows/live-examples.yml	New scheduled/manual workflow that installs Copilot CLI, runs verifyexamples for the canary, writes a job summary, and uploads artifacts.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

API Consistency Review

• Scope: internal-only (CI workflow, internal tooling, integration tests)
• Changed Go contract: None — no exported identifiers or user-visible behavior changed
• Upstream evidence reviewed: dotnet/tests/Microsoft.Agents.AI.GitHub.Copilot.IntegrationTests/GitHubCopilotAgentTests.cs (linked in PR description) — the three ported integration test cases (non-streaming, streaming, function-tool invocation) map directly to the upstream test class and cover the same scenarios
• Result: Out of scope for parity review

This PR adds a CI workflow, modifies internal example-verification tooling (cmd/verifyexamples), and adds integration tests. None of these changes export new Go symbols or alter user-observable runtime behavior. The parity-approved label is correctly set and requires no change.

Generated by Go API Consistency Review Agent for #931 · sonnet46 · 14 AIC · ⌖ 4.96 AIC · ⊞ 6.4K · ◷

[Quim Muntal (qmuntal)]

Like the direction 😸

[Quim Muntal (qmuntal)]

Port all tests from https://github.com/microsoft/agent-framework/blob/main/dotnet/tests/Microsoft.Agents.AI.GitHub.Copilot.IntegrationTests/GitHubCopilotAgentTests.cs.

[Quim Muntal (qmuntal)]

The integration tests should run as part of the test workflow, they are orthogonal to the examples.