feat(examples): add coding agent harness example

[Agentic-JJ-Web3]

Part of #885. This is the first of three PRs — steps 1–3 of your outline
(harness loop, basic tools, running it with tests and the Burr UI). CLI and
permissions follow separately.

What's here

examples/coding-agent/ — an agent that reads files, writes files, lists a
directory, and runs shell commands, looping until the model answers without
requesting a tool or the step budget runs out.

The graph is human_input → create_prompt → call_llm, with four bound tool
actions fanning out of call_llm and looping back to it. Each tool is bound
separately rather than dispatched from a registry, so the Burr UI shows which
one ran at each step.

Decisions I made without you

You hadn't replied yet, so these are assumptions rather than agreements —
happy to change any of them:

1. Scripted LLM client by default. notebook.ipynb is a required file and
   has to execute, so the example falls back to a client replaying a fixed
   sequence of tool calls when OPENAI_API_KEY isn't set. Real OpenAI client
   when it is.
2. run_bash is not sandboxed. File paths are confined to a workspace
   directory, but a shell command can cd out. The README says so plainly,
   which leaves step 5 something concrete to fix.
3. Four tools, not three — added list_files, since an agent that can't
   see the directory can't orient itself.
4. One tool call per iteration. If the model requests several, the extras
   are dropped. Documented as a known simplification.

Notes

• validate_examples.py passes for coding-agent.
• The tool schema marks only parameters without defaults as required. The
  tool-calling example marks all of them, which misreports optional args —
  happy to open a separate issue for that.
• examples/hello-world-counter crashes at app.visualize() unless the
  Graphviz binary is installed, and passing a stale app_id after clearing
  ~/.burr raises a pydantic error on PointerModel.app_id. Both separate
  from this PR; can file them if useful.

[Agentic-JJ-Web3]

Opened #886 with steps 1–3. I made a few calls in the absence of answers to my questions above — they're listed in the PR description, happy to change any of them.

[DeepGandhi01]

Thanks for putting this together — the state-machine framing of the agent loop reads really well, and the scripted client makes it easy to try. I checked out this branch and ran it in a clean Python 3.14 venv built only from examples/coding-agent/requirements.txt (burr 0.42.0, openai 3.22.1), then probed the loop with ScriptedClient scripts. A few things came up that seem worth fixing before merge; each is reproducible with the scripted client and no API key.

1. Example fails to import from its own requirements

tools.py imports requests (and openai, inspect, json, typing, and burr.core, plus os twice), none of which it uses. requests is not in requirements.txt, so in a clean environment:

$ pip install -r requirements.txt && python application.py
  File ".../tools.py", line 25, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'

Suggested fix: drop the unused imports from tools.py (it only needs os and subprocess). With requests installed manually, the scripted run completes, which confirms this is the only blocker.

2. create_prompt only runs once per task, not before each LLM call

The README describes create_prompt as the seam "for shaping the prompt before each call", but the tool actions transition straight back to call_llm:

(["read_file", "write_file", "list_files", "run_bash"], "call_llm"),

Observed trace for one tool call: human_input → create_prompt → call_llm → list_files → call_llm → respond. Anyone adding a file tree or history summary in create_prompt (the extension the README suggests) would only see it applied to the first call. Routing tool actions back to create_prompt fixes it:

(["read_file", "write_file", "list_files", "run_bash"], "create_prompt"),

3. An unknown tool name halts the run silently

If the model requests a tool that isn't registered (common with real models), no transition out of call_llm matches, and app.run(halt_after=["respond"]) stops at call_llm with final_answer=None and no error. Trace: human_input → create_prompt → call_llm. One option is to validate in call_llm and feed an error back as a tool message, or add a fallback transition such as ("call_llm", "respond", default) after the tool transitions so the run always terminates in respond.

4. Tool errors crash the whole run instead of going back to the model

execute_tool calls tool_function(**state["next_args"]) directly, so:

• Wrong argument names from the model ({"filename": "x"} for read_file) raise TypeError: read_file() got an unexpected keyword argument 'filename'.
• run_bash before the workspace directory exists raises FileNotFoundError from subprocess.run(cwd=...) (only TimeoutExpired is caught). ./workspace isn't created until the first write_file, so a model that starts with run_bash crashes on the first step.

Since the example's point is that the agent "sees them fail, and tries again", catching exceptions in execute_tool and returning {"error": ...} as the tool message would make the loop self-correcting. Creating the workspace up front (os.makedirs(WORKSPACE, exist_ok=True)) fixes the second case directly.

5. Malformed tool-call JSON crashes OpenAIClient

json.loads(c.function.arguments) raises JSONDecodeError on malformed arguments, which models occasionally produce. Worth catching and returning an error tool message, same as (4).

6. Minor: _resolve can be escaped via symlinks

_resolve uses os.path.abspath, which doesn't resolve symlinks, so a symlink inside the workspace pointing outside it passes the check (read_file("link/secret.txt") returned a file from outside the workspace). The README is clear that this isn't a sandbox and run_bash can cd anywhere, so this is low severity, but os.path.realpath for both root and full would make _resolve do what its docstring says.

7. Nit

Without the Graphviz binary, app.visualize() in __main__ prints a ModuleNotFoundError: No module named 'graphviz' traceback before the run (it still completes). Wrapping it in a try/except ImportError or only visualizing on request would keep first-run output clean.

Happy to help with any of these or open a follow-up PR against this branch if that's useful.

[skrawcz]

Thanks for putting this together, and apologies for the delayed maintainer response. Your initial design choices make sense:

• The scripted client fallback is the right default so the example runs without credentials and can be tested in CI.
• Unsandboxed run_bash is acceptable for this first teaching stage, provided the warning remains prominent; permissions are intentionally a later step.
• Adding list_files is useful.
• One tool call per iteration is acceptable as a clearly documented simplification.
• The proposed sequence of separate PRs also makes sense.

The functional findings from @DeepGandhi01 should be addressed before merge: clean-install imports, routing back through create_prompt, unknown-tool handling, tool/argument failures, workspace creation, and the symlink confinement issue.

Two additional requirements:

1. examples/coding-agent/README.md needs the standard ASF license header; otherwise Apache RAT will reject the release.
2. This PR should include automated tests for the scripted happy path, step-budget termination, unknown tools, invalid tool arguments/tool failures, and workspace confinement. Testing the harness was part of the original issue, and currently CI does not exercise it.

The inactivity bot removed the assignment today, but I’d still like to move this contribution forward. Are you interested in making this follow-up? If useful, I’m also happy to push a cleanup commit to your branch since maintainer edits are enabled.

[haigou-web]

Following up on the review above — I opened a PR against this branch with the fixes and the tests:

Agentic-JJ-Web3#1 (base: docs/coding-agent-harness-example)

It covers all seven findings from @DeepGandhi01 and the two requirements from @skrawcz:

Finding	Fix
1. Clean-install imports	dropped the unused requests / openai / inspect / json / typing / burr.core imports and the duplicate os in tools.py
2. create_prompt only ran once	tool actions now route back through create_prompt, so the seam runs before every model call
3. Unknown tool halted the run	the error is fed back to the model as a tool message and the loop retries
4. Tool errors crashed the run	execute_tool catches and returns an error result; run_bash creates the workspace first
5. Malformed tool-call JSON	JSONDecodeError is caught and surfaced as a tool error
6. _resolve symlink escape	realpath for both root and full
7. Graphviz nit	visualize() wrapped in try/except
ASF header	added to README.md
Tests	new test_coding_agent.py — 10 tests, driven by ScriptedClient, no API key needed

Verified locally: pytest test_coding_agent.py → 10 passed; pytest validate_examples.py → 48 passed; python application.py runs clean without credentials; black 23.11.0 / isort 5.12.0 / flake8 6.1.0 clean.

The PR's base is this branch rather than main, so its diff is only the follow-up — the example itself is unchanged from #886.

Happy to adjust anything, and if it's easier to fold this straight into #886 rather than merging the follow-up, that works too.

[Agentic-JJ-Web3]

Yes, I’d like to make the follow-up — apologies for the slow response. @DeepGandhi01’s findings are all fair and I’ll work through them: the imports, routing tool actions back through create_prompt, unknown-tool handling, catching tool and argument failures, creating the workspace up front, realpath in _resolve, and the graphviz guard. Plus the ASF header and the test file. Should have something up this week.

[Agentic-JJ-Web3]

Thanks both — @DeepGandhi01 that review was genuinely useful, and all seven
findings were real.

Pushed fixes for everything, plus the two requirements:

Finding	Fix
1. Clean-install imports	removed the unused imports from tools.py; verified with a fresh venv built only from requirements.txt
2. create_prompt ran once	tool actions now route back through create_prompt; there's a test asserting it runs twice for a one-tool run
3. Unknown tool halted silently	the error goes back to the model as a tool message so it retries, plus a default transition to respond as a safety net
4. Tool errors crashed the run	execute_tool catches and returns {"error": ...}; workspace is created at import so run_bash works first
5. Malformed tool-call JSON	caught and surfaced as a bad argument
6. _resolve symlink escape	realpath for both root and full
7. Graphviz nit	visualize() wrapped in try/except
ASF header	added to README.md
Tests	test_coding_agent.py — 10 tests via ScriptedClient, no API key

Also regenerated notebook.ipynb from the fixed sources — it inlined the old
code, so it would have shipped the bugs this PR removes.

Verified: pytest test_coding_agent.py → 10 passed; validate_examples.py →
passes for coding-agent; pre-commit clean; python application.py runs
without credentials.

@haigou-web thanks for the offer on Agentic-JJ-Web3#1 — I've worked through
these directly, so I'll close that one.

[Agentic-JJ-Web3]

The documentation workflow shows a startup failure — that’s the ASF allowlist rejecting hasura/comment-progress@v2.2.0 before any job runs, not anything in this change. Looks like it affects all PRs.