docs: DOC-1395: Devin skill to write better code examples - #8340
Open
elijahpetty wants to merge 2 commits into
Open
docs: DOC-1395: Devin skill to write better code examples#8340elijahpetty wants to merge 2 commits into
elijahpetty wants to merge 2 commits into
Conversation
elijahpetty
added
documentation
Contributor
No docs changes detected for 0151639 |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds Devin guidance for creating and validating Deephaven documentation examples against live workers.
Changes:
- Adds a detailed verification and debugging skill.
- Adds a concise workflow entry point.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
.devin/workflows/verified-example.md |
Defines the verification workflow. |
.devin/skills/verified-code-example/SKILL.md |
Provides worker, execution, diagnosis, and cleanup instructions. |
Suppressed comments (4)
.devin/skills/verified-code-example/SKILL.md:46
- Both MCP tools accept the session as
id;session_idis not part of their contracts. As written, environment discovery fails for every selected worker.
mcp1_session_pip_list(session_id="<session_id>")
mcp1_session_tables_list(session_id="<session_id>")
.devin/skills/verified-code-example/SKILL.md:65
session_script_runrequires the session argument to be namedid. Withsession_id, the workflow cannot execute the example, which defeats the skill's primary verification step.
mcp1_session_script_run(session_id="<session_id>", script="<the example, verbatim>")
.devin/skills/verified-code-example/SKILL.md:73
- These calls use the unsupported
session_idargument, and the schema tool is actually singular (session_table_schema) and accepts onetable_name, nottable_names. This output-verification block cannot run against the current Deephaven MCP API.
mcp1_session_tables_list(session_id="<session_id>")
mcp1_session_table_data(session_id="<session_id>", table_name="<table>", max_rows=10)
mcp1_session_tables_schema(session_id="<session_id>", table_names=["<table>"])
.devin/skills/verified-code-example/SKILL.md:78
- Renaming every published variable with a unique prefix conflicts with the reference-doc convention requiring table names
sourceandresult(docs/python/templates/_style-guide.md:70and_reference-template.md:57-62) and with the requirement to execute the published code verbatim. Prefix only verification-only artifacts; use an isolated worker or ask before running when published names collide.
Use a unique prefix for table and variable names so you do not collide with the user's existing session state, and mention any names you created in your final response.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| 2. Pick a candidate session. Prefer a `COMMUNITY` session whose `programming_language` matches the example language. Verify it is actually reachable: | ||
|
|
||
| ``` | ||
| mcp1_session_details(session_id="<session_id>", attempt_to_connect=true) |
| ``` | ||
| mcp0_docs_chat( | ||
| prompt="<the error message, plus the line that produced it, plus what the code is trying to do>", | ||
| programming_language="python", |
| **If no session is reachable, stop and prompt the user.** Do not write the example, do not guess, and do not silently skip verification. Tell the user exactly what is missing and offer the ways to get a worker running: | ||
|
|
||
| - Start the local server from this repo: `./gradlew server-jetty-app:run` (add `-Pgroovy` for a Groovy worker). The PSK is printed in the startup log. | ||
| - Or start a container: `docker compose up` from the repo root. |
|
|
||
| ## 3. Write the example | ||
|
|
||
| Follow the repo documentation style guide at `docs/python/templates/_style-guide.md`. |
| - Include every import the snippet needs. A reader must be able to paste the block and run it. | ||
| - Use meaningful variable and table names. | ||
| - Keep the example self-contained: generate or synthesize its own data unless the point of the example is reading from a specific external source. | ||
| - Match the fenced-block attributes the docs use (`python`, `python skip-test`, `python syntax`, `python test-set=N order=...`) to how the example will actually be tested. |
|
|
||
| 3. Follow the skill's steps in order. The non-negotiable parts: | ||
| - Confirm a live Deephaven worker over MCP **before** writing any code. If none is reachable, stop and prompt the user to start one. | ||
| - Run the exact example you intend to publish, then inspect its output — not just its exit code. |
| mcp1_session_details(session_id="<session_id>", attempt_to_connect=true) | ||
| ``` | ||
|
|
||
| 3. Check the response. Continue only if `success` is `true`, `available` is `true`, and `liveness_status` is `"ONLINE"`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This skill has the agent actually run code examples with Deephaven MCP to make sure they work before handing them off to a human.