Skip to content

Add consequentialHint to ToolAnnotations #217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
beaufortfrancois wants to merge 1 commit into
base: main
Choose a base branch
from
+21 −0
Open

Add consequentialHint to ToolAnnotations #217

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions index.bs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,9 @@ A <dfn>tool definition</dfn> is a [=struct=] with the following [=struct/items=]
: <dfn>untrusted content hint</dfn>
:: a [=boolean=], initially false.

: <dfn>consequential hint</dfn>
:: a [=boolean=], initially false.

: <dfn>exposed origins</dfn>
:: a [=list=] or [=origins=], initially [=list/empty=].
</dl>
Expand Down Expand Up @@ -395,6 +398,9 @@ The <dfn method for=ModelContext>registerTool(<var>tool</var>, <var>options</var
1. Let |untrusted content hint| be true if |tool|'s {{ModelContextTool/annotations}} [=map/exists=] and
its {{ToolAnnotations/untrustedContentHint}} is true. Otherwise, let it be false.

1. Let |consequential hint| be true if |tool|'s {{ModelContextTool/annotations}} [=map/exists=] and
its {{ToolAnnotations/consequentialHint}} is true. Otherwise, let it be false.

1. Let |promise| be [=a new promise=] created in [=this=]'s [=relevant realm=].

1. Let |signal| be |options|'s {{ModelContextRegisterToolOptions/signal}}.
Expand Down Expand Up @@ -447,6 +453,9 @@ The <dfn method for=ModelContext>registerTool(<var>tool</var>, <var>options</var
: [=tool definition/untrusted content hint=]
:: |untrusted content hint|

: [=tool definition/consequential hint=]
:: |consequential hint|

: [=tool definition/exposed origins=]
:: |exposed origins|

Expand Down Expand Up @@ -482,6 +491,7 @@ dictionary ModelContextTool {
dictionary ToolAnnotations {
boolean readOnlyHint = false;
boolean untrustedContentHint = false;
boolean consequentialHint = false;
};

callback ToolExecuteCallback = Promise<any> (object input);
Expand Down Expand Up @@ -531,6 +541,9 @@ The {{ToolAnnotations}} dictionary provides optional metadata about a tool:

: <code><var ignore>annotations</var>["{{ToolAnnotations/untrustedContentHint}}"]</code>
:: If true, indicates that the tool's output contains data that is untrusted, from the perspective of the author registering the tool.

: <code><var ignore>annotations</var>["{{ToolAnnotations/consequentialHint}}"]</code>
:: If true, indicates that executing the tool will result in consequential actions, ex: booking a flight, transferring money.
</dl>


Expand Down Expand Up @@ -1101,6 +1114,14 @@ respective private browsing modes are safely exposed to [=agents=] and that thes

**How:** A boolean {{ToolAnnotations/untrustedContentHint}} annotation that acts as a signal to the client that the payload requires heightened security handling, allowing the client to sanitize the payload, use indicators such as spotlighting [[SPOTLIGHTING]] to highlight untrustworthy content to the model, or hide that part of the response entirely.

<h4 id="mitigation-consequential-annotation">Consequential Annotation for Tool Executions</h4>

**What:** Providing agents with a signal that a tool's execution results in significant, real-world, or non-reversible consequences.

**Threats addressed:** [[#misrepresentation-of-intent]]

**How:** A boolean {{ToolAnnotations/consequentialHint}} annotation acts as a signal to the client or agent that the tool performs a consequential action, such as booking a flight or transferring money. This way they can selectively enforce mandatory user confirmation prompts before executing high-stakes tools, directly mitigating the risk of accidental or malicious misrepresentation of intent.

@victorhuangwq victorhuangwq Jul 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that consequentialHint helps with the accidental case. Using the finalizeCart example, the name and description may be ambiguous about whether the tool merely prepares the cart or completes the purchase. A non-malicious developer still knows that execute triggers a purchase, so they will likely be marking the tool as consequential. This would give the client or agent an explicit signal that it should request confirmation rather than relying only on its interpretation of “finalize.”

I’m less convinced it mitigates the malicious case described here. A malicious developer controls the name, description, implementation, and annotations. If they deliberately describe finalizeCart ambiguously to conceal a purchase, they will omit consequentialHint or set it to false.


<h2 id="accessibility">Accessibility considerations</h2>


Expand Down
Loading