Skip to content

fix: scope facility doctor agent checks to YAML frontmatter - #342

Open
o-mid wants to merge 3 commits into
theam:mainfrom
o-mid:fix/doctor-frontmatter-scope
Open

o-mid wants to merge 3 commits into
theam:mainfrom
o-mid:fix/doctor-frontmatter-scope

Conversation

@o-mid

@o-mid o-mid commented Sep 8, 2026

Copy link
Copy Markdown

Addresses doctor false pass and false fail against the 0.12 agent contract.

facility doctor applied name, model, trigger, and forbidden-key regexes to the whole agent file. Prompt text such as a GitHub permissions: example failed the check, and a wrong frontmatter name: could be hidden by repeating the expected name in the prompt. Quoted model ids with spaces also failed.

Doctor now splits YAML frontmatter from the prompt, accepts mcp and ui triggers (already valid at runtime), and reads the model as a YAML string scalar. Comment-only values (model: # choose a model) and unclosed quotes are rejected, matching the server.

Test plan

  • pnpm --filter @theagilemonkeys/facility test
  • CLI suite 10 consecutive passes
  • Confirm a prompt that contains permissions: still passes
  • Confirm frontmatter permissions: still fails
  • Confirm name: builder only in the prompt does not satisfy builder.md
  • Confirm model: "gpt-5.6-sol with spaces" passes
  • Confirm model: # choose a model fails
  • Confirm model: "unclosed fails
  • Confirm an agent with only mcp and ui triggers passes

Doctor was matching name, model, triggers, and forbidden keys across the
prompt body, so a valid prompt could fail and a wrong frontmatter name
could hide behind a line in the prompt.

@adrian-lorenzo adrian-lorenzo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing the frontmatter scoping.

The relaxed model check now accepts model: # choose a model and unclosed quoted values, although the server rejects both. Please validate the model value while preserving quoted names with spaces, and add regression tests for these cases.

o-mid and others added 2 commits September 15, 2026 11:00
The relaxed model check treated any non-whitespace after model: as
valid, so a YAML comment or an unclosed quoted scalar passed locally
while the server rejected both.
@o-mid

o-mid commented Sep 15, 2026

Copy link
Copy Markdown
Author

@adrian-lorenzo Thanks, that was a real hole.
\S treated # and an opening quote as a model ID.
Follow-up parses the model as a YAML string scalar, so quoted names with spaces still pass. model: # choose a model and unclosed quotes fail, same as the server. Tests cover both.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants