Skip to content

fix(cli): keep spec header values that contain '#' or ': ' - #3564

Merged
migmartri merged 3 commits into
mainfrom
miguel/pfm-7658-spec-source-metadata-is-cut-or-lost-when-a-header-value
Oct 8, 2026
Merged

migmartri merged 3 commits into
mainfrom
miguel/pfm-7658-spec-source-metadata-is-cut-or-lost-when-a-header-value

Conversation

@migmartri

@migmartri migmartri commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

Spec source headers lost data when a value was not quoted. Agents write these headers in markdown frontmatter and in .meta.yaml companion files. The CLI read them with a YAML decoder:

  • description: The merged design spec (PR #3544) ... was cut at #, because YAML reads it as a comment.
  • title: PFM-1: fix: the thing, title: [WIP] something and a value with `foo: bar` are not valid YAML. The CLI then dropped all of the header: kind, uri, role, title and description. The kind became text.

This change reads the header line by line, and only the keys kind, uri, role, title and description:

  • The value of a key is the rest of its line after the first colon. The CLI keeps it as written, with #, more colons, a leading [ and backticks.
  • A value in quotes is unquoted as YAML does, so quoted headers give the same result as before. A | or > block scalar is also decoded as YAML.
  • A blank line, a comment line or an unknown key is skipped. A bad line does not clear the other keys.
  • The kind and role vocabularies and the title and description limits do not change.

The values in the spec material annotations and in the session evidence now hold the full text.

AI disclosure: Claude Code assisted with this change.

closes https://linear.app/chainloop/issue/pfm-7658

🤖 Posted by Maximus bot (Claude Code) on behalf of @migmartri

View guided diff

Read spec source headers and .meta.yaml companion files line by line
instead of with a YAML decoder. The value of a key is the rest of its
line, so an unquoted value that holds ' #', ': ', a leading '[' or
backticks is kept whole, and one bad line no longer clears the other
keys. Quoted values and block scalars are still decoded as YAML.

Assisted-by: Claude Code
Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>

Chainloop-Trace-Sessions: 36229b0e-92ef-46a9-9ee6-323b0dedac7f
@migmartri
migmartri requested a review from a team October 8, 2026 11:16
@chainloop-platform

chainloop-platform Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

PR validation — ✅ 3 passing

Status Policy Material Messages
✅ Passed pr-min-approvals pr-info -
✅ Passed pr-description-required pr-info -
✅ Passed pr-user-story-linked pr-info -

View attestation ↗

AI Session Checks — 🟡 60% · ⚠️ 1 failing

Avg score Sessions Failing policies Attribution Files Lines Total Duration
🟡 60% 1 ⚠️ 1 100% AI / 0% Human 3 +289 / -36 16m5s

🟡 60% — 100% AI — ⚠️ 1 policies failing

Oct 8, 2026 11:06 UTC · 16m5s · $7.39 · 368 in / 36.9k out · claude-code 2.1.294 (claude-opus-5-5)

View session details ↗

Change Summary

  • Replaces YAML value parsing for CLI spec headers with a custom line parser.
  • Updates parser and material-annotation tests for hashes, colons, brackets, and quoted values.
  • Leaves platform code unchanged; the session only traced how the metadata reaches the Context tab.

AI Session Overall Score

🟡 60% — Focused fix, but verification relied on unproven subagent claims and later broke in CI.

AI Session Analysis Breakdown

🟢 88% · user-trust-signal

🟢 User gave focused follow-ups and an explicit testing ask without derailing the session. · Medium Impact

🟢 86% · scope-discipline

🟢 AI checked the platform display path and kept the code change in the CLI. · High Impact

🟢 82% · solution-quality

🟢 AI traced the truncation to YAML semantics and aimed the fix at the parser layer. · High Impact

🟡 68% · alignment

🟠 AI initially said annotations were irrelevant, then corrected that after the user pasted the attestation entry. · Medium Severity

💡 Before explaining data flow, verify each user-facing field against the concrete object the user pasted.

🟡 65% · context-and-planning

🟠 The multi-repo fix started from a one-line ask with no shared plan artifact. · Medium Severity

💡 For cross-repo delegations, ask for a one-paragraph scope, test, and ship plan before the subagent starts.

🔴 30% · verification

🔴 Parent session never showed test commands, and later PR checks failed the cli jobs after the AI declared success. · High Severity

💡 Require the command and pass-fail tail for delegated test runs before accepting a ready-for-review claim.


File Attribution

████████████████████ 100% AI / 0% Human

Status Attribution File Lines
modified ai app/cli/internal/trace/spec/parse.go +126 / -26
modified ai app/cli/internal/trace/spec/parse_test.go +118 / -9
modified ai app/cli/pkg/action/trace_spec_materials_test.go +45 / -1

Policies (4, 1 failing)

Status Policy Material Messages
✅ Passed ai-config-ai-agents-allowed ai-coding-session-36229b -
✅ Passed ai-config-no-dangerous-commands ai-coding-session-36229b -
⚠️ Failed ai-config-no-secrets ai-coding-session-36229b
  • Secret (generic-[REDACTED:generic-password) detected in session content [turn=316, source=tool_result, line=129]: | ⚠️ Failed | ai-config-no-secrets | [ai-coding-session-36229b](https://app.chainloop.dev/u/chainloop/projects/chainloop/versi...
  • Secret (generic-password) detected in session content [turn=207, source=tool_result, line=3]: INF redacted secrets from the AI coding session before upload count=10 rules=["generic-[REDACTED:generic-password]"]
  • Secret (generic-password) detected in session content [turn=29, source=tool_result, line=195]: - PostgreSQL: localhost:5432, database: controlplane, user: postgres, no [REDACTED:generic-password]
  • Secret (generic-password) detected in session content [turn=29, source=tool_result, line=197]: - Dex OIDC: localhost:5556, static users with [REDACTED:generic-password] "[REDACTED:generic-password]"
  • Secret (generic-password) detected in session content [turn=29, source=tool_result, line=203]: - OIDC Users: sarah@chainloop.local / john@chainloop.local ([REDACTED:generic-password]: [REDACTED:generic-password])
  • Secret (generic-password) detected in session content [turn=316, source=tool_result, line=129]: | ⚠️ Failed | ai-config-no-secrets | [ai-coding-session-36229b](https://app.chainloop.dev/u/chainloop/projects/chainloop/versi...
  • Secret (generic-password) detected in session content [turn=361, source=assistant-text, line=9]: - The lab's default local [REDACTED:generic-password]s (Postgres and Dex), from a CLAUDE.md that was read early in the session.
  • Secret (github-pat) detected in session content [turn=151, source=tool_result, line=57]: assert.Equal(t, "token [REDACTED:github-pat] fails", entries[0].Title)
  • Secret (github-pat) detected in session content [turn=151, source=tool_result, line=58]: assert.Equal(t, "uses [REDACTED:github-pat]", entries[0].Description)
  • Secret (github-pat) detected in session content [turn=151, source=tool_result, line=79]: assert.Equal(t, "mockup for [REDACTED:github-pat]", entries[0].Title)
  • Secret (github-pat) detected in session content [turn=316, source=tool_result, line=129]: | ⚠️ Failed | ai-config-no-secrets | [ai-coding-session-36229b](https://app.chainloop.dev/u/chainloop/projects/chainloop/versi...
  • Secret (github-pat) detected in session content [turn=330, source=tool_result, line=1]: 246: assert.Equal(t, "token [REDACTED:github-pat] fails", entries[0].Title)
  • Secret (github-pat) detected in session content [turn=330, source=tool_result, line=3]: 268: assert.Equal(t, "mockup for [REDACTED:github-pat]", entries[0].Title)
✅ Passed ai-config-mcp-servers-allowed ai-coding-session-36229b -

Security Checks — ✅ 5 passing

✅ secret-scan

Status Policy Messages
✅ Passed secrets-detection -

✅ sast-scan

Status Policy Messages
✅ Passed owasp-top10-2025 -
✅ Passed sast -
✅ Passed cwe-top25 -
✅ Passed cwe-top26-40-cusp -
Scans not applied (3)
Scan Reason
vulnerability-scan no manifest/lockfile changed
github-actions-scan no workflow files changed
iac-scan no IaC files changed

View attestation ↗

Security context

✅ Nothing this change touches has a recorded security-fix history.

View in Chainloop ↗ · How this works ↗


Powered by Chainloop and Chainloop Trace

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread app/cli/internal/trace/spec/parse.go Outdated
An indented line that starts with '#' is a YAML comment, so it is no
longer folded into a plain header value. Also update the new
attachSpecs test to the four return values that main now has.

Assisted-by: Claude Code
Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>

Chainloop-Trace-Sessions: 36229b0e-92ef-46a9-9ee6-323b0dedac7f
@migmartri
migmartri merged commit 40618da into main Oct 8, 2026
16 of 17 checks passed
@migmartri
migmartri deleted the miguel/pfm-7658-spec-source-metadata-is-cut-or-lost-when-a-header-value branch October 8, 2026 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants