Skip to content

Master test matrix — 148 tests needed across all components and flows #68

Description

@KlementMultiverse

Current State: 2% Coverage (4/143 components, 0/9 flows)

Test Matrix

Category Total Tested Coverage Test Type
Scripts 25 3 12% BATS unit + integration
Agents 54 0 0% Schema validation (frontmatter, structure)
Commands 35 0 0% Schema validation (header, system-reminder)
Phase files 6 0 0% Reference validation (agents/skills exist)
Templates 15 0 0% Pipe 1/2 behavior, placeholder safety
Rules 7 0 0% Auto-load behavior, no context poisoning
Hooks 1 0 0% JSON validity, all events covered
Flows 9 0 0% Integration tests (end-to-end)
Total 152 3 2%

What "tested" means per type

Scripts: Each function/subcommand has a BATS or pytest test verifying inputs, outputs, edge cases, error handling.

Agents: Validate YAML frontmatter (name, description, tools present), heading format, referenced commands/scripts exist.

Commands: Validate header format (# /name — desc), system-reminder presence for critical commands, context:fork frontmatter where needed.

Phase files: All subagent_type="X" resolve to existing agents, all skill: "X" resolve to existing commands, all script references exist.

Templates: Placeholders inside HTML comments only (Pipe 1 safety), no raw {{}} in auto-load position.

Rules: No context poisoning, proper structure, auto-load strips correctly.

Hooks: Valid JSON, all 4 event types covered, script references resolve.

Flows: End-to-end integration tests with real git repos.

Priority Order

  1. Create /autoresearch command — blocks Phase 5 step 54 #60 — /autoresearch command (critical blocker)
  2. Create 10 sc:* namespace commands — blocks Phase 2, 4, 5 #61 — 10 sc:* commands (critical blocker)
  3. Write BATS tests for core enforcement scripts #62 — Core enforcement script tests
  4. Write BATS tests for traceability + ownership scripts #63 — Traceability + ownership tests
  5. Write BATS tests for infrastructure scripts #64 — Infrastructure script tests
  6. Write pytest tests for Python scripts #65 — Python script tests
  7. Write BATS tests for git hooks (commit-msg, pre-commit) #66 — Git hook tests
  8. Agent schema validation tests
  9. Command schema validation tests
  10. Phase file reference tests
  11. Template Pipe 1/2 tests
  12. Rule auto-load tests
  13. Hook JSON tests
  14. Flow integration tests
  15. Create forge-test-guard.sh — detect untested script changes #67 — forge-test-guard.sh

Protocol (for each new component going forward)

  1. Create test FIRST (RED)
  2. Create component (GREEN)
  3. Run full suite (no regressions)
  4. Update forge-registry.json (test field)
  5. PR → CR → loop → [approve] → merge

@coderabbitai Please review this test plan.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions