docs: restructure playbook & add AI-assisted engineering guidance

[vdstrizhkova]

For more information about how to contribute to this repo, visit this page

Description

This PR reorganizes the playbook around role-based entry points, introduces a dedicated AI-assisted engineering body of guidance, consolidates non-functional requirements into their own section, and applies a consistent tone of voice across existing docs. The goal is to make the playbook easier to navigate for different personas and to bring its AI/ML guidance up to date, without losing any existing content or cross-references.

Key changes:

• New "Start Here" section (docs/start-here/) with role-based on-ramps for engineers, data scientists, and leads, plus a project kickoff checklist. Replaces and supersedes the removed docs/the-first-week-of-an-ise-project.md.
• AI-assisted engineering hub (docs/ai-assisted-engineering/README.md) plus deeper domain content: generative AI & agentic systems, expanded responsible AI, model experimentation, ML observability, and threat-modelling additions. Synthesized and de-duplicated from two prior branches with all cross-references reconciled.
• New Non-Functional Requirements section (docs/non-functional-requirements/README.md) including accessibility, with relevant existing content relocated here.
• CI/CD & source-control guidance expanded around branching strategy and pipeline setup.
• Tone-of-voice alignment and cross-reference callouts applied across observability, security, code-reviews, documentation, developer-experience, and other sections.
• Tooling: internal-link checker script with a supporting entry in requirements-docs.txt; agentic-workflow skills/agent and copilot-setup-steps.yml under .github/.
• Navigation updates across .pages files and READMEs to reflect the new structure; docs/design/readme.md renamed to README.md for consistency.

Fix MegaLinter findings in docs and config

Resolves linter errors flagged by MegaLinter:

• Dead link: Replaced the broken LeanKit lean-development URL in docs/CI-CD/continuous-integration.md with its live Planview equivalent (LeanKit was acquired by Planview).
• Link checker config: Fixed an invalid header entry in lychee.toml that was preventing the lychee link checker from loading its config. Left it disabled not to increase the scope of the PR
• Spelling: Added legitimate technical terms (e.g., MSAL/msal, pygments, microsoftonline) to the cspell dictionary.
• GitHub Actions: Bumped outdated action versions in .github/workflows/mkdocs.yml (actions/checkout, actions/setup-python, peaceiris/actions-gh-pages).
• Markdown formatting: Corrected table-style and link-text lint violations across affected docs.

PR Checklist

• I have updated the documentation accordingly.
• I have added tests to cover my changes.
• All new and existing tests passed.
• My code follows the code style of this project.
• I ran the lint checks which produced no new errors nor warnings for my changes.
• I have checked to ensure there aren't other open Pull Requests for the same update/change.
• If AI assistance materially shaped this change, I disclosed it and included validation, dependency/license/security, and eval/test evidence where applicable.

Does This Introduce a Breaking Change?

• Yes
• No

Content reorganization only. docs/the-first-week-of-an-ise-project.md was removed; its content is superseded by the new Start Here section. Any external links to the old path should be redirected there.

Testing

• Ran the internal-link checker to confirm no broken cross-references after the restructure.
• Built/served the MkDocs site locally to verify navigation (.pages) renders correctly and all moved/renamed pages resolve.
• Verified the removed page's content is fully covered by the new Start Here section.

[shiranr]

I think this is still relevant :)
Do we have a replacement for this with the HVE style?

[vdstrizhkova]

just a rename. it is a project kick off now

[Copilot]

Pull request overview

This PR restructures the playbook’s documentation IA around role-based entry points, adds a dedicated AI-assisted engineering guidance hub, and consolidates non-functional requirements content—while also addressing several docs/config lint issues and introducing Copilot/agentic-workflow supporting configuration.

Changes:

• Introduces a new Start Here section (role-based reading paths + kickoff checklist) and removes the old “first week” page.
• Adds/expands AI-assisted / gen-AI / agentic guidance across security, observability, testing, design, and documentation, plus a dedicated hub page.
• Updates supporting tooling/config (MkDocs workflow action bumps, link-checker config cleanup, cspell additions, gitignore/gitattributes, and Copilot/gh-aw agent assets).

Reviewed changes

Copilot reviewed 66 out of 69 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
requirements-docs.txt	Adds docs build dependency constraint.
README.md	Updates top-level navigation links.
lychee.toml	Fixes/clarifies link checker settings.
docs/UI-UX/recommended-technologies.md	Markdown cleanup + updated resources.
docs/UI-UX/README.md	Adds AI-assisted UI/UX guidance + formatting.
docs/the-first-week-of-an-ise-project.md	Removes superseded kickoff document.
docs/start-here/README.md	Adds Start Here landing page.
docs/start-here/project-kickoff-checklist.md	Adds sprint-sequenced kickoff checklist.
docs/start-here/for-leads.md	Adds curated reading path for leads.
docs/start-here/for-engineers.md	Adds curated reading path for engineers.
docs/start-here/for-data-scientists.md	Adds curated reading path for DS/ML.
docs/start-here/.pages	Adds MkDocs nav for Start Here.
docs/source-control/secrets-management.md	Extends secrets guidance to AI artifacts.
docs/source-control/README.md	Adds AI-assisted source-control expectations.
docs/source-control/naming-branches.md	Adds agent-branch naming guidance.
docs/source-control/git-guidance/README.md	Adds AI-assisted traceability section.
docs/security/threat-modelling.md	Adds AI threat-modeling considerations.
docs/security/README.md	Adds AI/agent security overview section.
docs/README.md	Updates docs landing page entry points.
docs/observability/recipes-observability.md	Fixes formatting in observability recipe.
docs/observability/README.md	Adds AI observability guidance section.
docs/observability/pillars/tracing.md	Refines OpenTelemetry wording.
docs/observability/ml-observability.md	Adds LLM/RAG/agent telemetry guidance.
docs/observability/microservices.md	Refines OpenTelemetry wording.
docs/observability/logs-privacy.md	Adds gen-AI logging/privacy guidance.
docs/observability/correlation-id.md	Clarifies OpenTelemetry correlation guidance.
docs/observability/best-practices.md	Adds AI-specific observability practices.
docs/non-functional-requirements/README.md	Introduces consolidated NFR section index.
docs/non-functional-requirements/privacy/README.md	Adds AI privacy review prompts.
docs/non-functional-requirements/privacy/data-handling.md	Adds AI tool data-handling guidance.
docs/non-functional-requirements/maintainability.md	Links to testing taxonomy + clarifies.
docs/non-functional-requirements/accessibility.md	Adds AI experience + AI-assisted a11y guidance.
docs/ml-and-ai-projects/responsible-ai.md	Adds gen-AI/agent Responsible AI prompts.
docs/ml-and-ai-projects/README.md	Adds gen-AI/agent entry point framing.
docs/ml-and-ai-projects/model-experimentation.md	Adds gen-AI/agent evaluation guidance.
docs/ml-and-ai-projects/generative-ai-and-agentic-systems.md	New guide for gen-AI/agent systems.
docs/engineering-fundamentals-checklist.md	Adds AI-assisted checklist section.
docs/engineering-feedback/README.md	Adds AI tooling feedback guidance.
docs/documentation/README.md	Adds AI-assisted documentation guidance.
docs/documentation/guidance/code.md	Adds guidance for generated comments/explanations.
docs/documentation/best-practices/automation.md	Adds automation-vs-prose guidance for AI drafts.
docs/developer-experience/README.md	Adds AI-assisted DevEx section.
docs/developer-experience/copilots.md	Adds operating model + validation guidance.
docs/design/README.md	Adds AI-assisted/AI-enabled design guidance.
docs/design/design-reviews/recipes/high-level-design-recipe.md	Adds gen-AI design review prompts.
docs/design/design-patterns/non-functional-requirements-capture-guide.md	Adds gen-AI/agent NFR metrics table.
docs/code-reviews/README.md	Adds AI-assisted review guidance.
docs/code-reviews/pull-request-template.md	Adds AI assistance disclosure checkbox.
docs/code-reviews/process-guidance/reviewer-guidance.md	Adds AI/agent-authored change review checklist.
docs/CI-CD/README.md	Adds AI release-artifact expectations in CI/CD.
docs/CI-CD/continuous-integration.md	Adds AI artifact guidance + fixes external link.
docs/automated-testing/unit-testing/README.md	Formatting cleanup in TDD section.
docs/automated-testing/test-planning.md	Adds AI evaluation planning guidance.
docs/automated-testing/README.md	Adds AI-assisted/AI-enabled testing section.
docs/ai-assisted-engineering/README.md	New AI-assisted engineering hub document.
docs/agile-development/team-agreements/definition-of-done.md	Links DoD to testing taxonomy.
docs/agile-development/README.md	Adds AI-assisted baseline cross-link.
docs/agile-development/branching-and-cicd.md	Consolidates policy references to canonical docs.
docs/.pages	Updates global docs navigation order/entries.
.mega-linter.yml	Documents why lychee is disabled for now.
.gitignore	Ignores Copilot agent tracking artifacts.
.github/workflows/mkdocs.yml	Bumps GitHub Actions versions for site build.
.github/workflows/copilot-setup-steps.yml	Adds Copilot setup steps workflow.
.github/skills/agentic-workflows/SKILL.md	Adds gh-aw workflow router skill.
.github/skills/agentic-workflow-designer/SKILL.md	Adds workflow designer interview skill.
.github/mcp.json	Adds MCP server configuration for gh-aw.
.github/agents/agentic-workflows.md	Adds dispatcher agent definition for gh-aw.
.gitattributes	Marks lock files as generated + merge strategy.
.cspell.json	Adds additional technical terms to dictionary.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[shiranr]

I am not sure I would add this section.
It might create more confusion and diversion.
I would consider changing the main menu:

Into:
nav:

• Who are we?
• How to Use This Playbook
• For Engineers
• For Project Managers
• For Data Scientists
• Project Kickoff Checklist
• Engineering Fundamentals Checklist
• ....

[vdstrizhkova]

how about

Start Here

• How to Use This Playbook
• Who We Are (move or link ISE.md)
• For Engineers
• For Project Managers & Leads
• For Data Scientists
• For UX/UI Designers
• Project Kickoff Checklist
• Engineering Fundamentals Checklist
• How to Contribute