Skip to content

Skip redundant commit pack download during clone when prefetch packs exist#1995

Draft
tyrielv wants to merge 2 commits into
microsoft:masterfrom
tyrielv:tyrielv/skip-clone-commit-download
Draft

Skip redundant commit pack download during clone when prefetch packs exist#1995
tyrielv wants to merge 2 commits into
microsoft:masterfrom
tyrielv:tyrielv/skip-clone-commit-download

Conversation

@tyrielv
Copy link
Copy Markdown
Contributor

@tyrielv tyrielv commented Jun 1, 2026

Summary

When gvfs clone runs with a shared cache that already has prefetch packs (e.g., from a previous clone), defer the commit+trees pack download. This avoids redundantly re-downloading trees that are already available in the cache.

Changes

CloneVerb.cs — Clone optimization

  • Before downloading the commit pack, check if the commit+root tree already exist locally (CommitAndRootTreeExists), or if usable prefetch packs are present in the shared cache (HasUsablePrefetchPacks).
  • If prefetch packs exist but the commit is missing, defer the download. If CreateBranchWithUpstream or TryDownloadRootGitAttributes fails because the commit is needed, download on demand and retry.
  • The existing checkout fallback ("unable to read tree" → force re-download) handles missing subtrees.

GitObjects.cs — Shared helpers

  • IsUsablePrefetchPack(packPath) — checks a pack has a matching .idx and no .incomplete marker.
  • HasUsablePrefetchPacks() — returns true if at least one usable prefetch pack exists.
  • PrefetchStep.TryGetMaxGoodPrefetchTimestamp refactored to use IsUsablePrefetchPack instead of inline checks.

SharedCacheTests.cs — Functional tests

Two tests that manufacture specific shared-cache states via "cache surgery" (creating fake prefetch packs with git pack-objects):

  1. SecondCloneSucceedsWithMissingTreesAndPrefetchPacks — commit+root tree present as loose objects, fake prefetch pack exists, but subtrees are missing. Verifies the checkout fallback re-downloads the commit pack.

  2. SecondCloneWithPrefetchPacksButMissingCommit — target commit is completely absent but a fake prefetch pack exists. Exercises the deferred-download path (skippedCommitDownload) that falls back through CreateBranchWithUpstream/TryDownloadRootGitAttributes.

Scenarios

Cache state Behavior
Commit+root tree in prefetch packs CommitAndRootTreeExists → true → skip download
Prefetch packs exist, commit absent Defer download → fallback on branch creation or gitattributes
Prefetch packs exist, commit+root tree present, subtrees missing Skip download → checkout fails → fallback re-download
No prefetch packs Download as before (no behavior change)

tyrielv added 2 commits June 1, 2026 14:15
When gvfs clone runs with a shared cache that already has prefetch
packs (e.g., from a previous clone), defer the commit+trees pack
download. If the commit is already present in the prefetch packs,
the download is skipped entirely. If it is not, the fallback path
downloads on demand when branch creation, gitattributes lookup, or
checkout requires it.

This avoids redundantly re-downloading trees that are already in
prefetch packs when the tip commit happens to be covered by the
existing cache.

Assisted-by: Claude Opus 4.6
Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
Two tests that manufacture specific shared-cache states to verify
clone behavior when prefetch packs exist:

1. SecondCloneSucceedsWithMissingTreesAndPrefetchPacks: commit and
   root tree are present as loose objects, a fake prefetch pack
   exists, but subtrees are missing. Verifies the checkout fallback
   re-downloads the commit pack.

2. SecondCloneWithPrefetchPacksButMissingCommit: the target commit
   is completely absent but a fake prefetch pack exists. Exercises
   the deferred-download path (skippedCommitDownload) that falls
   back through CreateBranchWithUpstream/TryDownloadRootGitAttributes.

Both tests create a temp bare repo to run git pack-objects without
VFS hooks, producing a minimal prefetch-named pack in the shared
cache's pack directory.

Assisted-by: Claude Opus 4.6
Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant