Skip to content

Commit f9f6261

Browse files
committed
Make _http_exists send the ranged GET its docstring describes
The docstring states the requirement exactly: a ranged GET, so that servers which ignore HEAD still work. The body calls make_head_request, which is requests.head. Two consequences. The Range header is meaningless on a HEAD, so the 206 the function tests for is never observed on that path. And requests.head defaults to allow_redirects=False, so a 302 also reads as missing. That combination takes out the HuggingFace lane entirely: huggingface.py accepts only .bin, every real .bin weight is Git-LFS backed, and HF serves those from a CDN via a 302. Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
1 parent 27c4fc9 commit f9f6261

1 file changed

Lines changed: 9 additions & 2 deletions

File tree

‎src/fetchcode/utils.py‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -250,7 +250,14 @@ def _http_exists(url: str) -> bool:
250250
Lightweight existence check using a ranged GET so CDNs/servers that ignore HEAD still work.
251251
"""
252252
try:
253-
resp = make_head_request(url, headers={"Range": "bytes=0-0"})
254-
return resp is not None and resp.status_code in (200, 206)
253+
# A ranged GET rather than a HEAD, as the docstring says: HEAD ignores the
254+
# Range header so 206 is never observed on that path, and requests.head
255+
# does not follow redirects, so a CDN 302 reads as "missing". Git-LFS
256+
# backed files are served exactly that way.
257+
resp = requests.get(url, headers={"Range": "bytes=0-0"}, allow_redirects=True, stream=True)
258+
try:
259+
return resp.status_code in (200, 206)
260+
finally:
261+
resp.close()
255262
except Exception:
256263
return False

0 commit comments

Comments
 (0)