The IMDb tables and a torrent's title can come from nichedb - #216
Merged
Conversation
…f the dumps and TMDB bittorrented imported the IMDb dumps nightly and asked TMDB for every torrent page's poster. nichedb.dev keeps both now, so this adds the other path and leaves the old one in place. With NICHEDB_TITLES=1 a torrent page asks nichedb what its name is before TMDB, and takes the answer only when it comes with a poster or a synopsis; a bare IMDb row falls through to TMDB so no page loses its picture. The tmdb_data row written is the same shape either way. With NICHEDB_MIRROR=1 the nightly crontab line runs a walk of the screen collection instead of the dump import, filling imdb_title_basics and imdb_title_ratings from IMDb-sourced rows (TMDB and TVmaze rows only add a title they know the tconst of, and never write a rating, since theirs is a different scale). A cursor in nichedb_mirror_cursor resumes a walk cut short by the request budget; the first run is the backfill, about 1,700 pages seven seconds apart. Crew, principals, episodes, akas and names are not in nichedb, so those tables stop refreshing under the mirror; the director line on a DHT page freezes at its last dump value until nichedb carries crew. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014X1XCwt5iD3nqZfbddZqEP
ThreatCrush Security Scan99 finding(s) HIGH/CRITICAL: 11 | MEDIUM: 29 | LOW: 59
…and 49 more. Full results in the Security tab. Snippets are redacted; ThreatCrush never prints matched credential material. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
bittorrented imported the IMDb dumps nightly and asked TMDB for every torrent page's poster. nichedb.dev keeps both now, so this adds the other path and leaves the old one in place. Nothing changes until a switch is set.
NICHEDB_TITLES=1(app env): a torrent page asksnichedb.dev/api/v1/matchwhat its name is before TMDB, and takes the answer only when it comes with a poster or a synopsis; a bare IMDb row falls through to TMDB so no page loses its picture. Thetmdb_datarow written is the same shape either way.NICHEDB_MIRROR=1(when runningsetup-server.sh): the nightly crontab line runspnpm mirror:imdbinstead of the dump import, fillingimdb_title_basicsandimdb_title_ratingsfrom IMDb-sourced rows. TMDB and TVmaze rows only add a title they know the tconst of and never write a rating (different scale). A cursor innichedb_mirror_cursorresumes a walk cut short by the request budget; the first run is the backfill (pnpm mirror:imdb -- --all, about 1,700 pages seven seconds apart, ~3.3 h).Not carried by nichedb, so frozen at their last dump under the mirror: crew, principals, episodes, akas, names. The director line on a DHT page is the visible one.
Tests: 51 new (mapping, match pick, cursor paging, fetch order), suite 2,839 pass, tsc clean, lint 0 errors.
🤖 Generated with Claude Code
https://claude.ai/code/session_014X1XCwt5iD3nqZfbddZqEP