fix: post-merge follow-ups — demux hardening, edit lists, paced timing, load identity#5
Merged
Merged
Conversation
Remote MP4 reaches this parser (HLS/CMAF segments, VOD URLs), so treat every stream-supplied field as hostile: - parse_traf: validate tfhd/tfdt/trun field reads against the box body, and reject a trun whose declared per-sample table does not fit in the box (or exceeds a sane per-fragment sample cap). Unchecked counts could previously drive reads past the buffered moof. - Box walkers: rewrite child-size checks in subtraction form (sz > len - off) so a crafted size near INT_MAX cannot overflow the signed addition and bypass the bound. - consume_mdat / consume_progressive: widen run positions to int64 and make the sample fit checks overflow-safe; cap progressive sample sizes at the box cap so a crafted stsz entry cannot drive a multi-gigabyte allocation. - Timestamps: convert ticks to microseconds via quotient/remainder (ticks_to_us) instead of value * 1000000 / timescale, which overflows at stream-controlled timescales. - ctts: honour the FullBox version - version 0 offsets are unsigned, version 1 signed. Previously all offsets were cast signed. - stz2: detect the compact sample-size table and surface a clear unsupported-format error instead of starting playback with the affected track silently missing. - stsd: bound the mp4a fixed-header reads to the sample entry size. - Annex B sizing: move the worst-case AVCC-to-Annex-B output size into a shared helper (basis_avcc_annexb_cap) and use it from both MP4 paths and RTMP. RTMP previously allocated dlen + 64, which undersizes streams using 1-3 byte NAL length prefixes and silently dropped the access unit.
Remove the never-read next_dts track field, update the fMP4-only header comment (the demuxer has handled progressive MP4 since the classic-table work), refresh the top-of-file assumptions to match, and break the nested '/*' in basis_bitstream.h that warned on every translation unit.
Progressive playback previously ignored edts/elst, so every track started at presentation time zero regardless of its edit list. A track delayed by an initial empty edit (common muxer output for A/V alignment) played that much early, permanently desynced from the other track. Parse mvhd for the movie timescale and each track's elst, then map media time onto the movie timeline at emission: initial empty edits become a presentation-time delay, and the first normal edit's media_time becomes the media-time origin (encoder priming / initial trim). Samples ahead of the origin keep a negative presentation time rather than being dropped, since video there can still carry reference frames; multi-segment edits beyond the first normal entry are out of scope for a linear walk. Fragmented MP4 is unaffected - its timeline comes from tfdt.
Delivery pacing gated on the presentation timestamp of whatever the demux thread submitted last. A video AU whose composition offset puts its pts past the pacing lead therefore slept the demux thread out beyond its own decode turn - and any earlier samples of the other track queued behind it on the same thread arrived late enough for the audio queue to trim them as stale. Streams reordering less than the ~400 ms lead masked this. Carry the decode timestamp through the video sink alongside pts and gate on it, while the decoder and caption store keep receiving pts. Demuxers without decode timestamps (TS, RTSP) pass pts for both, which is the old behaviour; RTMP forwards the FLV tag timestamp it already had. The progressive MP4 walk had the file-order variant of the same stall: a muxer that interleaves chunks coarsely (a chunk of video ahead of the matching audio) blocked the forward reader on paced video before it could reach the audio bytes at all. Samples whose file order runs ahead of their delivery turn now park in bounded per-track queues and deliver in decode-time order across tracks; past the 32 MB bound (or once the mdat has nothing more to read) the earliest parked sample delivers regardless, degrading to the previous behaviour instead of buffering without limit. The fragmented path keeps its decode-time merge and hands the merge key to the sink as the decode timestamp.
basis_media_native.dll (win-x64, MSVC 19.51) and libbasis_media_native.so (android arm64-v8a, NDK clang 18) built from this branch.
The load-identity plumbing around metadata had three gaps, all variants of 'which load does this async continuation belong to': - LoadGeneration advanced only in LoadUrl, so a stale resolver continuation could pass its staleness guard after a LoadLocalPath, a direct LoadSource or a CPU Source assignment replaced the source. It now advances on every source replacement, whichever entry point. - The LoadUrl metadata seed was a bool that stayed armed when routing ended without a load (no resolver installed, or a resolver that errored before LoadSource), so the next unrelated load inherited the abandoned URL as its metadata origin. The seed is now keyed to the generation that armed it and is retired on the no-resolver path and by every non-continuation entry point. - LoadSource wrote the resolved metadata origin back into the caller's BasisMediaSource, where it went stale if the caller mutated the descriptor's Uri and loaded it again. The origin is now retained player-side and only trusted while the same instance reloads with an unchanged Uri; the caller's descriptor is no longer touched. Also: Metadata now returns a snapshot (and OnMetadataChanged delivers one), so external code can't mutate the player's live instance past the change event - ApplyMetadata is the mutation path. Assigning a CPU Source clears the previous media's metadata (event fires with null) instead of leaving it reported as current.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up batch from review feedback on #1, #2 and #4 that reached me outside the PR threads after they merged. One branch as agreed, a commit per concern — five source commits, smallest first:
Malformed-input hardening. Remote MP4 reaches this parser (HLS/CMAF segments, VOD URLs), so stream-supplied fields now get the full treatment:
tfhd/tfdt/trunreads are bounded to their box, atrun's declared per-sample table has to fit inside the box (with a sane per-fragment sample cap), the box walkers' size checks are rewritten in subtraction form so a size nearINT_MAXcan't overflow the comparison, run positions and sample-fit checks are 64-bit, and tick→µs conversion goes through a quotient/remainder helper instead ofv * 1000000 / timescale(which overflows at crafted timescales). Also:cttsversion 0 offsets are now read unsigned per spec,stz2surfaces a clear unsupported-format error instead of starting playback with that track silently missing, and the worst-case Annex B output size moved into a sharedbasis_avcc_annexb_capused by both MP4 paths and RTMP — RTMP previously allocateddlen + 64, which undersizes 1–3-byte NAL length prefixes and silently dropped the access unit.Cleanups. Dead
next_dtsfield, a stale fMP4-only header comment, a nested/*that warned on every TU.Track edit lists (progressive).
edts/elstwere ignored, so a track delayed by an initial empty edit — common muxer output for A/V alignment — played that much early, permanently desynced. The classic walk now maps media time onto the movie timeline: empty edits become a presentation delay, and the first normal edit'smedia_timebecomes the media-time origin (this also picks up ffmpeg's AAC priming edit, so audio/video line up sample-accurately). Fragmented MP4 is untouched — its timeline comes fromtfdt.Load identity scoped to every source replacement (C#, from #2's feedback).
LoadGenerationadvanced only inLoadUrl, so the resolver staleness guard couldn't see a newer load that arrived throughLoadLocalPath, a directLoadSourceor a CPUSourceassignment — it now advances on every source replacement. TheLoadUrlmetadata seed is keyed to the generation that armed it (a seed abandoned by a resolver-less or failed load no longer attaches to the next unrelated load),LoadSourceno longer writes the metadata origin back into the caller'sBasisMediaSource(it's retained player-side, trusted only while the same instance reloads with an unchangedUri),Metadatareturns a snapshot so external code can't mutate the live instance pastOnMetadataChanged, and assigning a CPUSourceclears the previous media's metadata instead of leaving it reported as current. The resolver contract is unchanged — captureLoadGeneration, resolve, compare,LoadSource— so the existing yt-dlp integration works as-is.Pace delivery on decode time, not presentation time. The substantive one. Delivery pacing gated on the pts of whatever the demux thread submitted last, so a video AU whose composition offset exceeds the ~400 ms pacing lead slept the thread past its own decode turn — and the other track's earlier samples, queued behind it on the same thread, arrived late enough for the audio queue to trim them as stale. The video sink now carries the decode timestamp alongside pts; the gate holds on dts while the decoder and caption store keep receiving pts. Demuxers without decode timestamps (TS, RTSP) pass pts for both, which is exactly the old behaviour; RTMP forwards the FLV tag timestamp it already had. The progressive walk had the file-order variant of the same stall — a coarsely interleaved file (a chunk of video ahead of the matching audio) blocked the forward reader on paced video before it could reach the audio bytes — so samples whose file order runs ahead of their delivery turn now park in bounded per-track queues (32 MB; past the bound they deliver anyway, degrading to the old behaviour rather than buffering without limit).
Verification
Closed-loop demux harness (the same one used for #1/#4), old vs new:
ffmpeg -itsoffset 2 … -c copy): audio now delivers at 1.978 s (2 s minus priming) — previously it played at 0 and sat 2 s early forever. The verbose dump shows early audio parking in the queue and releasing in decode order while video streams.Windows DLL (MSVC 19.51) and Android arm64
.so(NDK clang 18) rebuilt from this branch.Editor-validated (Windows desktop, Unity 6000.5.3f1, this branch consumed as the project's media player package):
Console clean throughout.
Notes and limitations
One residual load-identity gap is documented rather than closed: a direct
LoadSourceracing a resolver that already failed silently, with no other load in between, can still consume that load's seed. Closing it needs the continuation to carry an explicit token throughLoadSource's signature, which didn't seem worth the API churn while the plain call has to stay resolver-compatible — the wrong origin lasts only until anything real loads.Gating on dts can release an AU up to its composition offset earlier than the old pts gate; typical offsets are a small fraction of the decode ring's span.
Explicit
tfhd.base_data_offsetis still ignored (pre-existing — fragment placement derives from relativetrunoffsets). Kept out to keep this reviewable; it needs the moof's absolute stream offset threaded through.The harness and fixtures still live out of tree; porting them into the repo as a proper test target is a sensible follow-up if you want the repo to carry tests.
Heads-up: this conflicts with #3's rebuilt
.soby construction; I'll rebase #3 and rebuild it once this lands.