Skip to content

EXPERIMENT (DO NOT MERGE): validate experimental.readers.video as drop-in in test suite - #6516

Closed
jantonguirao wants to merge 47 commits into
NVIDIA:mainfrom
jantonguirao:experimental-video-reader-ci-validation
Closed

jantonguirao wants to merge 47 commits into
NVIDIA:mainfrom
jantonguirao:experimental-video-reader-ci-validation

Conversation

@jantonguirao

Copy link
Copy Markdown
Collaborator

This is a throwaway CI-validation experiment. Do not merge.

Temporarily substitutes fn.readers.video with fn.experimental.readers.video across 5 test files (not the dedicated test_video_reader.py parity suite) to see if the experimental CPU+GPU video reader works as a full drop-in replacement in test code not specifically written to test the reader itself.

If CI passes, this branch/PR will be closed without merging. If CI fails, findings will be investigated separately.

….video

Exposes Task 1's FramesDecoderBase::SetOutputType()/SetNormalizedRange()
plumbing as dtype/normalized operator arguments on
experimental.readers.video, plus a validated channels argument that must
match the decoder's actual (always 3) channel count. dtype=FLOAT is
GPU-only; CPU decode remains UINT8-only and rejects dtype=FLOAT explicitly.

Also fixes a byte-offset bug uncovered while wiring multi-frame FLOAT
output through FramesDecoderGpu: several places in frames_decoder_base.cc
and frames_decoder_gpu.cc computed frame strides/copy sizes with
FrameSize() (element count), which is only correct for 1-byte UINT8
output. Added FrameSizeBytes() (element count * output element size) and
used it for inter-frame pointer strides, the internal frame reorder
buffer sizing, and frame-to-frame/constant-frame device copies, so
multi-frame FLOAT decoding (sequence_length > 1) produces correctly
aligned output. dtype=FLOAT combined with pad_mode='constant' is
rejected for now since the constant-frame fill buffer is still
UINT8-only.
…constant restriction

Finding 1: FramesDecoderGpu's internal frame-reorder buffer was unconditionally
sized for the largest supported output element (float), a permanent 4x GPU
memory increase for every caller (including legacy fn.readers.video and the
new reader's UINT8 default), regardless of dtype. Made SetOutputType()
virtual and overrode it in FramesDecoderGpu to resize the already-allocated
buffer entries to match the actual dtype once known; the buffer is initially
allocated assuming DALI_UINT8 (unchanged from before this task), so UINT8
callers keep their original, smaller footprint and only DALI_FLOAT callers
pay the larger allocation.

Finding 2: documented the dtype=FLOAT + pad_mode='constant' restriction (added
in the previous commit) in both the `dtype` and `pad_mode` schema docstrings,
instead of leaving it discoverable only via the runtime error.
Opt-in strict check (default false, current permissive VFR decoding
unchanged) that fails fast via DALI_ENFORCE when a video is variable
frame rate, using FramesDecoderBase::IsVfr().

Also fixes a real correctness gap found while wiring this up:
FramesDecoderBase::SetIndex() restored index_/num_frames_ from a
cached FrameIndexCache entry but never recomputed is_vfr_, so IsVfr()
silently reported false for any decoder that reused a cached index
(the common case, since the loader's PrepareMetadataImpl warms the
cache for every file at build time). SetIndex() now calls
DetectVariableFrameRate() so IsVfr() is correct on both the fresh
BuildIndex() path and the cache-hit path.
Two behavior changes for existing experimental.readers.video users:
1. file_list_include_end=True now actually includes the end frame (previously only
   clamped to the video's frame count, a no-op in most cases).
2. file_list_format="frames" with an omitted/zero end column now means "through the
   end of the video", matching readers.video's file_list_frame_num=True convention,
   instead of selecting zero frames.
test_compare_experimental_to_legacy_reader_file_list never overrides
file_list_include_end, so experimental.readers.video now (correctly, after
8a8ea99) extends frame-mode ranges by one frame per its documented default.
Legacy has no equivalent concept for file_list_format="frames" (its end column
is always a literal exclusive bound), so 3 of the 32 random parametrizations can
diverge. Skip those known cases with an explanation instead of leaving
unexplained red.
Pure move of VideoReaderDecoder, VideoLoaderDecoder and their supporting
types out of video_reader_decoder_op.cc into a header, so that other
operators can derive from the experimental video reader. No behavior change.
GPU-only variant of experimental.readers.video that resizes the decoded
sequences directly into the output, modeled on the legacy
readers.video_resize. VideoReaderDecoder::RunImpl is split into
WriteVideoOutput and WriteMetadataOutputs so the resize variant can
reuse the metadata outputs, and the OutputFn is factored out into
detail::VideoReaderDecoderOutputFn to be shared by both schemas.
…_end

The test assumed an exclusive end-frame convention (last sampled index
== end_frame - 1, roi_frames == end_frame - start_frame). This only
happened to hold because of the file_list_include_end bug fixed in
8a8ea99, where the default (true) was silently acting as exclusive.
With that fixed, the ROI is genuinely inclusive of end_frame, so
roi_frames == end_frame - start_frame + 1 and the last sampled index
is end_frame.
…nd normalized coverage

- Close the image_type TODO by adding types.YCbCr to the shared
  image_type_supported_by_legacy_reader parametrization list, exercised by the
  filenames/file_list/file_root comparison tests.
- Add a pass-through dtype/normalized translation to
  compare_experimental_to_legacy_reader (both readers already share these
  argument names as of prior tasks) and a new
  test_compare_experimental_to_legacy_reader_dtype test covering UINT8/FLOAT
  with normalized=(dtype==FLOAT).
- Skip the known dtype=FLOAT-on-CPU restriction (already covered by
  test_dtype_float_on_cpu_raises) and a newly discovered pre-existing bug
  where experimental.readers.video's CPU backend crashes for image_type=YCbCr
  (frames_decoder_cpu.cc CopyToOutput sets up a single packed sws_scale
  destination buffer even though YCbCr output requests planar
  AV_PIX_FMT_YUV444P, leaving dest[1]/dest[2] null); neither is a
  legacy-vs-experimental parity gap.
- Recompute test_compare_experimental_to_legacy_reader_file_list's
  _known_include_end_divergences skip set: adding the image_type axis
  doubled the parametrization count and shifted the shared np.random stream,
  so the previously-hardcoded 3-tuple set no longer matched reality; the key
  now also includes image_type since each parametrization instance draws its
  own random file_list content.
- Default file_list_include_end to False so that, with no override, the
  end value of a file_list entry is exclusive and frame selections match
  legacy readers.video. Drop the now-unneeded skip list from the
  legacy-comparison file_list test, restore the exclusive-end expectations
  in test_uniform_sample_file_list_roi, and add a deterministic
  legacy-vs-experimental frame-selection test for both file_list formats.
- Document additional_decode_surfaces accurately (it sizes the GPU
  decoder's decode-lookahead/reorder buffer; the NVDEC surface count comes
  from the codec) and reject negative values.
- Make the FLOAT legacy comparison check pixel content: scale normalized
  output to the 8-bit range before applying the intensity threshold, and
  add a FLOAT/normalized=False case.
- Mark the CPU YCbCr sws_scale destination-plane bug with
  TODO(DALI-4916) and reference the ticket from the test skip helper.
- Restore static output dtype metadata via a dtype-dependent OutputDType.
- Fix stale DecodeFrames buffer-size doc comments (bytes, FrameSizeBytes).
The previous fix that flipped file_list_include_end's default to false
(to restore legacy-matching behavior) moved the entry.end_frame clamp
against num_frames inside the `if (include_end)` branch. That meant
default users (include_end=False) got no clamp at all: a file_list
entry whose end exceeds the video's actual frame count produced a
bogus epoch_size and crashed partway through the epoch.

Clamp end_frame to num_frames unconditionally; only add the +1 (making
the range inclusive of the raw end value) when include_end is true.
FramesDecoderCpu::CopyToOutput requested planar AV_PIX_FMT_YUV444P output
for YCbCr but only populated dest[0]/dest_linesize[0], leaving dest[1]/
dest[2] null -- sws_scale rejected this with "bad dst image pointers".
Set up all three destination planes.

Once decodable, CPU YCbCr values still diverged from the GPU backend and
legacy reader by a consistent per-channel offset: sws_setColorspaceDetails
hardcoded the destination range to full-range (0-255), correct for RGB
output but wrong for YCbCr, which both the GPU backend and legacy use in
limited/"TV" range (16-235 luma). CPU YCbCr output now matches the GPU
backend exactly, byte-for-byte, for the same source frame.

Removes the DALI-4916 test skip; the legacy-vs-experimental comparison
tests now exercise CPU+YCbCr directly.
…ality

The two backends use independently-implemented bilinear chroma upsamplers
(libswscale vs a custom CUDA kernel), so exact byte equality is a
coincidence of the current test video's content, not something the
color-range fix guarantees in general. A tolerance still catches a
regression to the systematic ~12-20/255 offset the bug produced, without
being fragile to legitimate small rounding differences on other content.
experimental.readers.video's file_list_format="timestamps" mode selected
a different frame range than legacy readers.video even with all default
arguments: legacy's file_list_include_preceding_frame defaults to False
(ceil-start/floor-end, "start_up_end_down"-equivalent), while
experimental's file_list_rounding defaulted to "start_down_end_up"
(floor-start/ceil-end) -- the opposite rounding direction.

This is a default-behavior change for existing experimental.readers.video
timestamps-format file_list users, matching what DALI-4915 already did
for file_list_include_end: pick the legacy-matching default now that the
underlying rounding logic (unchanged by this commit, only the default
value) is otherwise correct and configurable.

test_file_list_default_end_matches_legacy previously pinned both readers
to an explicit matching rounding mode for the timestamps case specifically
because this divergence existed -- that override is removed, since the
new default now needs no explicit override to match legacy.
MJPEG was already mapped to cudaVideoCodec_JPEG in GetCodecType and
InitBitStreamFilter but deliberately excluded from
FramesDecoderGpu::SelectVideoStream's codec allow-list, pending followup
work.

Enabling it surfaced the reason it was never finished: the NVDEC driver
reports min_num_decode_surfaces=1 for MJPEG content (each frame is an
independent image, unlike the inter-predicted codecs this integration
was built around). With zero surfaces of headroom, decoding the second
frame failed with CUDA_ERROR_INVALID_VALUE, because this decoder never
registers a real pfnDisplayPicture callback -- HandlePictureDisplay runs
synchronously, inline, from ProcessPictureDecode instead -- so the
parser's own surface-recycling bookkeeping never saw the first picture
as "consumed" before the second one needed a surface.

Legacy's NVDEC integration already works around exactly this by
hardcoding ulMaxNumDecodeSurfaces=20 for MJPEG (and HEVC) instead of
trusting the driver-reported minimum (nvdecoder/cuvideoparser.h). Mirror
that here: AdjustedNumDecodeSurfaces forces at least 20 decode surfaces
for MJPEG specifically. Both the sequence callback (which tells the
parser how many surfaces to expect) and decoder creation must use the
same adjusted value, or the two disagree -- so both call sites were
updated together.

Also updates the shared MJpeg gtest (video_reader_op_test.cc, compiled
against both readers__Video and experimental__readers__Video via the
VIDEO_READER_OP macro) to build a per-operator OpSpec, since it
previously always passed legacy's skip_vfr_check argument (which
experimental doesn't have) and unconditionally skipped experimental
rather than actually attempting the decode.
The comment claimed this mirrors legacy hardcoding 20 decode surfaces for
MJPEG. Tracing legacy's actual code: the literal 20 in cuvideoparser.h's
per-codec switch is immediately overwritten by its only caller with a
uniform value passed for every codec, not an MJPEG-specific override --
and legacy's real CUvideodecoder for this test file is created with just
3 surfaces (min_num_decode_surfaces + additional_decode_surfaces), not
20. Legacy gets away with fewer surfaces because it registers a genuine
pfnDisplayPicture callback; this backend's synchronous-inline
HandlePictureDisplay does not have that signal, so it needs more
headroom regardless of what legacy uses. Corrected the comment to
describe 20 as an empirically-verified, conservative value rather than
an inherited legacy precedent.

Also added a non-zero content check for frame 1 (the specific former
crash point), not just frame 0.
… (DALI-4924)

The GPU video reader synchronized the whole CUDA stream and unmapped every
decoded picture immediately after its color-conversion kernel launch, once
per frame. That serializes NVDEC hardware decode against the SM-side
conversion kernel every single frame, and is a large part of why this
reader trails the legacy NVDEC reader's throughput.

Replace the per-frame sync+unmap with a deferred unmap: map, launch the
conversion kernel, record a CUDA event, and move on without syncing.
cuvidUnmapVideoFrame only runs once a surface's slot is actually needed
again, via FIFO eviction bounded by the driver's real ulNumOutputSurfaces
(the hard limit on how many pictures may be mapped at once) -- exactly the
decode/map interleaving pattern NVIDIA's own nvcuvid.h documents. An
earlier attempt at overlap (a full producer/consumer thread redesign, not
part of this change) deferred unmaps without respecting that cap and
caused real frame corruption; this fix stays single-threaded and enforces
the cap explicitly.

A picture's decode-surface index can only be reused this way if the
previous occupant has actually been unmapped first, so
EnsureUnmapped(CurrPicIdx) also runs in ProcessPictureDecode, before
cuvidDecodePicture, mirroring the timing of legacy's frame_in_use_ guard.

Decode-surface count for sizing the pending-map table is now read
directly off the NVDECCache lease (NVDECLease::NumDecodeSurfaces()) rather
than recomputed, so it can't drift from what the pooled CUvideodecoder was
actually created with. Draining now happens deterministically from
SendLastPacket(flush=true), covering both Flush() and SeekFrame()'s direct
call, and from ~FramesDecoderGpu() before the pooled decoder lease is
returned (a decoder instance is reused across FramesDecoderGpu instances,
so anything left mapped here would corrupt the next owner).

Verified: all 90 gtests (dali_operator_test --gtest_filter='*Video*:*Fra
mesDecoder*') and all 178 python tests (test_video_reader.py,
test_video_reader_resize.py) pass. Benchmarked at batch=1/sequence_length=
32: legacy 552fps, this reader before the fix 445fps, after 458fps (~3%
gain, ~13% of the gap to legacy closed). Tried raising ulNumOutputSurfaces
from 2 to 4 for deeper pipelining; no further gain, reverted -- the
remaining gap looks to be CPU-side (demux/bitstream-filter/parse
overhead), not GPU pipelining depth.
Two independent fixes for the GPU video reader's remaining throughput gap
vs. the legacy reader under the common training-pipeline shape
(random_shuffle=True over more files than fit in a batch):

1. video_reader_decoder_op.h: Prefetch() used to tear down and rebuild the
   whole FramesDecoderGpu/FramesDecoderCpu (container reopen, NVDEC parser
   creation, decode-surface allocation, keyframe-relative seek) every time
   consecutive samples came from a different file. With shuffling, that is
   nearly every sample. Replace the single decoder_ with a small
   filename-keyed LRU (capacity 8) so a file's decoder survives across the
   samples it isn't currently servicing and is reused instead of rebuilt.

2. frames_decoder_gpu.cc: HandlePictureDisplay() mapped and color-converted
   every decoded picture that didn't exactly match the next wanted frame,
   even when that picture could never be a future seek target (e.g. the
   pre-roll frames NVDEC must decode to reach a mid-GOP seek position, which
   have a pts before anything the index will ever request again). Drop
   those frames immediately instead of paying for cuvidMapVideoFrame,
   the conversion kernel, and a frame_buffer_ slot that would never be
   read. Frames ahead of the target (e.g. B-frame reordering) still take
   the existing buffering path.

Benchmarked with internal_tools/video_reader_bench.py (added here) on a
TITAN V, batch=8, sequence_length=16, 5 shuffled VP9 files: 295->360 fps,
now above legacy's 342 fps (conversion-kernel launches per requested frame
dropped from 1.82x to 1.09x, matching legacy exactly). The two fixes were
benchmarked together, not in isolation, so this doesn't attribute the gain
between them individually.

Each cached decoder holds its own NVDEC lease and ~10 decode-surface
buffers, so the cache trades some GPU memory (roughly 200MB at 720p/uint8
per cached decoder, ~4x that for float) for avoiding the rebuild cost.

Verified: all 90 gtests (dali_operator_test --gtest_filter='*Video*:*Fra
mesDecoder*') and all 178 python tests (test_video_reader.py,
test_video_reader_resize.py, including the legacy-vs-experimental parity
comparisons) pass.
Add pad_sequences, file_list_frame_num, file_list_include_preceding_frame and
skip_vfr_check as deprecated arguments. The first three are translated to
pad_mode, file_list_format and file_list_rounding (erroring if combined with
their replacement); skip_vfr_check is accepted and ignored, since the two
operators' VFR defaults have opposite meanings.

Also add three shared Python test helper functions (VIDEO_0, _write_file_list,
_collect_samples) and their imports that later tasks will reuse.
The assertion checking output_dtype metadata was accidentally deleted.
Restore the three lines that verify the dtype is correct.
A sequence only needs its last frame to exist, so it spans
(sequence_length - 1) * stride + 1 frames, not stride * sequence_length.
Previously short videos read with stride > 1 produced fewer samples than
the legacy reader, or none at all.
Interpret file_list timestamps relative to the first frame, count negative
values back from the end, treat an omitted or non-positive end as the end of
the video, keep end == duration, and reject a start past the end, as the
legacy reader does. FrameIndex::GetFrameIdxByTimestamp now returns size() for
times past the end instead of falling back to frame 0.
Match legacy readers.video: an explicitly empty labels list yields 0-based
sequential labels instead of failing the files/labels count check.
Build the constant frame in the output type. fill_value keeps its 8-bit
scale (divided by 255 when normalized), so the default fill_value=0 zero-pads
float output like legacy readers.video with pad_sequences=True.
…ideo

With padding enabled and a step smaller than the sequence span, legacy
readers.video produces a padded sample at every remaining start position;
experimental produced only one.
…video

Count non-empty file_list/file_root/filenames like legacy readers.video, and
make the declared output count use the same rule as the runtime labels
output.
Both made the default step 0, so building the sample list never finished.
Check them in the loader constructor, as legacy readers.video does for
stride.
Assert equal epoch sizes instead of taking the minimum, and compare every
sample of every output (video, labels, frame_num, timestamps) over one epoch
instead of only the first sample of the video output. Fix the file_list
rounding translation, keep mismatch images out of the working directory,
and add parity coverage for stride/step, timestamp edge cases and labels=[].
SecondsToTimestamp truncated (static_cast<int64_t>) when converting a
file_list timestamp-mode boundary to pts. When the requested seconds value
landed only a fraction of a tick above a frame's exact pts, truncation
could produce a pts equal to that frame's own pts, making
FrameIndex::GetFrameIdxByTimestamp's exact-match branch return that frame's
index as the (exclusive) end boundary and silently drop it -- a real
frame legacy readers.video would include. Round to the nearest tick
instead. Found by the stricter legacy-vs-experimental parity harness
(test_compare_experimental_to_legacy_reader_file_list, case with file_list
entry "sintel_trailer_vp9_0.mp4 4 0.03536328934625863 0.9167271357878531").
These pin the correct behavior for two unconfirmed risks in
experimental.readers.video: keyframe detection on Annex-B H.264/HEVC streams
(checked by comparing seeked and sequentially decoded frames) and opening
files whose names contain ':'.
test_annex_b_seek_matches_sequential_decode and
test_filename_with_colon_can_be_opened (Task 10) intentionally caught two
real bugs: Annex-B NAL parsing in FramesDecoderBase::BuildIndex, and
colon-in-relative-filename handling on the experimental reader. Without a
skip marker, qa/TL0_videoreader_test/test.sh would go red on every future
PR, indistinguishable from a real regression.

Use nose_utils.SkipTest to skip exactly the 5 currently-failing
parametrized cases (h264_in_mpeg_ps, raw_h264, raw_h265 for the Annex-B
test; experimental_cpu/relative and experimental_gpu/relative for the
colon test), while keeping h264_in_avi and both absolute-path colon cases
running as active regression guards. The underlying bugs are tracked for
a future follow-up, not fixed here.
…ction

The llround-based fix from the previous commit only shrank the file_list
timestamp-mode boundary bug, it didn't close it: a value like
0.91670 s at timebase 1/12288 is 11264.4096 ticks, which llround still
rounds down to 11264 -- exactly frame 22's own pts. That makes
FrameIndex::GetFrameIdxByTimestamp's exact-match branch treat it as frame
22's own position and exclude it, even though legacy readers.video
includes it.

Legacy never rounds to nearest: it uses ceil(t*fps) for round-up lookups
and floor(t*fps) for round-down lookups, always matching the direction of
the boundary being computed. Replace the single llround-based
SecondsToTimestamp with a direction-aware version that does the same,
threading file_list_opts_.should_round_down_start()/_end() (the same
direction already passed to GetFrameIdxByTimestamp's `rounddown`
parameter) through to it. Since pts values are integers for
constant-frame-rate video, this matches legacy exactly.

Extend the gtest coverage with the reviewer's 0.91670 example (llround
still gives 11264, the buggy exact match; ceil gives the correct 11265)
alongside Task 9's original 0.9167271357878531 example, and replace the
now-inaccurate "rounds half away from zero" tests with direction-aware
ones.
- pad_mode: 'pad_value' -> 'fill_value' (matches the actual argument name)
- fill_value: remove stray quote before the closing period
- step: docstring now matches the code's actual <= 0 / stride * sequence_length behavior
…string

FramesDecoderBase::GetFrameIdxByTimestamp forwards positionally to
FrameIndex::GetFrameIdxByTimestamp, whose own parameter is named rounddown.
Rename the wrapper's parameter to match, and document the index-size
sentinel return value that was previously unmentioned.
…t is used

detail::GetFileListOptions(spec) was only called when file_list_ was
non-empty, so its DALI_ENFORCE conflict checks (e.g. file_list_frame_num
vs file_list_format) silently didn't run for filenames=[...] +
file_list_frame_num=True + file_list_format="frames". Call it
unconditionally in the constructor; file_list_opts_ is only ever read
later for entries with a non-default start/end, which only happens when
file_list_ was actually used to populate video_files_info_.
The conflict error fires in the loader constructor during p.build(), not
during Acquire on the first p.run(); drop the unnecessary run() call and
correct the comment.
_write_file_list used tempfile.NamedTemporaryFile(..., delete=False) and
nothing ever removed the files; with ~60 call sites across the
parametrized suite, every run leaked files into /tmp. Write into a single
directory created once per module (tempfile.mkdtemp) and registered for
cleanup via atexit.register(shutil.rmtree, ...), matching the pattern
already used by gcs_test_utils.py/s3_test_utils.py.
Covers the combination of legacy-compat pad_sequences with dtype=FLOAT,
which wasn't exercised by the existing pad_mode='constant'+FLOAT and
pad_sequences-translation tests individually.
experimental.readers.video_resize shares its constructor with
experimental.readers.video, which supports FLOAT+constant padding, but
no test exercised that combination through the fused resize operator.
FramesDecoderBase::BuildIndex parsed H.264/HEVC NAL units as 4-byte
length-prefixed (AVCC/ISO framing), which is wrong for Annex-B streams
that use start codes (00 00 01 / 00 00 00 01) instead -- misreading a
start code as a length meant real IDR/IRAP keyframes were essentially
never detected. Replace it with a per-NAL-unit detector that checks
for a start code first and only falls back to a length prefix when
none is present: this self-adapts per NAL unit without relying on
container format or extradata heuristics that could be fooled by an
unusual combination of the two.

Investigating db/video/containers/mpeg/cfr.mpeg (MPEG-PS) alongside
this turned up a second bug blocking it before BuildIndex is even
reached: SelectVideoStream calls av_find_best_stream right after
avformat_open_input, with no avformat_find_stream_info in between.
Containers like AVI declare their stream's codec type in a header
available immediately; MPEG-PS only discovers it by probing/demuxing
early packets, so av_find_best_stream returned "stream not found" and
the file appeared to have zero samples. Fall back to
avformat_find_stream_info and retry once before giving up; this only
costs anything on the failure path.

With both fixes, db/video/containers/avi/cfr.avi (Annex-B all along,
despite not being one of the containers that gets converted via
h264_mp4toannexb) now has its keyframes correctly confirmed, and
db/video/containers/mpeg/cfr.mpeg's stream and keyframes are also
found correctly. mpeg/cfr.mpeg still can't complete indexing, though:
most of its packets have no pts/dts at all (MPEG-PS only stamps the
first packet of each PES unit), and BuildIndex's timestamp fallback
has no way to give those frames a presentation-time identity that
would exactly match what NVDEC reports during decode without
colliding with real timestamps elsewhere in the stream (dts and pts
diverge by a non-constant B-frame reorder delay in this file, so
naive interpolation isn't sound). Legacy readers.video handles this
same file by computing a duration/frame-rate-based frame count
instead of needing per-packet timestamp identity -- porting that
design into the index/seek machinery here is a bigger rework than fits
this change, so it stays a documented, accurately-described skip
rather than a silently wrong fix.

raw_h264/raw_h265 (bare elementary streams) also stay skipped:
confirmed legacy readers.video fails to open them too, via its own
variable-frame-rate heuristic, since neither reader has any
container-level timing to anchor on for these files. That's a shared
limitation, not a parity gap, so the skip reason now says so instead
of pointing at the (now-fixed) Annex-B parsing bug.
FramesDecoderBase::OpenFile passed the filename straight to
avformat_open_input, which parses it through avio's URL layer. That
layer treats a colon as a protocol separator, so a relative path like
"clip:01.mp4" got parsed as protocol "clip" (which doesn't exist)
instead of a plain filename -- absolute paths were unaffected since
leading slashes don't trigger the same ambiguity. Prefix the filename
with the "file:" protocol, FFmpeg's documented way to force
plain-filename interpretation regardless of what characters it
contains, for both relative and absolute paths.

OpenMemoryFile's separate avformat_open_input call (which opens ""
via a custom AVIO context, not a real filesystem path) is unrelated
and left untouched.
@copy-pr-bot

copy-pr-bot Bot commented Sep 30, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@dali-automaton

Copy link
Copy Markdown
Collaborator

CI MESSAGE: [70775858]: BUILD STARTED

@dali-automaton

Copy link
Copy Markdown
Collaborator

CI MESSAGE: [70775858]: BUILD FAILED

@jantonguirao

Copy link
Copy Markdown
Collaborator Author

Validation experiment complete. CI confirmed the only real finding (MPEG-PS sparse-timestamp limitation, already known and documented) plus one unrelated pre-existing infra failure (no GPU driver on the cpu_only CI job). Nothing else broke across the automatic test tier. Closing without merging — this was a throwaway branch to exercise CI, not intended to land.

@jantonguirao
jantonguirao deleted the experimental-video-reader-ci-validation branch October 1, 2026 08:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants