Skip to content

Reduce CI model test matrix runtime#1111

Draft
ochougul wants to merge 1 commit into
mainfrom
reduce_test_time
Draft

Reduce CI model test matrix runtime#1111
ochougul wants to merge 1 commit into
mainfrom
reduce_test_time

Conversation

@ochougul

Copy link
Copy Markdown
Contributor

Summary

  • add QEFF_TEST_SCOPE=pr|exhaustive and wire Jenkins to default PR runs to pr
  • route slow parametrized model suites through a shared test matrix selector
  • shrink model/diffusion test configs to tiny valid shapes while preserving architecture/feature coverage keys
  • keep exhaustive scope available for full configured model lists

Validation

  • pip install -e . using ~/envs/main
  • ruff check <changed-python-files>
  • ruff format --check <changed-python-files>
  • pytest tests/unit_test/models/test_model_quickcheck.py -k 'model_matrix' -q
  • QEFF_TEST_SCOPE=pr pytest tests/transformers/models/causal_lm_models tests/transformers/caching tests/transformers/sampler tests/transformers/spd tests/transformers/subfunction -m '((llm_model or feature) and not qnn) and (not full_layers) and (not few_layers)' --collect-only -q
  • QEFF_TEST_SCOPE=pr pytest tests/transformers/models/image_text_to_text tests/transformers/models/reranker tests/transformers/models/embedding_models/test_qwen3vl_embedding_mad.py -m '(multimodal) and (not qnn) and (not full_layers) and (not few_layers)' --collect-only -q
  • QEFF_TEST_SCOPE=pr pytest tests/diffusers -m 'diffusion_models' --collect-only -q
  • QEFF_TEST_SCOPE=pr pytest tests -m '(not on_qaic) and (not finetune) and (not full_layers) and (not few_layers)' --ignore tests/vllm --ignore tests/unit_test --ignore tests/nightly_pipeline --collect-only -q

Notes

  • AI assistance used for implementation and validation command execution.
  • Human review should confirm the tiny configs still represent the intended architecture/feature paths before merge.

Signed-off-by: ochougul <ochougul@qti.qualcomm.com>
@vbaddi vbaddi marked this pull request as draft June 23, 2026 14:44
@vbaddi

vbaddi commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

per pr fast lane is being is worked here: #1075 and #1094

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants