🐌 Test Explorer vs Terminal: 30x Performance Difference in Large Monolithic Project

[Noam Erez (nerez)]

Environment:

• VS Code Python Extension: 2025.6.1
• Python: 3.11.0
• Django: 4.2.7
• pytest: 8.3.3
• Setup: Devcontainer with large monolithic Django project (30 apps, 3150 production code files, 737 test files)

Problem Statement

I'm experiencing a significant performance difference between running pytest via terminal vs VS Code Test Explorer:

• Terminal pytest: ~5 seconds ⚡
• VS Code Test Explorer: ~150 seconds (2.5 minutes) 🐌
• Performance ratio: 30x slower in VS Code

Reproduction Steps

1. Large monolithic project with 30 Python apps, 3150 production code files, 737 test files
2. Run single unit test via terminal: pytest path/to/test.py::test_function → 5s
3. Click "Run Test" button on same test in Test Explorer → 150s
4. Key finding: When I disable 29/30 apps in settings, Test Explorer runs same test in ~5s

Hypothesis & Investigation

The performance degradation appears to scale with the number of Python files/apps in the workspace:

• 1 app enabled: 5 seconds
• 30 apps enabled: 150 seconds

Questions for the team:

1. What overhead occurs during Test Explorer execution that doesn't happen with direct terminal pytest?

2. Does VS Code's test runner:

   • Load/import all workspace Python files before running a single test?
   • Process all apps' dependencies even for isolated tests?
   • Perform workspace-wide discovery on each test run?

3. Is there any caching mechanism for:

   • Python environment setup
   • Dependency resolution
   • Module imports
   • Test discovery results

4. Are there configuration options to:

   • Limit test runner scope to specific directories
   • Cache environment between test runs
   • Skip workspace-wide processing for individual tests

Expected Behavior

Running a single test via Test Explorer should have similar performance to terminal pytest, regardless of workspace size.

Workaround Attempts

• ✅ Confirmed issue is workspace-size related (1 app = fast, 30 apps = slow)
• ❌ Tried various pytest configuration options without improvement
• ❌ No obvious VS Code Python settings to limit scope

Impact

This performance difference makes Test Explorer unusable for our team's migration from PyCharm to VS Code. Any guidance on optimization strategies or configuration options would be greatly appreciated!

Thank you