Skip to content

test: cover performance audit findings before fixing them - #46

Merged
jplacht merged 2 commits into
mainfrom
test/perf-audit-coverage
Sep 27, 2026
Merged

jplacht merged 2 commits into
mainfrom
test/perf-audit-coverage

Conversation

@jplacht

@jplacht jplacht commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Test-only PR (no application code changes) that pins down every code-level finding of the backend performance audit before any fix lands.

  • Known defects are xfail(strict=True) tests with an audit: reason. CI stays green; once a fix lands the test XPASSes, which fails the run until the marker is removed together with the fix. All 40 were verified to fail on their own assertion via --runxfail.
  • Real cache in tests: new PatternLocMemCache + locmem_cache fixture (backend/tests/cache_backends.py, conftest.py). The test DummyCache never stores and has no delete_pattern, so cache/invalidation bugs were invisible; the old tests only passed by mocking delete_pattern.
  • Replaced implementation-coupled tests: the four tests asserting delete_pattern was called with an exact pattern now assert on actual cache hits/misses and response data.

Coverage by finding

Finding Tests
Keyspace-scan invalidation scans per delete/save, user 1 vs 11 over-match, passing guards for all correct invalidation paths
Unthrottled search / stampede throttling of search, multiple, search_single; 100-id cap; order-independent cache key; concurrent rebuild
Oversized JSON / N+1 empire_state loaded in 3 views, CX list/retrieve N+1, shared-view N+1, repeated cx_data
Auth overhead API-key auth query count, Basic auth, pre_save double read
Snapshot task cache eviction per empire, lost-update race
Lows / side findings CSV re-render, gzip, pre-cache DB lookups, webhook counter, httpx reuse, redundant CXPC index, per-user Cache-Control: public, stale planet/building caches, api key in task args, insight ticker bug

New defects surfaced

  • Renaming a plan or empire never invalidates the cached CX list, which nests both (stale up to 1h).
  • sync_state runs two keyspace scans, although empire_state is in no cached payload.

Notes for the fixes

  • Contracts pinned by tests: throttle scope planet_search; planets/multiple accepts at most 100 ids.
  • Tests implying API/policy changes (check the frontend first): cx_data nesting in the plan list, owner empires on the public shared view, removing Basic auth, gzip (drop if the proxy compresses).
  • Not testable here: worker concurrency, ASGI worker setup, DEBUG default (deployment config); SSE (no pytest-asyncio).
  • Signal test files are prefixed (test_planning_signals.py, test_user_signals.py): the test tree has no __init__.py, so duplicate basenames collide.

Test plan

  • uv run pytest: 243 passed, 40 xfailed
  • uv run pytest --runxfail: exactly the 40 xfails fail
  • uv run ruff check / uv run ruff format --check
  • uv run ty check --exclude "**/migrations/*.py"

🤖 Generated with Claude Code

Behavioural tests for every code-level finding of the performance audit.
Known defects are documented as xfail(strict=True) tests, so they fail
loudly once fixed and the marker gets removed with the fix.

- add PatternLocMemCache + locmem_cache fixture: a real in-memory cache
  with delete_pattern, as DummyCache never stores and lacks delete_pattern
- replace tests mocking delete_pattern with cache-hit based assertions
- cover cache invalidation cost and scope, N+1 queries, oversized JSON
  loads, stampedes, throttling, auth overhead, snapshot lost update,
  stale gamedata caches and per-user Cache-Control

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codacy-production

codacy-production Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Coverage ∅ diff coverage · +0.12% coverage variation

Metric Results
Coverage variation ✅ +0.12% coverage variation (-1.00%)
Diff coverage ✅ ∅ diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (8de36d9) 3116 2808 90.12%
Head commit (11e4b9e) 3278 (+162) 2958 (+150) 90.24% (+0.12%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#46) 0 0 ∅ (not applicable)

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

The test enqueued the real Celery task, which needs a reachable broker and
failed in CI with its placeholder broker URL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jplacht
jplacht merged commit bee9950 into main Sep 27, 2026
6 checks passed
@jplacht
jplacht deleted the test/perf-audit-coverage branch September 27, 2026 09:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant