Skip to content

hasmember: one rule in field_search, as in DID-matlab (#236) - #108

Merged
stevevanhooser merged 6 commits into
mainfrom
claude/did-hasmember-quoting
Oct 24, 2026
Merged

hasmember: one rule in field_search, as in DID-matlab (#236)#108
stevevanhooser merged 6 commits into
mainfrom
claude/did-hasmember-quoting

Conversation

@stevevanhooser

@stevevanhooser stevevanhooser commented Oct 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Counterpart of VH-Lab/DID-matlab#239 (fixes VH-Lab/DID-matlab#236). Merge DID-matlab#239 first, with a merge commit, so the hashes recorded here stay on DID-matlab main.

Background

  • SQL injection in MATLAB. DID-matlab put a text hasmember value into its SQL unquoted, so 'value' matched every document and '1 OR 1=1' was SQL injection.
  • MATLAB's SQL couldn't see list members. It couldn't search the members of arrays and cells, which doc2sql stores as '[1 3] double'.
  • Three different rules. hasmember meant something different in each place:
    • MATLAB's fieldsearch used ismember, which compares texts character by character.
    • MATLAB's SQL matched a comma-list item.
    • Python's field_search used in: a substring test, dict keys, and a TypeError (no match) on a scalar number.

DID-matlab#239 gives fieldsearch one rule and makes MATLAB search hasmember by brute force with fieldsearch, as SQLiteDB here always has (_query_struct_to_sql_str returns None for it). The two languages now apply the same rule the same way. This PR is the Python side.

Changes

  • src/did/datastructures.py: field_search's hasmember calls a new _has_member:
    • text in text: the member is the whole value or an item of its comma-separated list (pattern (^|,\s*)M\s*(,|$))
    • anything in a list: an element equals the member
    • a number in an equal number
    • text is never a member of a number, nor a number of text
    • a dict has no members
  • tests/test_hasmember_quoting.py (new):
    • test_rule runs the same 19-case table as DID-matlab's TestHasMemberQuoting/testRule, each case also negated.
    • The other tests mirror that class: text values (including 'value', '1 OR 1=1', values with " and ', '5', ''), numbers including Inf/NaN, ~hasmember, text vs. numbers, a struct field, a missing field, list-valued fields, and each inside an AND and an OR. Each search is compared with field_search and with the documents DID-matlab finds.
  • Bridge:
    • database: matlab_last_sync_hash → 74ae15f1. The decision_log note covers c8adf741..74ae15f1, the DID-matlab brute-force switch (isBruteForceTerm), and the 2c08472c merge of DID-matlab main (#237). Neither did/database.py nor did/implementations/sqlitedb.py changes.
    • fieldsearch: matlab_last_sync_hash 051f535 → f4ed409e, the only commit touching fieldsearch.m since 051f535. The note records the shared rule.

This branch has main merged in, including #107 (the #237 port) and #109 (one decision_log per entry).

Checks run locally

  • black --check src tests: clean (pinned 26.5.1)
  • ruff check src tests: clean (pinned 0.16.5)
  • pytest: 857 passed, 10 skipped, 1 xfailed
  • bin/check_bridge_coverage.py --check file|hash|status|drift|member|missing against a non-shallow DID-matlab checkout at 74ae15f1: 0 problems each

🤖 Generated with Claude Code

https://claude.ai/code/session_01C7XsYS2DKSy2ucFzEEWR3D

@claude
Pin hasmember quoting cases; match a scalar in field_search (DID-matl…
0a59bff 
…ab#236)

DID-matlab put a text 'hasmember' value into its SQL unquoted. Python never
builds SQL for hasmember (_query_struct_to_sql_str returns None and the term
is brute-forced with field_search), so its search was never affected.

tests/test_hasmember_quoting.py mirrors DID-matlab's TestHasMemberQuoting:
text values ('value', '1 OR 1=1', values holding " and '), numbers and
~hasmember, alone, inside an AND and inside an OR, compared with field_search
and with the documents DID-matlab finds. It found that field_search never
matched a number stored as a scalar (5 in 5 is a TypeError) where MATLAB's
ismember(5, 5) does; field_search now compares a scalar with ==.

Bridge: database matlab_last_sync_hash -> c8adf741 with a decision_log note
(did/database.py does not change), and a fieldsearch note.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7XsYS2DKSy2ucFzEEWR3D
@stevevanhooser stevevanhooser mentioned this pull request Oct 24, 2026
Merged
claude added 3 commits October 24, 2026 20:57
@claude
Merge origin/main (DID-matlab#237 port) into claude/did-hasmember-quo…
07c8f16 
…ting

Bridge: database matlab_last_sync_hash -> 2c08472c, the DID-matlab merge of
main (#237, 2c9d8fa7) into the #236 branch; both decision_log notes kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7XsYS2DKSy2ucFzEEWR3D
@claude
Merge origin/main (#109, merged decision_log) into claude/did-hasmemb…
8addbbe 
…er-quoting

#109 merged the database entry's two decision_log keys into one. The #236
note and the 2c08472c merge note move to the end of that merged block;
matlab_last_sync_hash stays 2c08472c.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7XsYS2DKSy2ucFzEEWR3D
@claude
One 'hasmember' rule in field_search, as in DID-matlab (#236)
fdfc261 
DID-matlab f4ed409e gives fieldsearch and its SQL search one hasmember
rule; field_search now matches it too (_has_member): text is a member of
text that is the whole value or an item of its comma-separated list,
anything of a list holding an equal element, a number of an equal
number; text never of a number, nor a number of text; a dict has no
members. Before, `in` tested text for a substring and dict keys.

test_hasmember_quoting.py: test_rule runs the table of DID-matlab's
TestHasMemberQuoting/testRule; new cases for text vs. numbers and a
struct field. Bridge: database and fieldsearch matlab_last_sync_hash ->
f4ed409e, with their decision_log notes rewritten for the shared rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7XsYS2DKSy2ucFzEEWR3D
@stevevanhooser stevevanhooser changed the title hasmember: pin DID-matlab#236 cases; match a scalar in field_search hasmember: one rule in field_search, as in DID-matlab (#236) Oct 24, 2026
claude added 2 commits October 24, 2026 21:16
@claude
test_hasmember_quoting: Inf and NaN members, as DID-matlab
47db1bc 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7XsYS2DKSy2ucFzEEWR3D
@claude
Bridge: DID-matlab now brute-forces hasmember too (#236)
780a3f7 
DID-matlab 74ae15f1 searches hasmember by brute force with fieldsearch,
as SQLiteDB always has, so neither did/database.py nor sqlitedb.py
changes. database matlab_last_sync_hash -> 74ae15f1, its decision_log
note rewritten. test_hasmember_quoting.py adds the cases brute force
now covers in both languages: a missing field and list-valued fields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7XsYS2DKSy2ucFzEEWR3D
@stevevanhooser
stevevanhooser merged commit cc55732 into main Oct 24, 2026
7 checks passed
@stevevanhooser
stevevanhooser deleted the claude/did-hasmember-quoting branch October 24, 2026 21:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

SQLite search: 'hasmember' puts a text value into the SQL unquoted (errors, wrong results, SQL injection)

2 participants

@stevevanhooser @claude