Add semantic schemas and access-rights support#69
Open
yoavnash wants to merge 45 commits into
Open
Conversation
- Remove dead-code group objects with underscore IDs (interally_public, externally_public) that conflicted with config defaults (hyphens) - Add refresh_public_groups(config) so DSMS can update the module-level INTERNALLY/EXTERNALLY_PUBLIC_GROUP constants after its own config is set, fixing the import-time staleness bug - Fix min_access_level / max_access_level to return Role (not int) and raise a clear ValueError for operations with no role mapping (e.g. CREATE) - Fix inverted comments in serialize_role_json - Cache user_groups and users on the DSMS instance (pattern matches ktypes); add refresh_user_groups() and refresh_users() invalidation methods - Fix get_user_by_id: accept dsms as first argument (consistent with all other util functions) and return a typed User object instead of a raw dict - Expose get_user_by_id as DSMS.get_user(user_id)
test_access_extended.py: - Role hierarchy ordering and >= comparison - min/max_access_level return Role instances (not int) - min/max_access_level raise ValueError for unmapped operations (CREATE) - Error message lists valid operations - serialize_role_json: JSON mode → int, Python mode → name string - model_dump(mode='json') produces integer roles for wire format - model_dump(mode='python') produces string role names - Round-trip from backend dict (int roles) through model and back - user_by_role property - None user_access/group_access converted to [] test_groups.py: - Group model: basic, subgroups, deeply nested - GroupList: flat flattening, flat returns BaseGroup, by_id, by_name (including subgroup traversal) - User model: basic, with groups - UserList: by_id, by_username, by_name, __getitem__, missing key - INTERNALLY/EXTERNALLY_PUBLIC_GROUP IDs use hyphens - refresh_public_groups with custom config and without arg - DSMS.user_groups caches result; refresh_user_groups() re-fetches - DSMS.users caches result; refresh_users() re-fetches - DSMS.get_user() returns typed User object; raises on 404
added 15 commits
June 5, 2026 06:44
New features: - Full v2 ktype CRUD via _v2_create/get/update/delete/list_ktypes - Remote registry operations: _v2_list_remote_ktypes/schemas/versions, _v2_export/import_ktype, _v2_refresh_ktype, _v2_remote_diff, _v2_restore_stash; exposed as DSMS.get_v2_ktype*, create_v2_ktype, etc. - KTypeV2, KTypeSpec, KTypeSpecPayload, CreateKTypeRequest, RemoteDiffOut, RemoteKTypeSummary, RemoteKTypeVersion, RemoteSchemaInfo models - _get_ktypes_by_parent / DSMS.get_ktypes_by_parent: fetch ktype descendants - schema_data support: KItemSchemaData, KItemSchemaDataList, _get/put/delete_schema_data; diff logic in _update_schema_data; exposed as DSMS.get_schema_data(kitem_id) - SPARQL context queries: SparqlInterface.query_context, graph_context - DSMS.search gains contexts and attachment_extensions parameters - DSMS.get_kitems gains name filter parameter - KItemCompactedModel gains avatar_exists, has_contexts, attachment_extensions fields Fixes: - attachment_extensions guard: if attachment_extensions is not None (consistent with contexts guard above it) - _get_ktypes_by_parent docstring: remove misleading "v1" qualifier - KItemSchemaData added to TYPE_CHECKING imports in dsms.py
USER as a role name was ambiguous — everyone is a user of the system. MEMBER better describes the capability: a member of the platform or a group can read the item. The progression MEMBER → CONTRIBUTOR → OWNER → ADMIN now reads as a natural escalation of responsibility. Also resolves the tension with INTERNALLY_PUBLIC_GROUP: assigning member-level access to the internal-public group reads naturally as 'all platform members can access this item'.
The user-service GET /api/users/{id} endpoint returns these fields.
Declaring them as Optional allows SDK consumers to access full name
information without falling back to raw dicts.
- Add CHANGELOG.md and CONTRIBUTING.md - Update README: capabilities list, compatibility table, copyright year, replace Authors section with Contributing pointer - Update docs/dsms_sdk/dsms_sdk.md and dsms_kitem_schema.md: new KItem fields (contexts, access_properties, schema_data), Widget table, KItemCompactedModel, KItemAccessProperties, KItemSchemaData - Add docs/release-checklist.md and link it from docs/index.md - Add scripts/run_notebooks.sh for notebook test and refresh modes - Add pytest-nbmake to [tests] extras in setup.cfg - Rename 3_updation.ipynb to 3_updating.ipynb; update all 8 notebooks: fix typos, add v5 API examples (access_properties, KTypeV2, contexts, context SPARQL), make self-contained (no hardcoded UUIDs), wrap asynchronous operations (subgraph, dataframe, app run) in try/except, replace non-existent Testingmachine ktype with MeasurementDevice, use unique sdk-tutorial ktype ID in nb7 to avoid name collision, fix Specimen Number widget validation (scalar not list)
… properties - Rename INTERNALLY_PUBLIC_GROUP/EXTERNALLY_PUBLIC_GROUP to INTERNAL_GROUP/PUBLIC_GROUP - Rename config fields: id_internally_public/id_externally_public/label_internally_public/label_externally_public to id_internal/id_public/label_internal/label_public - Default group IDs changed from dsms:internally-public / dsms:externally-public to dsms:internal / dsms:public - Remove Role.ADMIN (max role is now OWNER); roles serialize as lowercase strings on the wire - Add visibility field to KItemAccessProperties (private/internal/public) - Update tests and docs to match
The backend update endpoint accepts visibility as a top-level field, separate from access_properties. Previously utils._get_kitems_diffs embedded visibility inside the access_properties dict, causing all SDK-driven visibility changes to be silently ignored by the server. Also documents the visibility field in KItemAccessProperties schema docs, fixes the role range from 1-4 to 1-3 (ADMIN removed), and adds two unit tests covering the promoted-visibility and unchanged-visibility cases.
- DSMS: add create_group, update_group, delete_group, get_group_members, add_group_member, remove_group_member with cache invalidation - utils.py: add corresponding HTTP helpers for all six operations - groups/models.py: remove GroupListBase (empty list subclass); flat now returns List[BaseGroup] directly; clean up docstrings - groups/__init__.py: remove GroupListBase from exports - properties/__init__.py: remove dead UserGroup import and export - kitem.py: update docstring to reference access_properties instead of the removed user_groups/UserGroup field - tests/test_groups.py: 14 new tests covering all CRUD methods and the GroupListBase removal
Removes BaseAccessProperty, UserAccessProperty, GroupAccessProperty and the user_access/group_access fields. Replaces them with a single grants: List[AccessGrant] where each grant carries id, type (user|group), and role. Duplicate detection now keyed on (id, type) pairs. Properties by_user/by_group/user_by_role/group_by_role replaced by by_id, by_role, and operation_by_principal. Updates both test files accordingly.
- Delete dsms/knowledge/groups/public.py (never deployed) - Remove INTERNAL_GROUP, PUBLIC_GROUP, refresh_public_groups from groups __init__ - Remove id_internal, id_public, label_internal, label_public from BaseConfiguration - Remove refresh_public_groups call from DSMS.__init__
Follow-up to 8c8ba77: test_groups.py still imported from the deleted dsms/knowledge/groups/public module. Drop the import and the 6 tests that covered those constants.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
MEMBER(read-only)CONTRIBUTOR(read + update)OWNER(read + update + delete + manage)KItemAccessPropertieswith avisibilityfield (private / internal / public) and a unifiedgrants: List[AccessGrant]where each grant carriesid,type(user | group), androleuser_access/group_accessmodelby_id,by_role,operation_by_principalRoleMappingwith operation-level permission queries (min_access_level,max_access_level)Group,User,GroupList,UserListmodels;firstName,lastName,emailonUserDSMS.user_groups/DSMS.users(cached) /DSMS.get_user()/DSMS.get_schema_data()DSMS.create_group,update_group,delete_group,get_group_members,add_group_member,remove_group_memberwith cache invalidationDSMS.get_group_subgroups,add_group_to_group,remove_group_from_group— group-in-group nesting with cache invalidationKItemSearchResult.fuzzywidened tobool | float | strto support the'id'exact-UUID-match markerschema_datasupport with diff logic in commitSparqlInterface.query_contextandgraph_contextDSMS.searchgainscontextsandattachment_extensionsparameters>=2,<3pytest-nbmaketo docs extrasBug fixes
visibilityis now sent as a top-level field in KItem update diffs (was incorrectly embedded insideaccess_properties, causing visibility changes to be silently ignored by the server)min_access_level/max_access_levelreturnRole(notint); raiseValueErrorfor unmapped operationsuser_groupsanduserscached on DSMS instance;refresh_user_groups()/refresh_users()for invalidationBreaking changes
INTERNAL_GROUP,PUBLIC_GROUP,refresh_public_groupsand the correspondingid_internal,id_public,label_internal,label_publicconfig fields — these constants were never deployed to any production instance and are superseded by thevisibilityfield onKItemAccessProperties