v1.12 documentation improvement and fixups

ChangeLog.md

@@ -5,6 +5,11 @@
 
 # SW360 Base Library for Python
 
+## NEXT
+
+* Fix unintentional parallel changes in `api_get_all()` which broke the `SW360Response` feature
+  in V1.12.0.dev2. See Readme.md for details how the `SW360Response` class can now be used.
+
 ## V1.12.0
 
 * Dependency updates.
@@ -18,19 +23,22 @@
 
 ## V1.12.0.dev2
 
-* new helper functions `get_linked_id()` and `get_embedded()` to help parsing the
+* New helper functions `get_linked_id()` and `get_embedded()` to help parsing the
   `obj["_links"]` and `obj["_embedded"]` sections of HAL responses.
-* All `get_*()` and `update_*()` methods now return a `SW360Response` which is a dict
+* Most `get_*()` and `update_*()` methods now return a `SW360Response` which is a dict
   subclass. So you can still use the result as a normal dict, existing code should work
-  unchanged. The new class however also provides the convenience methods `linked_id()`,
-  `linked_ids()`, `embedded_list()`, and `embedded_lists()` to easily access the HAL
-  sections of the response.
+  unchanged. The new class provides convenience methods to easily access the HAL
+  sections of the response (`_links` and `_embedded`), see Readme.md.
 * New methods `is_above_version_18()` and `is_above_version_19()` to check which SW360
   version we are talking to.
-* Major updates of all search methods due to to changes in the SW360 REST API.
-  * New parameters to specify search criteria.
-  * New methods to support pagination (`api_get_all()`, etc.).
-  * `get_projects()` now returns `List[Dict[str, Any]]` instead of `Optional[Dict[str, Any]]`.
+* Major updates of methods returning multiple results to reflect SW360 v20 REST API changes:
+  * new parameters `page`, `page_size`, and `sort`, defaulting to `page_size=-1` to get all
+    results (as before).
+  * the `sort` parameter is now an `Optional[SortParam]` (see new `sw360.sorting` module)
+    instead of a plain string.
+  * new internal methods to support pagination (`api_get_all()`, etc.).
+  * (temporary change, rolled back in dev3: `get_projects()` now returns `List[Dict[str, Any]]`
+     instead of `Optional[Dict[str, Any]]`.)
 * New methods for report generation: `generate_project_license_info()`,
   `generate_project_clearing_report()`, `generate_project_source_code_bundle()`.
 

Readme.md

@@ -33,20 +33,50 @@ Install sw360 and required dependencies:
 
 ### Using the API
 
-* Get a REST API token from your SW360 server
-* Export required environment variables (optionally but recommended):
+* Get a REST API token from your SW360 server. In default configuration, tokens can be
+  generated in your Preferences.
+* Starting with SW360 v20, tokens can also be generated using a Keycloak client_id and
+  client_secret which you can request from your SW360 admin team. There's a special helper
+  class to generate the access tokens:
 
-  ```shell
-  export SW360ProductionToken=<your_api_token>
+  ```python
+  from sw360 import SW360Keycloak
+  kc = SW360Keycloak(sw360_url)
+  sw360_api_token = kc.get_keycloak_token(client_id, client_secret, write_access=False)
   ```
 
 * Start using the API:
 
   ```python
   import sw360
-  client = sw360.SW360(sw360_url, sw360_api_token)
+  # oauth2=True is required for tokens created using client_id and client_secret.
+  client = sw360.SW360(sw360_url, sw360_api_token, oauth2=True)
+  client.login_api():
+  r = client.get_release(release_id)
+  ```
+
+* Starting with v1.12, most responses are wrapped in `SW360Response`, a `dict` subclass,
+  so existing code keeps working unchanged, but you get new convenience methods for the
+  `_links` and `_embedded` sections of the SW360 HAL responses:
+
+  ```python
+  release = client.get_release(release_id)
+
+  # get the component id from the release, instead of
+  #   client.get_id_from_href(release["_links"]["sw360:component"]["href"])
+  component_id = release.linked_id("component")
+  component = client.get_component(component_id)
+
+  # get the first attachment for the release, instead of
+  #   client.get_id_from_href(release["_embedded"]["sw360:attachments"][0]["_links"]["self"]["href"])
+  attachment_id = release.embedded_list("attachments")[0].linked_id()
+  attachment = client.get_attachment(attachment_id)
   ```
 
+  For now, the `get_all_*` and `get_*_by_*` methods (e.g. `get_all_components` or
+  `get_releases_by_external_id`) still return a list of dicts, this is planned to
+  be changed in a future release to also return a list of `SW360Response` objects.
+
 ### Contribute
 
 * All contributions in form of bug reports, feature requests or merge requests!

sw360/base.py

@@ -164,7 +164,7 @@ def api_get(self, url: str = "") -> Optional[SW360Response]:
 
     def api_get_all(self, url: str, sort: Optional[SortParam] = None,
                     batch: int = -1, page: int = 0,
-                    _data: Optional[Dict[str, Any]] = None) -> Optional[Dict[str, Any]]:
+                    _data: Optional[SW360Response] = None) -> Optional[SW360Response]:
         """
         Retrieve all pages of data from the specified URL.
 
@@ -175,40 +175,42 @@ def api_get_all(self, url: str, sort: Optional[SortParam] = None,
                       `default_batch_size` if -1
         :param _data: Internal param for aggregating data.
         :return: The combined JSON data from all pages.
-        :rtype: Optional[Dict[str, Any]]
+        :rtype: Optional[SW360Response]
         """
         if _data is None:
-            _data = {}
+            _data = SW360Response()
         if batch == -1:
             batch = self.default_batch_size
 
         paginated_url = self._add_pagination(url, page, batch, sort)
         resp = self.api_get(paginated_url)
         if resp is not None and 'page' in resp:
             total_pages = resp['page']['totalPages']
-            # Clean up meta info
+            # Clean up meta info. It only has "curies", "first"/"next"/"last",
+            # becoming meaningless once we merge multiple pages.
             if '_links' in resp:
                 del resp['_links']
             del resp['page']
-            # Update data and get next page
-            _data = self.__merge_responses(_data, resp)
+            # Merge in place into _data and get next page.
+            self.__merge_responses(_data, resp)
             if page + 1 < total_pages:
                 return self.api_get_all(url, sort, batch, page + 1, _data)
         else:
-            # Clean up meta info
+            # Clean up meta info, see comment above.
             if resp is not None and '_links' in resp:
                 del resp['_links']
-            _data = self.__merge_responses(_data, resp)
+            # Merge in place into _data.
+            self.__merge_responses(_data, resp)
         return _data
 
     def __merge_responses(self, previous: Dict[str, Any],
                           next: Optional[Dict[str, Any]]) -> Dict[str, Any]:
         """
         Function to merge previous and next response of the same API
-        recursively.
+        into ``previous``, which is also returned.
         :param previous: The previous response data.
         :param next: The next response data.
-        :return: The merged response data.
+        :return: The merged response data (same dict as ``previous``).
         """
         if next is None:
             return previous

sw360/sw360_api.py

@@ -84,7 +84,6 @@ def __init__(
         session: Optional[requests.Session] = session_default,
         default_batch_size: int = 50
     ) -> None:
-        super().__init__(url, token, oauth2)
         """Constructor"""
         super().__init__(url, token, oauth2, default_batch_size)
         if url[-1] != "/":

tests/test_sw360_projects.py

@@ -19,7 +19,7 @@
 
 sys.path.insert(1, "..")
 
-from sw360 import SW360, SW360Error  # noqa: E402
+from sw360 import SW360, SW360Error, SW360Response  # noqa: E402
 from sw360.sorting import ProjectSortColumn  # noqa: E402
 
 
@@ -206,6 +206,7 @@ def test_get_projects(self) -> None:
 
         projects = lib.get_projects()
         self.assertIsNotNone(projects)
+        self.assertIsInstance(projects, SW360Response)
         if projects:  # only for mypy
             self.assertTrue("_embedded" in projects)
             self.assertTrue("sw360:projects" in projects["_embedded"])
@@ -227,6 +228,7 @@ def test_get_projects_v18_style(self) -> None:
 
         projects = lib.get_projects()
         self.assertIsNotNone(projects)
+        self.assertIsInstance(projects, SW360Response)
         if projects:  # only for mypy
             self.assertTrue("_embedded" in projects)
             self.assertTrue("sw360:projects" in projects["_embedded"])