[FC-0118] docs: add OEP-69 REST API Conventions - #805
[FC-0118] docs: add OEP-69 REST API Conventions#805Abdul-Muqadim-Arbisoft wants to merge 3 commits into
Conversation
Consolidate the FC-0118 REST API standardization ADRs (openedx-platform docs/decisions/0025-0037, plus the 0017 authentication pointer ADR) into a single Best Practice OEP, as requested during community review of the ADRs. The OEP captures 13 conventions covering DRF serializers, ViewSets, endpoint consolidation, JWT authentication, permission classes, idempotent GETs, pagination, filtering/sorting, response shaping, error responses, API documentation, versioning, and the canonical MFE configuration endpoint. Refs: openedx/openedx-platform#38137
42c5d54 to
ce038a9
Compare
bradenmacdonald
left a comment
There was a problem hiding this comment.
Thanks, this is great! I left some comments inline.
One major omission (?) though seems to be URLs.
Our REST API URLs today follow no consistent patterns whatsoever:
/transcripts/upload
/xblock/:id/handler/:handlerName/
/xblock/container/:id/
/organizations/
/api/toggles/
/event/
/api/user/v1/accounts
/api/v1/course_runs/
/course_team/:course_id/
/course_rerun/:course_id/
/videos/:course_id/:video_id/
/api/learning_sequences/...
I think this OEP should clearly define URL expectations like:
- Whether we should use singular nouns (
/user/:x) or plural nouns (/users/:x) - That REST API URLs should always start with
/api/ - That new CMS and LMS URLs should not conflict with each other so they can potentially be merged without needing major changes to API clients other than domain name.
- Whether APIs are organized hierarchically using logical components
/api/course/:course_key/teams/:team_id/or the URLs should strictly map to the underlying organization of the codebase/api/teams/v1/course/:course_id/:team_idor some other organizational scheme ? - etc.
| The reference implementation is the ongoing FC-0118 work in ``openedx-platform``, | ||
| tracked under the umbrella issue `#38137 | ||
| <https://github.com/openedx/openedx-platform/issues/38137>`_ and recorded as ADRs | ||
| ``docs/decisions/0025`` through ``0037`` (plus the pointer ADR | ||
| ``openedx/core/djangoapps/oauth_dispatch/docs/decisions/0017``). Priority | ||
| migration targets identified by the ADRs include the Enrollment API ViewSet | ||
| consolidation, the ``drf-spectacular`` rollout, the ``BearerAuthentication`` | ||
| deprecation via the ``view_auth_classes`` decorator, and the canonical | ||
| front-end configuration endpoint. This OEP may move to "Final" once the conventions | ||
| have representative, merged implementations across the highest-impact endpoints. |
There was a problem hiding this comment.
Shouldn't the reference implementation be in a single library, and outside of openedx-platform?
e.g. openedx_rest_lib could provide DefaultPagination, permission_classes, django-filter, drf-spectacular, Error-response standardization, CI tooling, and any of the other requirements specified in this OEP. So then following this OEP would be mostly about strictly using openedx_rest_lib.
I would personally make openedx_rest_lib re-export all of the DRF API as well, so that consumers can enjoy a much cleaner import story:
Today 🤢
from drf_spectacular.utils import (
OpenApiParameter,
OpenApiRequest,
OpenApiResponse,
extend_schema,
)
from edx_rest_framework_extensions.auth.jwt.authentication import JwtAuthentication
from edx_rest_framework_extensions.paginators import DefaultPagination
from rest_framework import permissions, status, viewsets
from rest_framework.decorators import action
from rest_framework.exceptions import NotFound, ValidationError
from rest_framework.generics import ListAPIView
from rest_framework.response import Response
from rest_framework.views import APIView
from openedx.core.lib.api.mixins import StandardizedErrorMixin
from openedx.core.lib.api.permissions import ApiKeyHeaderPermissionIsAuthenticatedTomorrow? 🌈
from openedx_rest_lib import (
OpenApiParameter,
OpenApiRequest,
OpenApiResponse,
extend_schema,
StandardViewSet, # Has DefaultPagination, JwtAuthentication, StandardizedErrorMixin, etc.
ListAPIView, # Has DefaultPagination, JwtAuthentication, StandardizedErrorMixin, etc.
APIView, # Has DefaultPagination, JwtAuthentication, StandardizedErrorMixin, etc.
rest_exceptions, # DRF NotFound, ValidationError, etc. (standardized)
permissions,
Response,
)There was a problem hiding this comment.
Great idea. The pieces already exist but are split: DefaultPagination/JwtAuthentication are in edx-drf-extensions (a library, reusable), but StandardizedErrorMixin + the ADR-0029 handler live in openedx.core.lib.api inside the platform, and each pilot app defines its own permission classes (e.g. HasCourseAuthorAccess in the xblock PR). So plugins and other IDAs can't reuse the standardized bits today, exactly the problem a shared library solves. edx-drf-extensions looks like the natural home since it already ships pagination + JWT.
One caveat from the current implementation: the ADR-0029 handler currently imports ignored_error_exception_handler from openedx.core.lib.request_utils, so it's coupled to platform internals, extracting it means breaking that coupling first, not a straight move. On the re-export idea: my only caution is keep-in-sync cost, so I'd lean toward a curated set + a StandardViewSet/StandardAPIView base (pre-wiring pagination, JWT, and the error mixin) rather than a 1:1 re-export of all of DRF.
Since this is the same shape as the URL point, a design decision best captured as its own ADR + implementation rather than folded into this OEP, and FC-0118 is almost at close: @feanil , could we get your consensus on whether to add another ADR (extract a reusable REST library into edx-drf-extensions) under FC-0118 and start migrating the pilot endpoints onto it now, or track it as a documented follow-up? We'll then update OEP-69's Reference Implementation section to point at the library as the primary deliverable.
There was a problem hiding this comment.
I agree with Braden here, that a lot of the re-usable bits should live in edx-drf-extensions if possible. I'm not as sure about re-exporting the DRF Base classes, I think that can lead to more complexity when reading if you are new to the codebase.
There was a problem hiding this comment.
Hi @feanil @bradenmacdonald
The artifacts we're planning to move to edx-drf-extensions are the following:
- StandardizedErrorMixin | mixins.py
- Envelope formatters + Conflict | errors.py
- project() + MinimalViewMixin | shaping.py
- IterablePaginationMixin + paginate_manually() | paginators.py (existing)
- Opaque-key lookup regex constants | routers.py
- assert_error_envelope() | testing.py
- Error-type URI catalog + extension helper | errors.py
- ErrorResponseSerializer (plain DRF) | errors.py
- Exception-handler dependency inversion | errors.py + settings
These were chosen based on the feasibility of moving them without introducing much coupling into already-implemented pieces. Let mw know if you think we can move any other then these.
This is a great point. The OEP already fixes URL versioning (Convention 12 mandates /api//vN/), but you're right that it says nothing about URL naming and organization and none of the source ADRs (0025–0037) did either. So this is worth pinning down. It would cover the items you listed:
@feanil could we get your consensus on whether it makes sense to add one more ADR (URL conventions) under FC-0118 and implement it against the selected endpoints now, given we're near close, or capture it as a documented follow-up to pick up separately? |
feanil
left a comment
There was a problem hiding this comment.
All of the sections should link to their respective ADRs directly.
I also think it might be worth it to have code examples for these here, but could be convinced that having them in the linked ADRs is sufficient. Though in that case, we should explicitly link to the code examples sections.
| The reference implementation is the ongoing FC-0118 work in ``openedx-platform``, | ||
| tracked under the umbrella issue `#38137 | ||
| <https://github.com/openedx/openedx-platform/issues/38137>`_ and recorded as ADRs | ||
| ``docs/decisions/0025`` through ``0037`` (plus the pointer ADR | ||
| ``openedx/core/djangoapps/oauth_dispatch/docs/decisions/0017``). Priority | ||
| migration targets identified by the ADRs include the Enrollment API ViewSet | ||
| consolidation, the ``drf-spectacular`` rollout, the ``BearerAuthentication`` | ||
| deprecation via the ``view_auth_classes`` decorator, and the canonical | ||
| front-end configuration endpoint. This OEP may move to "Final" once the conventions | ||
| have representative, merged implementations across the highest-impact endpoints. |
There was a problem hiding this comment.
I agree with Braden here, that a lot of the re-usable bits should live in edx-drf-extensions if possible. I'm not as sure about re-exporting the DRF Base classes, I think that can lead to more complexity when reading if you are new to the codebase.
We already have the respective code examples in the ADRs so i think linking them over here would suffice. |
Hi @bradenmacdonald @feanil ,so the URL conventions sahred by Branden now have a dedicated ADR, docs/decisions/0038-standardize-rest-api-url-structure.rst (PR #39003), and I've consolidated it into OEP-69 as well named as Convention 14: Standardize REST API URL structure, Moreover Convention 14's Source link will give 404 until PR #39003 merges. |
…nvention Address review feedback on PR openedx#805: - Link every convention to its source ADR and that ADR's code-example section; link the docs/decisions folder from the Abstract and Reference Implementation. - Add Convention 14 (REST API URL structure) from ADR 0038. - Rework Convention 3: reframe as consolidating RPC-style action endpoints, clarify it is not about HTTP verbs, add when-to-apply guidance and an illustrative example. - Fix Convention 1 to require @extend_schema(request=..., responses=...) when request and response serializers differ. - Remove the redundant inline error-response example in favour of the linked ADR. Refs: openedx/openedx-platform#39003
feanil
left a comment
There was a problem hiding this comment.
A couple of nits but otherwise this looks good to me. I'd like ADR 39 to land first but then I think this is good to land as well.
| @@ -0,0 +1,691 @@ | |||
| .. _OEP-69 Open edX REST API Conventions: | |||
|
|
|||
| OEP-69: Open edX REST API Conventions | |||
There was a problem hiding this comment.
| OEP-69: Open edX REST API Conventions | |
| OEP-69: REST API Conventions |
It's all Open edX in this repo, I don't think we need that in the title.
| :widths: 25 75 | ||
|
|
||
| * - OEP | ||
| - :ref:`OEP-69 <OEP-69 Open edX REST API Conventions>` |
There was a problem hiding this comment.
| - :ref:`OEP-69 <OEP-69 Open edX REST API Conventions>` | |
| - :ref:`OEP-69 <OEP-69: REST API Conventions>` |
| * - OEP | ||
| - :ref:`OEP-69 <OEP-69 Open edX REST API Conventions>` | ||
| * - Title | ||
| - Open edX REST API Conventions |
There was a problem hiding this comment.
| - Open edX REST API Conventions | |
| - REST API Conventions |
Drop "Open edX" from the title (heading, Title row, anchor, and self-ref) per review; the repo is entirely Open edX so the qualifier is redundant.
Consolidate the FC-0118 REST API standardization ADRs (openedx-platform docs/decisions/0025-0037, plus the 0017 authentication pointer ADR) into a single Best Practice OEP, as requested during community review of the ADRs.
The OEP captures 13 conventions covering DRF serializers, ViewSets, endpoint consolidation, JWT authentication, permission classes, idempotent GETs, pagination, filtering/sorting, response shaping, error responses, API documentation, versioning, and the canonical MFE configuration endpoint.
Refs: openedx/openedx-platform#38137
Discussion: https://discuss.openedx.org/t/request-for-community-review-on-adrs-for-standardizing-the-open-edx-platform-api-endpoints/18717