Skip to content

feat(setup): Update pythong version to 3.12 for RDM #545

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sakshamarora1 wants to merge 1 commit into
base: master
Choose a base branch
from
Open

feat(setup): Update pythong version to 3.12 for RDM #545

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
runs-on: ubuntu-24.04
strategy:
matrix:
python-version: [3.9]
python-version: [3.12, 3.14]
requirements-level: [pypi]
db-service: [postgresql14]
include:
Expand Down
13 changes: 11 additions & 2 deletions cds_migrator_kit/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,18 @@

"""Migration tool kit from old Invenio to new Invenio."""


from .ext import CdsMigratorKit

__all__ = ("__version__", "CdsMigratorKit")

__version__ = "0.1.0.dev20180000"
# Fix for Python3.10 breaking changes
# The underlying module `invenio-query-parser` uses MutableMapping and Sequence from collections
# But they are moved to collections.abc starting from Python3.10

import collections
import collections.abc

collections.MutableMapping = collections.abc.MutableMapping
collections.Sequence = collections.abc.Sequence
Comment on lines +19 to +23

@kpsherva kpsherva Jul 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will this work in production image? for some time we will need backwards compatibility with 3.9 until we update invenio image to run newer versions

@sakshamarora1 sakshamarora1 Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No it is a breaking change, in python3.10
for backwards compatibility, i can add a try catch

@kpsherva kpsherva Jul 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please make it work in both 3.9 and higher version because we are not yet ready to switch the prod image


__version__ = "0.1.0.dev20260000"
1 change: 1 addition & 0 deletions setup.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ install_requires =
sentry-sdk>=1.45,<2.0.0
cds-dojson @ git+https://github.com/CERNDocumentServer/cds-dojson@master#egg=cds-dojson
invenio-rdm-migrator>=5.0.0
setuptools<82.0.0 # `invenio-rdm-migrator` has a dependency on `pypeln` which depends on `pkg_resources`

@kpsherva kpsherva Jul 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about the invenio-query-parser dependency we discussed?

@kpsherva kpsherva Jul 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is pypeln needed in invenio-migrator? can we eliminate it? otherwise we are not upgrading for performance if we depend on older setuptools and calling pkg_resources somewhere in the code

@sakshamarora1 sakshamarora1 Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Invenio-query-parser problem is described above in the setup.cfg file

As for pypeln yes it's being used: https://github.com/search?q=repo%3Ainveniosoftware%2Finvenio-rdm-migrator%20pypeln&type=code
It could be possible to remove it from invenio-rdm-migrator, should I check with Alex?

@kpsherva kpsherva Jul 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it used for anything? If not then we can remove without involving Alex

@sakshamarora1 sakshamarora1 Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

It is used for multi processing, in the transform run step and either we need to find an alternative for it or pin setuptools

@kpsherva kpsherva Jul 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you finalise the upgrade without pinning setuptools?

lxml>=4.6.5
ipython!=8.1.0
uwsgi>=2.0
Expand Down
1 change: 1 addition & 0 deletions tests/cds-rdm/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
# the terms of the MIT License; see LICENSE file for more details.

"""Pytest fixtures."""

import os
from collections import namedtuple
from os.path import dirname, join
Expand Down
1 change: 1 addition & 0 deletions tests/cds-rdm/test_bulletin_issue.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
# the terms of the MIT License; see LICENSE file for more details.

"""Tests suites."""

import json
from pathlib import Path

Expand Down
1 change: 1 addition & 0 deletions tests/cds-rdm/test_hr_migration.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
# the terms of the MIT License; see LICENSE file for more details.

"""Tests suites."""

import json
from pathlib import Path

Expand Down
1 change: 1 addition & 0 deletions tests/cds-rdm/test_thesis_migration.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
# the terms of the MIT License; see LICENSE file for more details.

"""Tests suites."""

import json
from pathlib import Path

Expand Down
Loading