Python 3.11/3.12 upgrade

[DawoudSheraz]

Open edX Redwood release planned for June this year will be upgrading from Python 3.8 to Python 3.11 (for edx-platform) and Python 3.12 (for other services). That means the tutor core and its plugins need to be upgraded to support the new Python version. The Python upgrade is focused on two areas:

1. Ensuring the plugin uses 3.11/3.12 to run the underlying service.
2. Ensuring the tutor and tutor plugins code is compatible with 3.12.

The service upgrade is under progress (openedx/public-engineering#228). In the meantime, the upgrade can be done for tutor core itself. This task/issue will act as an epic for the underlying upgrade tasks.

General Guidelines

• Ensure any issue encountered in the tutor, be it due to tutor or the upstream service, is documented on this main issue. If needed, create a separate task/issue to fix it.
• Prioritize dockerfile changes for 3.11/3.12 compatibility. This is needed for the Redwood cut in late April.
• Once done, you can add appropriate CI checks for 3.12
• Start with existing test suites upgrade/compatibility i.e. ensure the test suites in the repo are passing on Python 3.12. Document any test failure and the fix done for remediate the failure
• To make sure that changes are working as expected, ensure the following where needed
  • verify that the images are built correctly
  • verify the service is up and running after tutor local launch
• The aim is to merge the Py3.12 changes directly in master (similar to Tutor claims to support Python 3.12, but apparently doesn't (if plugins are involved) #966).
• Do not hesitate to ask for help, guidance, and/or clarity where needed.

This means the set of changes would need to be compatible with Py 3.8 as well (atleast until the redwood is not released). However, Dockerfile changes would go into nightly.

Action Items

• Ensure the build is working as expected (core or plugins) with Py311/Py312 (via Dockerfile updates)
• Add Python 3.12 CI pipeline alongside 3.8. This would remain in place until redwood release
• Ensure the fixups for failures are done and the code is compatible with Py38

Tutor Codebase Upgrade Status

Depend on Upstream changes

• Tutor - Tutor Python 3.11 upgrade #1006
• tutor-discovery - tutor-discovery codebase Python 3.12 upgrade tutor-discovery#62
• tutor-ecommerce - tutor-ecommerce codebase Python 3.12 upgrade tutor-ecommerce#68
• tutor-credentials - tutor-credentials codebase Python 3.11 upgrade tutor-credentials#34
• tutor-forum - tutor-forum codebase Python 3.12 upgrade tutor-forum#31
• tutor-notes - tutor-notes codebase Python 3.12 upgrade tutor-notes#33
• tutor-xqueue - tutor-xqueue codebase Python 3.12 upgrade tutor-xqueue#29

Independent of An Upstream

• tutor-android - tutor-android codebase Python 3.12 upgrade tutor-android#19
• tutor-cairn - tutor-cairn codebase Python 3.12 upgrade tutor-cairn#31
• tutor-indigo - tutor-indigo codebase Python 3.12 upgrade tutor-indigo#72
• tutor-jupyter - tutor-jupyter codebase Python 3.12 upgrade tutor-jupyter#6
• tutor-mfe - tutor-mfe codebase Python 3.12 upgrade tutor-mfe#192
• tutor-minio - tutor-minio codebase Python 3.12 upgrade tutor-minio#36
• tutor-webui - tutor-webui codebase Python 3.12 upgrade tutor-webui#12
• cookiecutter-tutor-plugin - cookiecutter-tutor-plugin codebase Python 3.12 upgrade cookiecutter-tutor-plugin#30
• openedx-scorm-xblock - Scorm xblock Python 3.12 upgrade openedx-scorm-xblock#67

[kdmccormick]

@DawoudSheraz We discussed in Lahore that upgrading Python to 3.12 in the Dockerfiles is top priority for Redwood, whereas testing the actual plugin code with 3.12 is not important for Redwood. Could you update the ticket description with that?

[DawoudSheraz]

@DawoudSheraz We discussed in Lahore that upgrading Python to 3.12 in the Dockerfiles is top priority for Redwood, whereas testing the actual plugin code with 3.12 is not important for Redwood. Could you update the ticket description with that?

Updated. We have CI checks PRs up for most of the repos, so that is sort-of already done. For dockerfile changes, the changes have been categorized into "Dependent on Upstream" and "Independent of Upstream". For dependents, the issues/failures are being added to the related issues. We are opening PRs in upstreams where needed for 3.12 compatibilties. But for most parts, we are also monitoring 3.12 upgrades in upstream using openedx/public-engineering#234

[DawoudSheraz]

All the tasks have been completed, closing this issue.