chore: Bump to Python3.10 #24112

EugeneTorap · 2023-05-18T08:10:55Z

SUMMARY

Running CI integration and unit tests for Py3.10 and Py3.11.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

Has associated issue:
Required feature flags:
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided
Introduces new feature or API
Removes existing feature or API

setup.py

# Conflicts: # .github/workflows/docker_build_push.sh # Dockerfile

codecov · 2023-07-15T16:09:13Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.72%. Comparing base (c0f8dfc) to head (0651b6b).
Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #24112      +/-   ##
==========================================
+ Coverage   69.71%   69.72%   +0.01%     
==========================================
  Files        1920     1912       -8     
  Lines       75234    74920     -314     
  Branches     8423     8313     -110     
==========================================
- Hits        52447    52239     -208     
+ Misses      20725    20636      -89     
+ Partials     2062     2045      -17

Flag	Coverage Δ
mysql	`77.88% <ø> (-0.01%)`	⬇️
postgres	`77.99% <ø> (-0.01%)`	⬇️
python	`82.91% <ø> (-0.01%)`	⬇️
sqlite	`77.43% <ø> (-0.01%)`	⬇️
unit	`56.78% <ø> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

john-bodley · 2023-07-15T23:05:52Z

@betodealmeida, @eschutho, @michael-s-molina, @villebro et al. do we have a policy in terms of what versions of Python we support (I wasn’t able to find anything in the repo via a quick code search)? If not we probably should formalize this, i.e.,

The official supported versions (excluding main)
The non-prerelease official supported versions
The n latest versions
…

betodealmeida · 2023-07-19T17:13:07Z

@betodealmeida, @eschutho, @michael-s-molina, @villebro et al. do we have a policy in terms of what versions of Python we support (I wasn’t able to find anything in the repo via a quick code search)? If not we probably should formalize this, i.e.,
* The [official supported versions](https://devguide.python.org/versions/#supported-versions) (excluding `main`)

* The non-prerelease official supported versions

* The _n_ latest versions

* …

Agree, @john-bodley, we should have this formalized. My suggestions would be every major version of Superset should support the latest 3 major versions of Python available at the time (so 3.9-3.11 today), and probably when 4.0 is out we'll support 3.10-3.12.

michael-s-molina · 2023-07-19T17:14:54Z

My suggestions would be every major version of Superset should support the latest 3 major versions of Python available at the time (so 3.9-3.11 today), and probably when 4.0 is out we'll support 3.10-3.12.

This is a great topic for the next Town Hall @rusackas

mdeshmu · 2023-10-07T02:04:51Z

@michael-s-molina Is there any decision made on officially supported Python versions? It would be great if this is also officially documented.

If there is no decision made yet then I would like to resubmit this PR. Please let me know if any concerns.

# Conflicts: # .github/workflows/docker_build_push.sh # requirements/development.txt # requirements/testing.txt

EugeneTorap · 2023-10-08T09:23:48Z

Hi @betodealmeida @john-bodley @michael-s-molina @mdeshmu We have a problem with Py3.11 and prophet==1.1.1 but we can't update the prophet lib because that nasty bug hasn't been fixed yet.
@mdeshmu Can you ask the developers of this prophet library to fix this bug?

mdeshmu · 2023-10-08T11:49:06Z

@EugeneTorap I will follow up on the bug.

Someone said here that combination of prophet 1.1.4 and holidays 0.32 is working. Can you check that?

Also, there is another potential Redis issue with Python 3.11.

mdeshmu · 2023-10-25T12:00:13Z

@EugeneTorap prophet 1.1.5 is released and someone said here it works for them: facebook/prophet#2354 (comment)

EugeneTorap · 2023-10-27T18:21:00Z

Thanks @mdeshmu for this information! Can you create a small PR to update the prophet to version 1.1.5?

mistercrunch · 2024-03-22T16:11:31Z

Nice! Oh one more thing, can you add an entry on ./UPDATING.md about Python 3.10 being the new minimum python version to run Superset and referencing this PR.

Otherwise LGTM, but for something that important we could use more approvals.

EugeneTorap · 2024-03-22T17:05:37Z

@john-bodley @villebro @dpgaspar @rusackas can you also approve & merge this PR?

villebro

Nice, LGTM!

villebro · 2024-03-22T17:36:10Z

setup.py

@@ -208,13 +208,12 @@ def get_git_sha() -> str:
        "starrocks": ["starrocks>=1.0.0"],
        "doris": ["pydoris>=1.0.0, <2.0.0"],
    },
-    python_requires="~=3.9",
+    python_requires="~=3.9",  # TODO: change it to "~=3.10"


I missed this just now, can we make it

Suggested change

python_requires="~=3.9", # TODO: change it to "~=3.10"

python_requires="~=3.10",

michael-s-molina

This is breaking change for folks using Python 3.9. It should become a proposal for the next breaking window.

villebro · 2024-03-22T18:31:12Z

This is breaking change for folks using Python 3.9. It should become a proposal for the next breaking window.

@michael-s-molina thanks for reviewing, I suspected this would be the case.. In that case, I propose the following:

We change this PR to continue support for 3.9
Open a new card on the 5.0 board that removes 3.9 when the window opens again, assigning it to @EugeneTorap

Thoughts?

michael-s-molina · 2024-03-22T18:34:55Z

We change this PR to continue support for 3.9

That would work.

Open a new card on the 5.0 board that removes 3.9 when the window opens again, assigning it to @EugeneTorap
Thoughts?

Added here.

mistercrunch · 2024-03-22T19:08:55Z

Ok, so main / published docker builds start pointing to3.10, and docker-compose / helm chart start pointing to 3.10 as a result,... But we still:

officially support 3.9 in setup.py
run all or some of the tests on both 3.9 and 3.10?

I noticed we have a lot of "matrix" in GHA that are 3.9 only, would we want to have some or all of those do both? I wish we could DRY those up and have more consistency there.... The fact that this PR touches ~30 files to change the main python version is a bit insane (and due to GHA not having good variable/templating options for this type of stuff)

This reverts commit 1706d1d.

setup.py

.github/actions/setup-backend/action.yml

scripts/build_docker.py

This reverts commit 4538a7b.

setup.py

mistercrunch

I made the suggested changes. Surprising how editing on-the-web is pretty viable now with GitHub.

@michael-s-molina I'll need an approval from you to merge since you're still in "Request changes" from a previous review

mistercrunch · 2024-04-02T16:52:08Z

UPDATE YOUR VENVs EVERYONE! :)

Or just use docker-compose for your dev env

Co-authored-by: Maxime Beauchemin <[email protected]>

chore: Bump to Python3.10

63b7edc

EugeneTorap requested review from villebro, geido, eschutho, rusackas, betodealmeida, nytai, mistercrunch, craig-rueda, john-bodley and kgabryje as code owners May 18, 2023 08:10

pull-request-size bot added the size/M label May 18, 2023

EugeneTorap added 2 commits May 18, 2023 11:23

Remove 'sasl' lib

a6803a1

Add 'sasl' lib in hive CI

11db1b4

villebro reviewed May 18, 2023

View reviewed changes

setup.py Outdated Show resolved Hide resolved

EugeneTorap added 2 commits July 15, 2023 18:43

Merge branch 'master' into chore/bump-to-python3.10

a961133

# Conflicts: # .github/workflows/docker_build_push.sh # Dockerfile

Update python version

affc394

EugeneTorap added 2 commits July 25, 2023 19:55

Merge branch 'master' into chore/bump-to-python3.10

0b371d0

Merge branch 'master' into chore/bump-to-python3.10

a5abd13

EugeneTorap added 2 commits October 8, 2023 11:46

Merge branch 'master' into chore/bump-to-python3.10

c88087b

# Conflicts: # .github/workflows/docker_build_push.sh # requirements/development.txt # requirements/testing.txt

Use python 3.10.13 in Dockerfile

8a432f8

Fix PY 3.11 docker image

275f129

Add an entry on ./UPDATING.md

73f8a65

villebro approved these changes Mar 22, 2024

View reviewed changes

villebro reviewed Mar 22, 2024

View reviewed changes

michael-s-molina requested changes Mar 22, 2024

View reviewed changes

michael-s-molina added the risk:breaking-change Issues or PRs that will introduce breaking changes label Mar 22, 2024

EugeneTorap added 2 commits March 29, 2024 13:01

Merge branch 'master' into chore/bump-to-python3.10

5c4f697

Revert "Revert 'python_requires' to "~=3.9""

45e2411

This reverts commit 1706d1d.

mistercrunch requested changes Mar 29, 2024

View reviewed changes

setup.py Outdated Show resolved Hide resolved

.github/actions/setup-backend/action.yml Outdated Show resolved Hide resolved

scripts/build_docker.py Show resolved Hide resolved

Revert "Update action.yml"

0651b6b

This reverts commit 4538a7b.

michael-s-molina reviewed Apr 1, 2024

View reviewed changes

setup.py Outdated Show resolved Hide resolved

mistercrunch mentioned this pull request Apr 1, 2024

chore: bump python version to 3.11 in Dockerfile #23804

Closed

9 tasks

mistercrunch added 2 commits April 2, 2024 08:36

Add python 3.9 support back in setup.py

13d5df4

Merge branch 'master' into chore/bump-to-python3.10

e7a6e9b

mistercrunch approved these changes Apr 2, 2024

View reviewed changes

michael-s-molina removed the risk:breaking-change Issues or PRs that will introduce breaking changes label Apr 2, 2024

michael-s-molina approved these changes Apr 2, 2024

View reviewed changes

mistercrunch merged commit 3a34c7f into apache:master Apr 2, 2024
31 checks passed

EugeneTorap deleted the chore/bump-to-python3.10 branch April 2, 2024 16:57

mistercrunch mentioned this pull request Apr 2, 2024

fix: pkg-config dependency in Dockerfile #27855

Merged

jzhao62 pushed a commit to jzhao62/superset that referenced this pull request Apr 4, 2024

chore: Bump to Python3.10 (apache#24112)

aeecb50

Co-authored-by: Maxime Beauchemin <[email protected]>

EandrewJones pushed a commit to UMD-ARLIS/superset that referenced this pull request Apr 5, 2024

chore: Bump to Python3.10 (apache#24112)

061cfd9

Co-authored-by: Maxime Beauchemin <[email protected]>

EnxDev pushed a commit to EnxDev/superset that referenced this pull request Apr 12, 2024

chore: Bump to Python3.10 (apache#24112)

b24813b

Co-authored-by: Maxime Beauchemin <[email protected]>

qleroy pushed a commit to qleroy/superset that referenced this pull request Apr 28, 2024

chore: Bump to Python3.10 (apache#24112)

a940615

Co-authored-by: Maxime Beauchemin <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: Bump to Python3.10 #24112

chore: Bump to Python3.10 #24112

EugeneTorap commented May 18, 2023 •

edited by michael-s-molina

Loading

codecov bot commented Jul 15, 2023 •

edited

Loading

john-bodley commented Jul 15, 2023 •

edited

Loading

betodealmeida commented Jul 19, 2023

michael-s-molina commented Jul 19, 2023

mdeshmu commented Oct 7, 2023 •

edited

Loading

EugeneTorap commented Oct 8, 2023 •

edited

Loading

mdeshmu commented Oct 8, 2023

mdeshmu commented Oct 25, 2023

EugeneTorap commented Oct 27, 2023

mistercrunch commented Mar 22, 2024

EugeneTorap commented Mar 22, 2024

villebro left a comment

villebro Mar 22, 2024

michael-s-molina left a comment

villebro commented Mar 22, 2024

michael-s-molina commented Mar 22, 2024

mistercrunch commented Mar 22, 2024 •

edited

Loading

mistercrunch left a comment •

edited

Loading

mistercrunch commented Apr 2, 2024

	python_requires="~=3.9", # TODO: change it to "~=3.10"
	python_requires="~=3.10",

chore: Bump to Python3.10 #24112

chore: Bump to Python3.10 #24112

Conversation

EugeneTorap commented May 18, 2023 • edited by michael-s-molina Loading

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

codecov bot commented Jul 15, 2023 • edited Loading

Codecov Report

john-bodley commented Jul 15, 2023 • edited Loading

betodealmeida commented Jul 19, 2023

michael-s-molina commented Jul 19, 2023

mdeshmu commented Oct 7, 2023 • edited Loading

EugeneTorap commented Oct 8, 2023 • edited Loading

mdeshmu commented Oct 8, 2023

mdeshmu commented Oct 25, 2023

EugeneTorap commented Oct 27, 2023

mistercrunch commented Mar 22, 2024

EugeneTorap commented Mar 22, 2024

villebro left a comment

Choose a reason for hiding this comment

villebro Mar 22, 2024

Choose a reason for hiding this comment

michael-s-molina left a comment

Choose a reason for hiding this comment

villebro commented Mar 22, 2024

michael-s-molina commented Mar 22, 2024

mistercrunch commented Mar 22, 2024 • edited Loading

mistercrunch left a comment • edited Loading

Choose a reason for hiding this comment

mistercrunch commented Apr 2, 2024

EugeneTorap commented May 18, 2023 •

edited by michael-s-molina

Loading

codecov bot commented Jul 15, 2023 •

edited

Loading

john-bodley commented Jul 15, 2023 •

edited

Loading

mdeshmu commented Oct 7, 2023 •

edited

Loading

EugeneTorap commented Oct 8, 2023 •

edited

Loading

mistercrunch commented Mar 22, 2024 •

edited

Loading

mistercrunch left a comment •

edited

Loading