-
Notifications
You must be signed in to change notification settings - Fork 13.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Flask App factory PR #1 #8418
Flask App factory PR #1 #8418
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @craig-rueda for tackling this. I love the idea of detangling aspects of the Superset app which historically has been problematic when extending due to potential cyclical dependencies.
logger = logging.getLogger(__name__) | ||
|
||
|
||
def create_app(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I generally like this approach as we (Airbnb) use it elsewhere. Any reason for having the SupersetAppInitializer
class rather than the pattern suggested in the Flask cookie-cutter template.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea, the thinking here is that an end user can simply extend the SupersetAppInitializer
and override only the parts they're interested in.
Although I could just do all the work in app:create_app()
, it's a bit more OO to break the task of initialization into a set of phases.
I think a good example of this need would be someone that wanted to override something specific, such as the configuration of FAB, but wanted to carry forward the rest of Superset's config. SupersetAppInitializer
allows you to compose your config from reusable bits as an end user.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wondering if you tried to override SupersetAppInitializer
in a local config, but it screams circular dependency, though maybe with all the proxying here it's not an issue anymore (!?)
module_datasource_map.update(self.config.get("ADDITIONAL_MODULE_DS_MAP")) | ||
ConnectorRegistry.register_sources(module_datasource_map) | ||
|
||
def configure_cache(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Like the cookie-cutter example one could have a method which registers all extensions.
superset/__init__.py
Outdated
module_datasource_map = app.config["DEFAULT_MODULE_DS_MAP"] | ||
module_datasource_map.update(app.config["ADDITIONAL_MODULE_DS_MAP"]) | ||
ConnectorRegistry.register_sources(module_datasource_map) | ||
app = create_app() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we still need this file? Having logic in __init__.py
can be problematic for cyclical dependencies.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We do and we don't. My goal is to clear it out completely... However, the changes required in order to facilitate this will end up being quite a large PR. If you think it's reasonable to "just do it", then I'd be happy to pull that thread.
One thing of note here is that removing app
from here will REQUIRE people to register their FLASK_APP
env var differently superset.app
-> superset.app:create_app()
|
||
|
||
try: | ||
# Allow user to override our config completely |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I’m not sure we want to allow them to completely override the default config as there’s implicit requirements in the code that certain config variables exist.
superset/app.py
Outdated
csrf.exempt(ex) | ||
|
||
def register_blueprints(self): | ||
for bp in self.config.get("BLUEPRINTS"): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mypy
is going to barf at using config.get(...)
as the return type is Optional[...]
. In reference to my previous comment about the default config, config[...]
should be suffice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea, I haven't linted yet - just getting ducks in order in terms of structure
superset/app.py
Outdated
""" | ||
pass | ||
|
||
def init_views(self) -> None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we simply make all our views blueprints and register them accordingly? This may simplify things in the future and/or reduce the cyclical dependency issues we often run into. We (Airbnb) have found the model of using blueprints for everything has worked quite successfully and the Flask app really just glues everything together in a somewhat lightweight fashion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, definitely. I want to pull up all occurrences of things like appbuilder.add_view()
to this method. Ideally, the views themselves wouldn't register themselves, but rather this init logic would compose the views we're interested in.
Again, this PR was intended to be the first step in chipping away at this pattern migration. Thoughts on going for it?? @mistercrunch ??
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm pretty sure that FAB's add_view
creates a blueprint for each call
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yap, it's ModelView
, ModelRestApi
class is a blueprint and it's actually created on add_view
Read through, LGTM. This is many steps in the right direction. |
# Conflicts: # superset/__init__.py # superset/cli.py # superset/utils/core.py
setup.py
Outdated
@@ -110,6 +110,7 @@ def get_git_sha(): | |||
extras_require={ | |||
"bigquery": ["pybigquery>=0.4.10", "pandas_gbq>=0.10.0"], | |||
"cors": ["flask-cors>=2.0.0"], | |||
"flask-testing": ["Flask-Testing==0.7.1"], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit. Can we lowercase this like the other packages. Package names are case insensitive.
Codecov Report
@@ Coverage Diff @@
## master #8418 +/- ##
==========================================
+ Coverage 65.69% 65.77% +0.07%
==========================================
Files 474 477 +3
Lines 23583 23726 +143
Branches 2571 2571
==========================================
+ Hits 15494 15606 +112
- Misses 7920 7951 +31
Partials 169 169
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall LGTM. Just a few comments.
superset/__init__.py
Outdated
manifest_processor, | ||
results_backend_manager, | ||
security_manager as ext_security_manager, | ||
talisman as ext_talisman, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any reason these need to be names ext_
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For clarity at this point. I suppose I could have just imported extensions
and then referenced each item upon assignment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@craig-rueda what I meant was you import talisman
as ext_talisman
and then on line #59 there is talisman = ext_talisman
and thus why not just,
from superset.extensions import talisman
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I didn't know that was valid. Chaining imports :). I'm still a bit of a Python noob...
|
||
if __name__ == '__main__': | ||
cli() | ||
superset() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we could annex this file and have setup.py
reference superset.cli:superset
.
appbuilder.indexview = SupersetIndexView | ||
appbuilder.base_template = "superset/base.html" | ||
appbuilder.security_manager_class = custom_sm | ||
appbuilder.update_perms = False |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let FAB handle these:
FAB_UPDATE_PERMS
defaults toTrue
in FAB, setting superset's default toTrue
could be done onconfig.py
FAB_INDEX_VIEW
FAB_BASE_TEMPLATE
We also have FAB_SECURITY_MANAGER_CLASS
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was in master - would prefer to tune configs in a subsequent PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually I think we need to force-set those. Superset is force-pinning FAB configurations as it operates on these assumptions.
|
||
APP_DIR = os.path.dirname(__file__) | ||
|
||
appbuilder = AppBuilder(update_perms=False) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we force it? set FAB_UPDATE_PERMS
to False
on config
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was in master - would prefer to tune configs in a subsequent PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we want to force it. Docs are clear about running superset init
which takes care of that, and we have many other CLI commands and things where we don't want to operate against the db, create delays and flood the logs with unrelated operations.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This LGTM personally. It's a bit risky, but well overdue and a great step forward.
Let's push this forward if other committers are ok with it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM amazing refactor!
@craig-rueda we ran into an issue with Celery using the configuration defined here related to running outside of an app context and need to use,
instead. I was wondering whether you could update |
CATEGORY
Choose one
SUMMARY
WIP - This is the first of several PRs which addresses SIP-24
ADDITIONAL INFORMATION
REVIEWERS