Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core mephisto directory structure could use work #285

Closed
JackUrb opened this issue Oct 21, 2020 · 1 comment
Closed

Core mephisto directory structure could use work #285

JackUrb opened this issue Oct 21, 2020 · 1 comment

Comments

@JackUrb
Copy link
Contributor

JackUrb commented Oct 21, 2020

The current directory structure doesn't have as much clear subdivision as it probably should, likely due to natural shifting of the design. Right now it contains:

- client
   - <apis, clis, python server code>
- core
   - python classes that are any number of steps above the data model
- data_model
   - <elements of the data model> 
   - <some configuration for the data model> 
   - <base forms of the core abstractions>
- providers
   - <provider implementations>
- scripts
   - <arbitrary helper scripts>
- server
   - architects
      - <implementations of the architect class>
   - blueprints
      - <blueprint implementations>
   - channels
      - <Channels for architects to use>
- tasks
   - <empty suggested directory that people can use as a workspace>
- utils
   - file of utilities for people to use in scripts

At the moment the contents of providers and server should likely better display the abstractions. webapp is mostly related to the client, and the contents of data_model, core, and utils don't fully follow the high level architecture design laid out in our main doc.

I propose the following setup:

- client
   - <apis, clis>
   - full_app
      - <python server for the full application>
      - webapp
         - the webapp files for specific frontends
    - review_app
       - ...
    # Shared frontend code between views should exist in `packages` at the top level
- data_model
   - <only data model elements as described in the architecture document>
- abstractions
   - <Blueprint>
   - <Architect>
   - <CrowdProvider>
   - blueprints
      - <implementations of blueprints>
   - architects
      - channels
         - <implementations of channels>
      - <implementations of architects>
   - providers
      - <implementations of crowd providers>
- operations
   - <classes that implement parts of the operations layer, as described by the architecture docs>
- tools
   - <classes that operate on the Mephisto data model, but not as operations, such as the MephistoDataBrowser, and script utils>
- scripts
   - <arbitrary helper scripts>

To make this progression, I'd suggest we add a deprecation notice at v0.2 to the old imports, having them point to the new locations (and do a pass-through import) and then remove them in a v0.3.

Open to discussion on what this might look like or if there are better ideas!

@pringshia
Copy link
Contributor

pringshia commented Oct 21, 2020

Overall, I really like this and am all in favor of a re-org. As we've developed, enhanced, and clarified our mental model around Mephisto, I think it makes sense to adapt the structure of the code accordingly. I like how this also follows nicely with the structure of the architecture overview docs.

The transition strategy also seems sensible. For a while the codebase will likely look a little messy as we support both interfaces, but that should be fine. We can also tag certain directories or files with a marked for deletion flag so when it does come time to transition out, that process will be easy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants