Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add info on link_task_source to docs #885

Merged
merged 6 commits into from
Aug 10, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions docs/web/docs/guides/tutorials/custom_react.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,30 @@ mephisto:
task_tags: "test,simple,button"
```

The only change we'll make is to the `task_name` (as you should on any new tasks!), but we will update the other fields as we go. It's important to note the `task_source` and `extra_source_dir` arguments though, as this is where the `StaticReactBlueprint` class will be looking for the compiled React app's Javascript bundle as well as a folder for extra static resources for the page, respectively.
It is important to give a new `task_name` as we are creating a custom task. For local development **only** it also makes sense to set `link_task_source` to true. This allows changes to propagate to your localhost server when you reload the page (otherwise you would have to shutdown and restart the server to see changes).

The `task_source` and `extra_source_dir` arguments are also of importance, as this is where the `StaticReactBlueprint` class will be looking for the compiled React app's Javascript bundle as well as a folder for extra static resources for the page, respectively.

### 1.2 Launching the task
From the current directory, you should be able to execute the run script and get a job working. We're using a different `task_name` to prevent polluting our later task with data that won't share the same format. It is a good practice to do this with initial iterations, and to change the `task_name` any time you change input or output arguments.

You can update the `task_name` and `link_task_source` values in your config and run the task like below
```bash
python run_task.py mephisto.task.task_name=custom-react-tutorial-iterating
python run_task.py
```

or you can set them when you run the task:

```bash
python run_task.py mephisto.task.task_name=custom-react-tutorial-iterating mephisto.blueprint.link_task_source=true
```
This will launch a simple task where an annotator is supposed to note a sentence as being good or bad. Clicking a button auto-submits the task. In the next sections we'll add other content.

To establish a link where your changes will be propagated to the localhost server(when you reload), create a separate terminal window and run
```bash
cd webapp && npm run dev:watch
```

Moving forward, we'll update this task so that workers are able to edit the text as well as rate the original sentence.

## 2. Providing new data to the task
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@
from mephisto.abstractions.blueprint import (
SharedTaskState,
)
from mephisto.utils.logger_core import (
get_logger,
)

if TYPE_CHECKING:
from mephisto.data_model.task_run import TaskRun
Expand All @@ -31,6 +34,7 @@
from omegaconf import DictConfig

BLUEPRINT_TYPE_STATIC_REACT = "static_react_task"
logger = get_logger(name=__name__)


@dataclass
Expand Down Expand Up @@ -120,3 +124,9 @@ def assert_task_args(
assert link_task_source == False or (
link_task_source == True and current_architect in allowed_architects
), f"`link_task_source={link_task_source}` is not compatible with architect type: {args.architect._architect_type}. Please check your task configuration."

if link_task_source == False and current_architect in allowed_architects:
logger.info(
"If you want your server to update on reload whenever you make changes to your webapp, then make sure to set \n\nlink_task_source: [blue]true[/blue]\n\nin your task's hydra configuration and run \n\n[purple]cd[/purple] webapp [red]&&[/red] [green]npm[/green] run dev:watch\n\nin a separate terminal window. For more information check out:\nhttps://mephisto.ai/docs/guides/tutorials/custom_react/#12-launching-the-task\n",
extra={"markup": True},
)
5 changes: 2 additions & 3 deletions mephisto/operations/operator.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@
)
from mephisto.abstractions.database import MephistoDB, EntryDoesNotExistException
from mephisto.data_model.qualification import QUAL_NOT_EXIST
from mephisto.tools.data_browser import DataBrowser as MephistoDataBrowser
from mephisto.utils.qualifications import make_qualification_dict
from mephisto.operations.task_launcher import TaskLauncher
from mephisto.operations.client_io_handler import ClientIOHandler
Expand All @@ -56,7 +55,7 @@
logger = get_logger(name=__name__)

if TYPE_CHECKING:
from mephisto.abstractions.blueprint import Blueprint, TaskRunner
from mephisto.abstractions.blueprint import Blueprint
from mephisto.abstractions.crowd_provider import CrowdProvider
from mephisto.abstractions.architect import Architect

Expand Down Expand Up @@ -313,7 +312,7 @@ def launch_task_run_or_die(
raise e

live_run.task_launcher.create_assignments()
live_run.task_launcher.launch_units(task_url)
live_run.task_launcher.launch_units(url=task_url)

self._task_runs_tracked[task_run.db_id] = live_run
task_run.update_completion_progress(status=False)
Expand Down
5 changes: 3 additions & 2 deletions mephisto/operations/task_launcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@
if TYPE_CHECKING:
from mephisto.data_model.task_run import TaskRun
from mephisto.abstractions.database import MephistoDB

import threading
from mephisto.utils.logger_core import get_logger
import types
Expand Down Expand Up @@ -199,7 +198,9 @@ def launch_units(self, url: str) -> None:
"""launch any units registered by this TaskLauncher"""
self.launch_url = url
self.units_thread = threading.Thread(
target=self._launch_limited_units, args=(url,), name="unit-generator"
target=self._launch_limited_units,
args=(url,),
name="unit-generator",
)
self.units_thread.start()

Expand Down