Skip to content

Commit

Permalink
Fixed failing checks
Browse files Browse the repository at this point in the history
  • Loading branch information
meta-paul committed Dec 19, 2023
1 parent f82c7be commit aba6369
Show file tree
Hide file tree
Showing 12 changed files with 112 additions and 88 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/cypress-end-to-end-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ jobs:
browser: chrome
project: ./mephisto/abstractions/blueprints/static_html_task/source
config-file: ./cypress.config.js
start: python ./examples/simple_static_task/static_test_script.py
start: python ./examples/simple_static_task/run_task.py
wait-on: "http://localhost:3000/?worker_id=x&assignment_id=1"
headless: true

Expand Down
2 changes: 1 addition & 1 deletion docker/docker-compose.dev.vscode.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,6 @@ services:
/tmp/debugpy
--wait-for-client
--listen 0.0.0.0:5678
/mephisto/examples/simple_static_task/static_test_script.py
/mephisto/examples/simple_static_task/run_task.py
",
]
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,15 @@ Some users prefer to keep Mephisto entirely contained. Docker is one option for

```bash
# Build the docker image and tag with name 'mephisto'
$ docker build -t mephisto .
$ docker build -t mephisto .

# By default, the container just runs `mephisto check`:
$ docker run mephisto
Mephisto seems to be set up correctly.

# You can also bind ports and pass in shell commands, e.g. to run
# a task directly from the container
$ docker run -p 3000:3000 mephisto bash -c 'cd mephisto/examples/simple_static_task && python static_test_script.py'
$ docker run -p 3000:3000 mephisto bash -c 'cd mephisto/examples/simple_static_task && python run_task.py'

```
By default, Mephisto run data will be stored at `/mephisto/data` within the container.
40 changes: 20 additions & 20 deletions docs/web/docs/guides/tutorials/first_task.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ To get started, you can launch a task by executing a run script. For instance, l

```bash
$ cd examples/simple_static_task
$ python static_test_script.py
$ python run_task.py
```

This will launch a local HTTP server with the task hosted, based on the default configuration options:
Expand All @@ -31,21 +31,21 @@ defaults:
- /mephisto/architect: local
- /mephisto/provider: mock
```
We'll dig into *how* this works [later](#33-default-abstraction-usage).
We'll dig into *how* this works [later](#33-default-abstraction-usage).

### 1.2 Access the task

By default, the task should be hosted at `localhost:3000`. This default is set by the `LocalArchitect`, which is used based on the `- /mephisto/architect: local` line above. Navigating to this address should show you the preview view for the task.

Actually being able to access this task is done by providing `worker_id` and `assignment_id` URL params, like `localhost:3000/?worker_id=x&assignment_id=1` The `MockProvider` interprets these to be a test worker, which you can use to try out tasks.
Actually being able to access this task is done by providing `worker_id` and `assignment_id` URL params, like `localhost:3000/?worker_id=x&assignment_id=1` The `MockProvider` interprets these to be a test worker, which you can use to try out tasks.

Try navigating here and completing a task by selecting a value (no need to use the file upload). After hitting submit you'll note that the window alerts you to the data that was sent to the Mephisto backend.

To work on another task, you'll want to change the `assignment_id` in your url. This will let `Worker` "x" work on another task. Complete and submit this too.

If you try to work on another task, you'll note that the system states you've worked on the maximum number of tasks. On this task, this is because Mephisto has launched the same two tasks twice to attempt to get different workers to complete them, and as "x" you've already completed both tasks. More on this [later.](#unit-creation-explained)

If you change to another `worker_id`, however, you can complete two more tasks. Do this and the Mephisto process should shut down cleanly.
If you change to another `worker_id`, however, you can complete two more tasks. Do this and the Mephisto process should shut down cleanly.


### 1.3 Reviewing tasks
Expand All @@ -60,24 +60,24 @@ For your convenience, for the `html-static-task-example` task we've already prov
$ python examine_results.py
Do you want to (r)eview, or (e)xamine data? Default examine. Can put e <end> or e <start> <end> to choose how many to view
>> r
Input task name:
Input task name:
```

Here we can enter the default task name for this task, `html-static-task-example` to see the results. You can also leave the two qualifications blank, as we aren't dealing with these yet.

```
Input task name: html-static-task-example
If you'd like to soft-block workers, you'll need a block qualification. Leave blank otherwise.
Enter block qualification:
Enter block qualification:
If you'd like to qualify high-quality workers, you'll need an approve qualification. Leave blank otherwise.
Enter approve qualification:
Enter approve qualification:
Starting review with following params:
Task name: html-static-task-example
Blocking qualification: None
Approve qualification: None
Press enter to continue...
Press enter to continue...
```
After pressing enter, you'll be able to take action on some of the incoming tasks. You can `(a)ccept, (r)eject, (p)ass` using `a`, `r`, or `p`, using the caps version to apply to all the `Unit`s for a specific worker. Passing is used for work you think was attempted legitimately, but may have not been done
After pressing enter, you'll be able to take action on some of the incoming tasks. You can `(a)ccept, (r)eject, (p)ass` using `a`, `r`, or `p`, using the caps version to apply to all the `Unit`s for a specific worker. Passing is used for work you think was attempted legitimately, but may have not been done
```
Reviewing for worker x, (1/2), Previous (First time worker!) (total remaining: 4)
-------------------
Expand Down Expand Up @@ -105,7 +105,7 @@ Mephisto configuration options can be inherited from a number of different locat
```bash
$ mephisto wut architect=local

Architect Arguments
Architect Arguments
╭────────────────────┬──────┬───────────┬──────────────────────────────┬─────────┬──────────╮
│ dest │ type │ default │ help │ choices │ required │
├────────────────────┼──────┼───────────┼──────────────────────────────┼─────────┼──────────┤
Expand All @@ -127,10 +127,10 @@ $ mephisto wut architect=local
**For the blueprint:**
```bash
$ mephisto wut blueprint=static_task
Tasks launched from static blueprints need a source html file to display to workers,
as well as a csv containing values that will be inserted into templates in the html.
Tasks launched from static blueprints need a source html file to display to workers,
as well as a csv containing values that will be inserted into templates in the html.

Blueprint Arguments
Blueprint Arguments
╭───────────────────┬─────────┬────────────────────┬───────────────────┬─────────┬──────────╮
│ dest │ type │ default │ help │ choices │ required │
├───────────────────┼─────────┼────────────────────┼───────────────────┼─────────┼──────────┤
Expand Down Expand Up @@ -199,7 +199,7 @@ For our given example task, the values we are using for these options are availa
As a simple starting point, we can try launching the server on a different port. Right now the default is `3000`, but with the following command we can set that ourselves:
```
python static_test_script.py mephisto.architect.port=1234
python run_task.py mephisto.architect.port=1234
```
This should launch the same task, but now available on the port `1234` rather than `3000`.
Expand Down Expand Up @@ -237,7 +237,7 @@ mephisto:
Save this configuration file, and you're ready to launch again:
```bash
$ python static_test_script.py conf=my_config
$ python run_task.py conf=my_config
```
You'll note that Mephisto launches the task under your new task name:
```
Expand Down Expand Up @@ -282,7 +282,7 @@ mephisto:
```
Save this configuration file, and you're ready to see your task live:
```bash
$ python static_test_script.py conf=my_config
$ python run_task.py conf=my_config
```
Mephisto should print out a link to view your task on the mturk sandbox, like `https://workersandbox.mturk.com/mturk/preview?groupId=XXXXXXXXXXXXXXXX`. Navigate here and you're working on the same task, available on MTurk (on the sandbox at least)!
Expand All @@ -292,13 +292,13 @@ Complete this task, and you can review it in the same way as before.
## 3. Task breakdown
Now that you've gotten a task running, this section gives a quick overview on some of the components in the configs and `static_test_script.py` that led to the observed behaviors. The goal is to ensure you can write your own run files by the end of the tutorial sections.
Now that you've gotten a task running, this section gives a quick overview on some of the components in the configs and `run_task.py` that led to the observed behaviors. The goal is to ensure you can write your own run files by the end of the tutorial sections.
### 3.1 Config registration
Mephisto wires up to configuration using standard Hydra syntax, but with both `yaml` files (for ease of writing) _and_ structured configs (for ease of documentation).
Mephisto wires up to configuration using standard Hydra syntax, but with both `yaml` files (for ease of writing) _and_ structured configs (for ease of documentation).
Here's the config we've set up for this example:
```python
# static_test_script.py
# run_task.py
from mephisto.abstractions.blueprints.static_html_task.static_html_blueprint import (
BLUEPRINT_TYPE_STATIC_HTML,
)
Expand All @@ -312,7 +312,7 @@ This is all you really *need* to launch a Mephisto task! The `@task_script` deco
Of course, there's quite a bit of 'magic' happening underneath the hood thanks to the script utilities. This version is explicit to show where you may add customization, and re-ordered for understanding:
```python
# modified static_test_script.py
# modified run_task.py
from mephisto.abstractions.blueprints.static_html_task.static_html_blueprint import (
BLUEPRINT_TYPE_STATIC_HTML,
)
Expand Down
8 changes: 4 additions & 4 deletions examples/simple_static_task/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@ python run_task.py
and can additionally be launched with an onboarding step by specifying an onboarding qualification:

```console
python static_run_with_onboarding.py
python run_task_with_onboarding.py
```

## Submit button customization
### Hide the submit button
### Hide the submit button
Writing the markup below in `demo_task.html` will allow you to hide the submit button.

```html
Expand All @@ -44,10 +44,10 @@ python run_task.py
```
This will run the task.

Then go into the `mephisto/abstractions/blueprints/static_html_task/source` and run
Then go into the `mephisto/abstractions/blueprints/static_html_task/source` and run
```console
npm run test
```
```
to open cypress.

Select the Chrome browser and click on the one spec that shows up to run the tests.
2 changes: 1 addition & 1 deletion examples/simple_static_task/run_task_ec2_prolific.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ def main(operator, cfg: DictConfig) -> None:

# Mephisto qualifications
shared_state.qualifications = [
make_qualification_dict('sample_qual_name', QUAL_GREATER_EQUAL, 1),
make_qualification_dict("sample_qual_name", QUAL_GREATER_EQUAL, 1),
]

# Prolific qualifications
Expand Down
44 changes: 22 additions & 22 deletions mephisto/abstractions/providers/prolific/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,14 @@ Shared state handles both custom and Prolific-supported qualifications:
```python
shared_state.prolific_specific_qualifications = [
{
'name': 'AgeRangeEligibilityRequirement',
'min_age': 18,
'max_age': 100,
"name": "AgeRangeEligibilityRequirement",
"min_age": 18,
"max_age": 100,
},
]

shared_state.qualifications = [
make_qualification_dict('sample_qual_name', QUAL_GREATER_EQUAL, 1),
make_qualification_dict("sample_qual_name", QUAL_GREATER_EQUAL, 1),
]
```

Expand All @@ -48,7 +48,7 @@ under the key `prolific_eligibility_requirements` like so:
mephisto:
provider:
prolific_eligibility_requirements:
- name: 'AgeRangeEligibilityRequirement'
- name: "AgeRangeEligibilityRequirement"
min_age: 10
max_age: 20
```
Expand All @@ -63,35 +63,35 @@ List of supported Eligibility Requirements for `SharedState.prolific_specific_qu
```python
[
{
'name': 'AgeRangeEligibilityRequirement',
'min_age': '<value>',
'max_age': '<value>',
"name": "AgeRangeEligibilityRequirement",
"min_age": "<value>",
"max_age": "<value>",
},
{
'name': 'ApprovalNumbersEligibilityRequirement',
'minimum_approvals': '<value>',
'maximum_approvals': '<value>',
"name": "ApprovalNumbersEligibilityRequirement",
"minimum_approvals": "<value>",
"maximum_approvals": "<value>",
},
{
'name': 'ApprovalRateEligibilityRequirement',
'minimum_approval_rate': '<value>',
'maximum_approval_rate': '<value>',
"name": "ApprovalRateEligibilityRequirement",
"minimum_approval_rate": "<value>",
"maximum_approval_rate": "<value>",
},
{
'name': 'CustomBlacklistEligibilityRequirement',
'black_list': '<value>',
"name": "CustomBlacklistEligibilityRequirement",
"black_list": "<value>",
},
{
'name': 'CustomWhitelistEligibilityRequirement',
'white_list': '<value>',
"name": "CustomWhitelistEligibilityRequirement",
"white_list": "<value>",
},
{
'name': 'JoinedBeforeEligibilityRequirement',
'joined_before': '<value>',
"name": "JoinedBeforeEligibilityRequirement",
"joined_before": "<value>",
},
{
'name': 'ParticipantGroupEligibilityRequirement',
'id': '<value>',
"name": "ParticipantGroupEligibilityRequirement",
"id": "<value>",
},
]
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -300,38 +300,39 @@ function ModalForm(props: ModalFormProps) {

<hr />

{BONUS_FOR_WORKER_ENABLED && props.data.form.checkboxGiveBonus !== undefined && (
<>
<Form.Check
type={"checkbox"}
label={"Give Bonus"}
id={"giveBonus"}
checked={props.data.form.checkboxGiveBonus}
onChange={() =>
onChangeGiveBonus(!props.data.form.checkboxGiveBonus)
}
/>

{props.data.form.checkboxGiveBonus && (
<Row className={"second-line"}>
<Col xs={4}>
<Form.Control
size={"sm"}
type={"input"}
placeholder={"Bonus"}
value={props.data.form.bonus || ""}
onChange={(e) => onChangeBonus(e.target.value)}
/>
</Col>
<Col>
<span>Amount (cents)</span>
</Col>
</Row>
)}
{BONUS_FOR_WORKER_ENABLED &&
props.data.form.checkboxGiveBonus !== undefined && (
<>
<Form.Check
type={"checkbox"}
label={"Give Bonus"}
id={"giveBonus"}
checked={props.data.form.checkboxGiveBonus}
onChange={() =>
onChangeGiveBonus(!props.data.form.checkboxGiveBonus)
}
/>

{props.data.form.checkboxGiveBonus && (
<Row className={"second-line"}>
<Col xs={4}>
<Form.Control
size={"sm"}
type={"input"}
placeholder={"Bonus"}
value={props.data.form.bonus || ""}
onChange={(e) => onChangeBonus(e.target.value)}
/>
</Col>
<Col>
<span>Amount (cents)</span>
</Col>
</Row>
)}

<hr />
</>
)}
<hr />
</>
)}

{props.data.form.checkboxBanWorker !== undefined && (
<>
Expand All @@ -351,7 +352,9 @@ function ModalForm(props: ModalFormProps) {

<Form.Check
type={"checkbox"}
label={FEEDBACK_FOR_WORKER_ENABLED ? "Write Note" : "Write Note for Yourself"}
label={
FEEDBACK_FOR_WORKER_ENABLED ? "Write Note" : "Write Note for Yourself"
}
id={"reviewNote"}
checked={props.data.form.checkboxReviewNote}
onChange={() =>
Expand Down Expand Up @@ -380,7 +383,9 @@ function ModalForm(props: ModalFormProps) {
id={"reviewNoteSend"}
checked={props.data.form.checkboxReviewNoteSend}
onChange={() =>
onChangeWriteReviewNoteSend(!props.data.form.checkboxReviewNoteSend)
onChangeWriteReviewNoteSend(
!props.data.form.checkboxReviewNoteSend
)
}
/>
</Col>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,12 @@ function TasksPage(props: PropsType) {
}
};

exportTaskResults(taskId, onSuccessExportResults, setLoadingExportResults, onError);
exportTaskResults(
taskId,
onSuccessExportResults,
setLoadingExportResults,
onError
);
};

useEffect(() => {
Expand Down
Loading

0 comments on commit aba6369

Please sign in to comment.