-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Qset Generator #1582
base: dev/10.3.0
Are you sure you want to change the base?
Qset Generator #1582
Conversation
This is amazing, as discussed:
Some additional thoughts:
|
Thanks for the feedback! Included some fixes for the things we discussed.
Didn't even notice this, haha. We're definitely not "Previewing Prior Save". Apart from changing the title, are you suggesting changing "Select Cancel to go back to the version you were working on. Select Keep to commit to using this version." and/or altering the behavior of the "Cancel" button?
This could work for sure! |
… key is not set / invalid, & misc
Behavior of keep and cancel should probably be the same, I'm just thinking of adding some additional language to clarify that the creator loaded the questions generated by AI, and that if the creator did not load properly you can hit cancel to return to where you were. Something to that effect, since the likelihood of a creator-breaking qset is higher. |
Added custom prompts to the |
…tion generator class
…on feature to work for unsaved instances.
Major changes/additions since Cay's departure:
Compatible Widget Engines
How do I update a widget to support question generation?
|
…s without the basic_author role
Up-to-date details of this PR available here:
#1582 (comment)
Qset Generator
Uses OpenAI's gpt-3.5-turbo model to generate a qset based off the widget's
demo.json
structure and the dall-e-2 model for generating images.Setup
Generate an OpenAI key and add it to your
.env
file like:Run Materia as usual. Note: this PR adds the openai-php/client package with composer which may require a Materia reinstall for those who have it installed already.
Create a widget. The instance has to have an existing ID to work, so new instances must be saved first before using the generator.
Click on "Generate Questions" in action toolbar. Enter a descriptive topic (3+ words at least). You may also choose to extend the current qset so that the current questions are kept and include images. Including images will take much longer. See widget compatibility for what is supported.
Appending to the current qset doesn't get the exact number right all the time. It will also take longer than generating a new qset with the same number of questions.
A log file called
openai_usage.txt
will be generated and stored in the/public
folder with the generation times, tokens, costs, and more. I've attached an exampleopenai_usage.txt
from testing. Prompt tokens are for the input prompt, which includes the demo qset, your topic, and additional instructions. Completion tokens are for the output, or what the model used.openai_usage.txt
gpt-3.5-turbo costs:
Widget Compatibility
This list will be updated as individual widgets are tested.
install.yaml
): 'Each answer is stored in the answer's options > asset. Inside asset, set the 'materiaType' attribute to 'text', the 'type' attribute to 'text', and the 'value' attribute to the actual answer.'More on images...
Generating images is pricey, at $0.016 per 256x256 image. The time to generate these is even more exorbitant; however, you can still test it if you'd like on This or That. Also, OpenAI will yell at us if it tries generating images with real people in it, which may happen by accident if the questions it itself generated had real people's names.
Furthermore, image generation is not compatible for most widgets, mainly because widgets store their assets in weird places which make them hard to track and manipulate. Therefore, images will most likely never make it to production, but feel free to try it out!
To-Do
install.yaml
on generation compatibility. Disable in creator if not compatible.install.yaml