Drill complete sentences

[Peeja]

I think this would be a new study type:

• Have a corpus of sentences (from Project Gutenberg, for instance).
• Filter that corpus to select sentences which contain only the words in the lesson.
• Present the words in sentence order.

A lesson made of seen words using this study type would then let you "unlock" new sentences as you learn new words.

[didoesdigital]

Thanks for taking the time to share this @Peeja! You mentioned on the Plover discord that you'd be happy to help implement it too, but it might be a little deep for a first issue. We can discuss the issue a bit first and if you decide you want to tackle at some point we can assign it to you.

This is a great idea. By drilling sentences, students can get more realistic practice of the transitions between strokes for words that often show up together.

The gamification of "unlocking" new sentences might help students stay motivated, which is important with the steep learning curve of steno.

For sentences only containing words you know, that might make it feel like more achievable practice than, for example, stories with a few extra unknown words or other apps like Typeracer.

[didoesdigital]

I have some thoughts on approaches that I'll share later on!

[Peeja]

Thanks, @didoesdigital! It's definitely a meaty first issue, but I'm used to working in React, so I'm up for at least giving it a shot if no one's going to be working on it otherwise. I'm curious what you're thinking!

[didoesdigital]

It'll be a while before I am likely to tackle this. As I mentioned on Discord, you're also welcome to warm up to the code base by tackling smaller issues first if you want to work up to this.

Ok, some general considerations to think about:

• Lesson page performance. It's not great at the moment and some lessons are worse than others e.g. Project Gutenberg sentences using top 100 words. I can think of a few ways to approach sentence drills, some of which might slow things down.
• To log met word progress properly, we'd need to show each word in a sentence as a separate 'word' rather than a whole sentence as 1 'word'.
• Processing whole sentences. Lessons currently store each word on a separate line in source text files or in a separate array item once loaded in the app. To check that a whole sentence contains only words from some list like your previously seen words or a lesson's words, it would likely be simpler to process the whole sentence together than each word (on a separate line or in a separate array item) to keep track of where sentences end.
• Sentence order. As we go, we need to maintain sentence order of the words. If, for example, you started with a random collection of words like the Introduction lesson and then checked if there are sentences that are made up of these words, you might end up with "no sentences" that match, or need to re-order them to match the sentence.
• Punctuation. Would there be pro's and con's to including or ignoring any kinds of punctuation, such as quotes or brackets? Some similar considerations are noted in this comment on #11. Personally, for sentences I prefer drilling the stories with less dialogue/fewer quotes and focus on words.
• Assembling the corpus of sentences. Project Gutenberg would be a good choice because many Typey Type lessons are based on it. I'm not sure how to get a good, clean corpus of sentences out of it. Previously, I've done things like using hugovk/gutengrep to search for whole words or sentences containing words from the top 100 words using a ridiculous regex.
• Sentences made of common words. We might use all the available sentences, or only sentences made up of the top X words. This might help with performance.
• Minimum sentence length. Upon reflection, the Project Gutenberg sentences using top 100 words lesson would probably be more satisfying if it only included sentences over 14 characters or 4 words.

I'd previously thought about this feature as a special lesson type like custom lessons or progress revision lessons rather than a study type. It might loop over each sentence to see if it contains words not in the "seen words" and, if it doesn't, split the sentence into words and add them to the lesson. As a study type, what might students expect to see in fundamental and drill lessons that have random words in non-sentence order? Having this feature work for any lesson though could be really interesting.

It might be worth some exploration and experimentation to see what issues there are (is performance even a real concern here?) and how to balance trade-offs like performance and experience. To start with, I'll attach a sample file of sentences for anyone to use.

gutenberg-sentences-from-top-100-words-with-less-punctuation.txt