My review of Clozemaster Pro after 3 months

Now that I’ve spent 3 months with Clozemaster Pro, during which I’ve used the program every day, I want to share my impressions with you. Some of the points I bring up are mentioned in other threads, but some are not.

Biggest plusses:

  1. Cloze exercises are a good way to train my responses (either “think of the translation” or “think of the right form of the word”), so using a tool based on them feels productive.
  2. I like having a text input option, which I use exclusively. The fact that Duolingo forced me to use multiple-choice made it too easy.

Biggest negatives:

  1. The fact that the right answer is often flashed at me before I have a chance to guess it both irritates me and forces me to work around it (train myself to look away from the screen or block the input field for a second with my finger until the flash goes away). I’ve reported on this in a separate thread: Suppress the initial flash of the cloze word - #4 by alanf - Questions, Suggestions, Feedback - Clozemaster

  2. The source of the sentences is the Tatoeba corpus, with all its quirks and deficiencies (and I know them well, being an administrator of the site). In particular, the most prolific contributor to the site insists on using, for the most part, only two names, both of them very Anglophone (“Tom” and “Mary”), one city (“Boston”), one month (“October”), and one time (“two thirty”). The same imbalance and lack of diversity appear in the sentences imported by Clozemaster. This is a disaster for language learners, who never get a chance to learn and practice other names, cities, months, and times. It’s particularly bad for those learning a language like Russian, since the name “Mary” cannot be declined like a real Russian name. It could be mitigated with some work on the Clozemaster side (for instance, by performing searches that exclude these words and phrases, or that look for names like “Sergei” and “Olga”, and importing the results). I’m also working on writing sentences at Tatoeba with a variety of names and encouraging others to do the same. Once I’ve collected them, I can tell you how to import them.

  3. Since more than one word will often fit in the cloze, and there’s no clue given as to which one is wanted (for instance, “formal” or “informal” to indicate whether a verb form corresponding to the formal or informal second person is wanted when the sentence does not include a pronoun that makes this clear), I need to rely on (a) the function that show me whether I’ve typed the right characters so far and (b) the functionality that restricts the text box to the width of the right answer. This means that I train myself to cheat by guessing the first letter, then the letter after that, and so on. There is some value to the exercise, since I can apply the lessons learned (for instance, which prefixes are most common) when I guess the word in real life. But it’s still not ideal.

  4. Sentences are not really ranked in order of difficulty, so there’s no progression. In the fluency section, they’re ranked by frequency of cloze word, which is at least a start. However, “rare” words are often not very good ones (for instance, the Russian transliteration of the English name “Bill”). Outside the fluency section, there is no ranking by difficulty at all.

  5. Multiple sentences often have the same cloze word in the same context, so there’s a lot of duplication. For instance, there are many sentences where “Boston” is the cloze word and is preceded by “in”.

  6. Marking a sentence whose cloze word I already know as “ignore” requires two clicks (and currently cannot be done via a keyboard shortcut). See some of the posts on the thread More repetition scheduling options? - Questions, Suggestions, Feedback - Clozemaster It also cheats me of the points I would receive by guessing it correctly. Since I get 8 + 16 + 24 + 32 = 80 points (assuming I’m using text input) for guessing a word correctly at the 25%, 50%, 75%, and 100% mastery levels, this is a real disincentive if I’m going to pace myself by the number of points I collect.

  7. Outside the fluency area, indicating that I want to review a sentence even though I’ve gotten it right seems to have little effect. I’m forced to mark it a favorite and to rely on withdrawing it manually from the favorites list when I feel I’ve really learned it.

  8. The algorithm used to determine the spaced repetition, unlike the algorithm used by Anki, does not allow any input to indicate how easy I find the sentence. This means that I am forced to repeat easy sentences as many times as difficult ones (unless I mark them “ignore”, which, as I’ve explained above, is both time-consuming and destructive to the score incentive). See More repetition scheduling options? - Questions, Suggestions, Feedback - Clozemaster

  9. On an Android tablet, I can’t use a keyboard shortcut to press the “Next” button. This forces me to go back and forth between the keyboard and the mouse or mousepad.

Many of these problems magnify each other. For instance, the fact that sentences are not ranked by difficulty is aggravated by the fact that marking a sentence “ignore” is so cumbersome, with the net result being that after three active months on the site, I can’t choose to see sentences that are fundamentally different from the ones that I saw originally.

I welcome any discussion. I hope you will find this useful.

4 Likes

Thanks for all the feedback! This is indeed all really helpful. Glad to hear about the positives, and I’ll try to respond to each of the negatives below.

  1. Thanks for the reminder - I wasn’t able to reproduce this initially, will give it another shot.

  2. Sounds good and please keep me posted on the new sentences. disaster for language learners certainly seems a bit extreme/hyperbolic. In any case perhaps we could try removing/reducing those sentences on the Clozemaster side like you mentioned.

  3. What do you think would be ideal for indicating the correct missing word when multiple are possible? I’d added the color hints and dynamic text box width to provide clues to help with formal/informal and other possible cloze-word ambiguities like you described. I could try adding literal “first person singular formal” hints, but might argue that it’d just be another way to “cheat”.

  4. How would you determine the “difficulty” of a given sentence?

  5. I might argue that the duplication can often be good for learning vocab :slight_smile: “Boston” is indeed often preceded by the word “in”. That said I am planning on capping the number of sentences for a given cloze word in the Most Common Words collections.

  6. Improving the ignore functionality (single click + shortcut) is on the to-do list. It also cheats me of the points I would receive by guessing it correctly. I’m not sure I understand - it seems weird to give points when ignoring or manually mastering a sentence. What do you think would be a better approach here?

  7. indicating that I want to review a sentence even though I’ve gotten it right seems to have little effect. I’m not sure I understand - what actions are performing and what results would you like to see?

  8. Good point! I might change the spaced repetition algorithm to something more similar to Anki’s at some point in the future. For the moment, however, you can manually master sentence, which, depending on how you have your review intervals set, will set it to come up for review at some point in the future (versus ignore which removes it from your queue entirely).

  9. I’ll see what I can do for shortcuts on the mobile app. Thanks for letting me know. In the meantime the mobile version of the Clozemaster website should work just as well and allow for better handling of keyboard input.

Thanks again and please feel free to let me know if you think of anything else!

1 Like

Well, it’s a small disaster. :slight_smile:

You could use “second person singular” and “second person plural”. That would be exactly what I’m looking for, and it wouldn’t be cheating.

Frequency of missing word is okay within the fluency section. Not sure how it would be done outside the fluency section, but if anything occurs to me, I’ll let you know.

Capping the number of sentences for a given cloze word would be very nice. A little duplication is okay, but too much interferes with learning.

Glad to hear it! I got that impression from another thread.

I agree that points should not be given when ignoring or manually mastering a sentence. If you were starting the website from scratch, I would argue that it would be better to “flatten out” the number of points awarded when a word is successfully reviewed. Maybe +8, +10, +12, +14 instead of +8, +16, +24, +32? However, now that the point system has been in place for a while, it would be unfair to switch, given that you keep “all-time” records.

Let’s say I’m doing one of the “grammar challenges” and manage to get a cloze word correct with some amount of guesswork. I submit the word and am told that I’ve gotten it right. I then press the red “More practice!” button rather than the “Got it!” button. There’s no guarantee that I’m going to see that sentence again even though I’ve indicated that I want to.

I hadn’t used the “manually master” button yet (or bothered to figure out what the checkmark meant), but I will. It requires fewer clicks than the “ignore” button, since I don’t need to indicate whether I mean the sentence or the word. However, I would either rather have a confirmation dialog that accepts an easy-to-press shortcut key (such as Enter) not have the confirmation dialog at all.

You’re right, it does. I’ll try using that from now on.

You’re welcome, and I will! Thanks for your suggestions, and for listening to my feedback.

I can understand why some people would want to purge the database for duplicates but I do think this also needs some caution:-

  • As well as learning the Cloze word in a sentence, I also try to learn any other new words (to me) in the sentence as well. If numerous sentences were purged, I’d be concerned about losing touch with associated vocabulary that I’d learnt as a byproduct of a Cloze word that had been repeated numerous times. Does every word which appears in any sentence on CM then appear as the Cloze word in another sentence? If so, then this would not be an issue.
  • Connected to my last point, a word may sometimes repeated but it might have a different meaning or nuance in another context, or when used as part of a set phrase, or idiom. A wholesale purge might lose the ‘richness’ and complexity of the target language’s database offering on CM. This may not be a problem for a beginner but it could be for any intermediate or advanced user.

I’m not sure what the best solution would be. The app currently allows the user to exclude sentences themselves and to choose the fast track version, which cuts out the ‘fat’ of the whole available database, if they so wish. Maybe it is better to continue to allow the user to customise the database themselves according to their own needs, either with the current tools the app has, or enhanced ones?

I went through the sentences referring to the most repetitive words such as ‘‘Boston’’ or ‘‘Tom’’ or ‘‘tennis’’ and manually selected to ignore the easiest sentences, where I already know all of the words in the sentence.

One last point. As you may have seen, I’ve spent a lot of time adding numerous sentence notes on the app to help my learning process, as well as hopefully help other users. If you do purge the sentence database, would it be possible to exclude sentences that have a sentence note ‘attached’ to them?

Thanks!

1 Like

To speak to this point in particular, I personally don’t think there should be much concern with restricting site modifications/improvements to avoid “rocking the boat” when it comes to scores. Maybe this is just because I’m not always disciplined enough to practice every day (or even for a particularly long time each day when I do get on a streak) but it seems a bit counterproductive to constrain Clozemaster for that reason since productive language learning is the main goal here, not who has the most score.

2 Likes

How would you determine the “difficulty” of a given sentence?

A valid way to determine the difficulty of a question would be to use how often it is incorrectly answered by users.

3 Likes

That is a good idea!