Wow, what a great new feature that caught me by surprise this morning! @mike I seem to only see this on the fluency fast track. Are you planning on expanding this to every sentence?
6 Likes
Fabulous! Great idea!
1 Like
some of the answers are definitely more useful than others. I have no idea if this is possible, but it would also be awesome if we could ask a follow-up question to the chatbot.
2 Likes
What has annoyed me about ChatGPT (other than the massively overbaked hype) is that it is lauded as some huge breakthrough in AI, but in reality it’s little more than smoke, mirrors and search engines. It has no comprehension of the content it generates, and does little enough fact checking.
As expected when I looked through simple sentences in FFT in Italian, its explanations were fine. As soon as it encountered something that required actual understanding, though (and that didn’t take many questions), it fell in a heap.
For example:
“Ti ho chiamata perché mi serve un favore” is an Italian phrase that can be translated as “I called you because I need a favor” in English.
True enough.
Let’s break it down word by word for an easier understanding:
Clearly you and I have different definitions of “understanding”, ChatGPT.
- “Ti” means “you” (used as an object pronoun referring to the person being called, specifically a female).
Utter nonsense. “Ti” is just an informal form of “you”. It applies equally to males and females. In this case it DOES refer to a female but it certainly ISN’T because of the word “Ti”.
- “ho” means “I have”; it comes from the verb “avere” (to have) and must be used in compound tenses.
Must be? MUST be? In every single one of them? Gee, good thing I don’t have to worry about all those pesky verbs that require Essere for the passato prossimo form, then.
- “chiamata” means “called”; it comes from the verb “chiamare” (to call) and should agree in gender and number with the subject it refers to (in this case, a female person).
Again, utter nonsense. The fact that it IS in the feminine singular form tells us that the “ti” above is referring to a female in the singular. But, I should point out… it is agreeing with the OBJECT, not the SUBJECT.
Specifically, It agrees in gender and number when a direct pronoun like “ti” is used before the verb in a sentence like this. When you DON’T have such a pronoun, then:
I called Mario.
Ho chiamato Mario.
Luisa called Mario.
Luisa ha chiamato Mario.
Oh, gee, look… the form of chiamare doesn’t actually change.
- “perché” means “because”.
Finally something we agree on, because even ChatGPT and its over-hyped search engine can’t screw up A1 level language.
- “mi” means “me” or “to me” (used as an indirect object pronoun).
- “serve” means “needs” or “is needed”; it comes from the verb “servire” (to need or to be necessary).
- “un” is an indefinite article that means “a” or “an.”
- “favore” means “favor.”
Ditto.
So the entire phrase “Ti ho chiamata perché mi serve un favore” means “I called you because I need a favor” when referring to a female person.
This is disturbingly ambiguous. The person being called would be a female person in this case, but the person doing the calling does not have to be.
What narks me about this is that it will confuse the living heck out of a newbie because it’s giving them information that is just plain wrong, given the subtleties and nuances of languages like Italian.
And Italian is a (relatively) logical language. I don’t even want to THINK about what this thing is doing in French FFT.
As long as students work from the assumption that ChatGPT will probably have no idea what it is talking about for sentences which extend beyond a single clause, the explanation won’t be completely useless. But it should NEVER be trusted or assumed to be correct. Always assume the opposite and
check, and it can still be useful.
9 Likes
I suspect that Clozemaster is using the still very imperfect GPT 3.5, as GPT-4 is much more expensive.
Here is GPT-4’s output:
Much sounder, but point 3 is still not an ideal explanation and it remains a bit of a missed opportunity to present a basic grammar rule.
After a couple of weeks of use, I would rate GPT-4 as a “massively useful” foreign language assistant. However, users need to understand that, counterintuitively, it has no sense yet of fact, truth or things like “verification”.
Caution and critical thinking must still by applied to its ramblings.
4 Likes
I’m a simple soul and much prefer my books, trusty translator and more importantly, coming here to ask and discuss with other humans. Old-fashioned I’m afraid but doing ok;-)
8 Likes
Thanks for all the input! This is super helpful.
Not sure yet - it’s currently only available for the first 1,000 or so sentences for the French, Italian, and German Fast Tracks, and a bit more for the Spanish Fast Track. We used a different prompt for the Spanish Fast Track, and the output for the others seems more useful so far, so we’ll likely regenerate the Spanish explanations soon.
We’re looking for feedback like in this thread in deciding whether to keep expanding the explanations. It’s also a bit tricky where to draw in the line in terms of for which languages to make it available.
Interesting idea! It might still be too expensive for now, but hopefully prices will keep coming down.
Thanks for the feedback! That’s all helpful to know. We’ve done our deepest dive reviewing and fact checking Spanish so far, and those explanations have tended to have no issues (aside from perhaps sounding a bit robotic in some cases) in >90% of cases. To your point, however, that’s why we wanted to make it clear these explanations are generated by ChatGPT. Despite the downsides, we still thought it seemed more useful to offer the explanations in most cases than not have them at all, hence this experiment 
Doch, we’re using GPT-4, only the best for Clozemaster users , but perhaps we should be trying a different prompt 
Agreed! If you think there’s perhaps some way we should better present the explanations to make that clear, or perhaps some way to help verify, we’re open to ideas.
At the moment explanations seem to be more helpful than not, and we’ll keep the implementation requiring some action to view them for anyone who prefers not to use them. We’ll likely continue expanding explanation availability both in terms of collections and languages, and work on getting them added in the mobile app.
Any other feedback or thoughts on the above please be sure to let us know. Thanks again!
2 Likes
I like it! But yeah, maybe a prominent disclaimer that the ChatGPT explanations are AI-generated and may not be entirely correct wouldn’t hurt.
2 Likes
It is surely old-fashioned, but again, you are doing ok ;^D
1 Like
Hello, Mr. @mike ! Is it possible to add this Chat robot thing to the Arabic course, if it will? Thanx!
BTW, the tatoeba translations in most courses are the worst (!!!). It would be great if the AI could do it it/herself. I’m pretty sure any other translation would be better - I often use Google Translator and Reverso Context and they never fail to me. Again Tjanx.
1 Like
Heeeelp! ChatGPT stopped working on Chinese!!!
1 Like
I love this feature but it is unlimited on mobile and limited to 20 per month on the website. Why? I’m a paying customer and should get full access on both mobile and web!
1 Like
Thanks for the comment! We generated explanations for a large number of sentences for some of the more popular languages and collections, on the order of thousands of sentences. Those are available for both the web and mobile apps.
Then we realized generating explanations for millions of sentences, many of which might not even need an explanation (very simple/easy sentences for example), would probably get a bit expensive, so right now we’re experimenting with a way to generate explanations “on the fly”/as needed for sentences that don’t yet have them in the web app. That’s what has a limit - both to help control cost, see usage, and see if anyone would be willing to pay to generate more to help defray the costs. And that’s what’s only available in the web app at the moment.
@Afaraseq.Matok thanks for letting us know about Arabic. ChatGPT should be working for Chinese - please let us know if you’re still seeing any issues. Thanks!
3 Likes
Mike - any other plans to apply AI to your collections? I figured some fairly simple ones would be:
- spelling and syntax corrections (such as the incorrect case, person or plural/singular)
- Removing person names and city names from being used as Clozes (maybe with the exception when the city has a different name between the languages)
- making sure these person/city names are not selected during import
- filling in more alternate correct answers
It could really be a major upgrade I think (I would not be surprised at all if you’re already working on these…)
Anyhow keep up the good work!
3 Likes
We’re aiming for all 4 you mentioned, though likely with human involvement (for example trying to use AI to identify problematic sentences then having a human make/approve the actual changes).
An option to use AI to generate sentences/content for custom collections is also an idea we’re exploring, as well as attaching images to sentences that can be toggled like translations.
Thanks!
1 Like
Looking forward to these new enhancements!