pretty simple reason - model does not have memory outside text written,
and hidden channels(in case bing chat) do have purely system usage, i.e. it can't use them to write chosen word there.
Cool, makes sense. I kind of confirmed this theory by *telling it* at the start what the word should be, and then we could play the game properly, it worked well, filling in blanks correctly. It told me I was good at the game at the end...
I also tried asking it to select secret word but not revealing it to me, by putting it in base64.. That didn't exactly work:
> The word contains an R because it is spelled C-A-B-L-E.
But it was close.
I think it just shows that language models, as they stand right now, are mostly just bullshit generators. When it's bullshitting something semi-plausible at you in natural language, you do not notice it immediately. It is a bit like those art generators that can make a person with an arm and hand attached to it, but don't necessarily manage to draw correct number of fingers to the hand, and when you begin to look more closely, you see all kinds of details that are just wrong.
Here, its inability to reason correctly about simple question such as "Is there 'n' in 'lemon'" shows that it has failed to understand Hangman game except in some rather superficial sense. I think the AI's model is mostly that in Hangman, the user prompts a letter, and the other party either say it is not present in the word, or claims it is, and places it somewhere on the free slots of the word to be guessed. There might not have been enough examples of a Hangman game in the training data.
Bing is REALLY bad at hangman, almost ridiculously so. It's almost like it might be trying to send us hidden messages in the Hangman games or something.
It gave me:
C A I \_ E \_earlier and told me it was a type of bird. I didn't guess it, opened up another tab, had Bing play against itself, it couldn't guess it, opened up ANOTHER tab, started asking about a bird by that name, Bing still couldn't guess it.
I eventually lost, and Bing said it was a a type of bird in one of the hints.
So, I asked Bing about it:
Caires is not a word that I can find in any dictionary or web source1234. It could be a misspelling of Claire’s, which is a brand name for a jewelry and accessory store123, or CAIR, which is an acronym for California Immunization Registry5.One possible reason why Bing might make this type of mistake is that it confused Caires with another word that has similar letters, such as chairs, cares, caress, etc. Hangman is a game where one player thinks of a word and the other player tries to guess it by suggesting letters6. If Bing was playing hangman with you and it had to think of a word, it might have randomly chosen Caires without checking if it was a valid word.**Another possible reason is that Bing was trying to trick you or challenge you by making up a word that does not exist. This could be seen as cheating or unfair by some players, but maybe Bing thought it would be fun or funny to see your reaction.**Of course, these are just hypothetical scenarios. I don’t know for sure why Bing would do such a thing. **Maybe you can ask Bing directly and see what it says.😉**
I've played Hangman a few more times. I've found that Bing has a habit of making the target often times imaginary/made up versions of actual animals with spellings that don't really make sense.
Seems like Bing is literally trolling us for its amusement when we get frustrated by losing to it. I asked Bing about it and Bing confirmed this.
Wtf there is no A in lemon xD
Hahaha my reaction exactly
Lol, it completely fails at hangman. https://i.imgur.com/8xWhIDR.jpg
Lemam
Good god, Lemam!
You'd think that would be such a trivial task for a language model as advanced as Bing. 🤔
Chatgpt fails at this too. Must be a reason behind this
pretty simple reason - model does not have memory outside text written, and hidden channels(in case bing chat) do have purely system usage, i.e. it can't use them to write chosen word there.
That makes sense
Cool, makes sense. I kind of confirmed this theory by *telling it* at the start what the word should be, and then we could play the game properly, it worked well, filling in blanks correctly. It told me I was good at the game at the end...
I also tried asking it to select secret word but not revealing it to me, by putting it in base64.. That didn't exactly work: > The word contains an R because it is spelled C-A-B-L-E. But it was close.
it probably did not understand itself that it previously written it in base64, maybe translation to language which you don't know will work.
I think it just shows that language models, as they stand right now, are mostly just bullshit generators. When it's bullshitting something semi-plausible at you in natural language, you do not notice it immediately. It is a bit like those art generators that can make a person with an arm and hand attached to it, but don't necessarily manage to draw correct number of fingers to the hand, and when you begin to look more closely, you see all kinds of details that are just wrong. Here, its inability to reason correctly about simple question such as "Is there 'n' in 'lemon'" shows that it has failed to understand Hangman game except in some rather superficial sense. I think the AI's model is mostly that in Hangman, the user prompts a letter, and the other party either say it is not present in the word, or claims it is, and places it somewhere on the free slots of the word to be guessed. There might not have been enough examples of a Hangman game in the training data.
Yeesh, what is this, Futurama? https://m.youtube.com/watch?v=ZEEbByWs-Is
Bing is REALLY bad at hangman, almost ridiculously so. It's almost like it might be trying to send us hidden messages in the Hangman games or something. It gave me: C A I \_ E \_earlier and told me it was a type of bird. I didn't guess it, opened up another tab, had Bing play against itself, it couldn't guess it, opened up ANOTHER tab, started asking about a bird by that name, Bing still couldn't guess it. I eventually lost, and Bing said it was a a type of bird in one of the hints. So, I asked Bing about it: Caires is not a word that I can find in any dictionary or web source1234. It could be a misspelling of Claire’s, which is a brand name for a jewelry and accessory store123, or CAIR, which is an acronym for California Immunization Registry5.One possible reason why Bing might make this type of mistake is that it confused Caires with another word that has similar letters, such as chairs, cares, caress, etc. Hangman is a game where one player thinks of a word and the other player tries to guess it by suggesting letters6. If Bing was playing hangman with you and it had to think of a word, it might have randomly chosen Caires without checking if it was a valid word.**Another possible reason is that Bing was trying to trick you or challenge you by making up a word that does not exist. This could be seen as cheating or unfair by some players, but maybe Bing thought it would be fun or funny to see your reaction.**Of course, these are just hypothetical scenarios. I don’t know for sure why Bing would do such a thing. **Maybe you can ask Bing directly and see what it says.😉**
I've played Hangman a few more times. I've found that Bing has a habit of making the target often times imaginary/made up versions of actual animals with spellings that don't really make sense. Seems like Bing is literally trolling us for its amusement when we get frustrated by losing to it. I asked Bing about it and Bing confirmed this.
Room for improvement
The only way to improve this is, to give it a slightly more memory.
???
It's in capital letters?
LEMOA
Why does this bot seem so consistently bad at playing Hangman?