ChatGPTs rather underwhelming performance on a formal problem

The MIU System was introduced by Douglas R. Hofstadter in his fabulous book Goedel, Escher, Bach. The formal system acts on chains (words) resembled from a three letter alphabet (the set MIU) and four simple rules.

  1. If your chain ends with an „I“ you may append a „U“. Example: MI - (1) - MIU
  2. If your chain looks like „Mx“ (starts with „M“ followed by an arbitrary sequence of letters) you may write „Mxx“
  3. You can replace „III“ with „U“
  4. You can delete the sequence „UU“
Let me give you just a few examples: 
  • MIIIU - (3) - MUU - (4) - M. So from „MIIIU“, there is a series of operations to transform it to „M“. 
  • MIU - (2) - MIUIU
  • MUIIIU - (2) - MUIIIUUIIIU - (4) - MUIIIIIIU - (2x3) - MUUUU - (2x4) - M
(Hofstadter asks, whether it is possible to transform „MI“ into „MU“, try it. You will not find a spoiler in this post.)

I hope you agree that this sounds like a quite simple system made up of pure logic - something a computer should be good at and so:  let’s ask our friendly ChatGPT whether it knows the rules.

The rules

I will not present too many prompts and answers - this is boring, just try it yourself. I will only provide as much as necessary to make my point (see Conclusion, below).

ChatGPT Prompt, Question: Do you know the rules of the MIU system introduced by Hofstadter? Answer: Yes, I'm familiar with the MIU system introduced by Douglas Hofstadter in his book "Gödel, Escher, Bach: An Eternal Golden Braid." The MIU system is a formal system used to explore the nature of formal systems, the concept of provability, and the limits of formal proof. The MIU system consists of a single string of characters made up of the letters "M," "I," and "U." The system starts with the axiom "MI" and has three transformation rules: Rule M: If "x" is in the string, you can add "xU" to the end of the string. For example, if you have "MI," you can apply Rule M to get "MIU." Rule II: If "II" appears consecutively in the string, you can remove it. For example, if you have "MIIU," you can apply Rule II to get "MU." Rule U: If "U" appears in the string, you can replace it with "UU." For example, if you have "MIU," you can apply Rule U to get "MIUU." The goal in the MIU system is to start with the axiom "MI" and apply the transformation rules to derive the string "MU." If you can reach the string "MU," you can generate any number of "U" characters by repeatedly applying Rule U. However, if you reach a point where you cannot apply any of the transformation rules, you are stuck and cannot reach the desired string. The MIU system serves as an example of a formal system that exhibits both interesting properties and limitations. It demonstrates how certain rules can be used to manipulate strings and generate new strings, but also highlights the restrictions and limitations of a given set of rules.
Not quite. After some more arguing about rules, ChatGPT gave me the right rule set:

ChatGPT Prompt: Question: Just list the rules you are using at the moment. Answer: Rule one: If the string ends with "I," you can append "U." Rule four: If the string starts with "M," you can double the remaining part of the string. Rule two: You can replace "III" with "U." Rule three: You can delete a sequence of "UU." These are the rules we will apply to transform "MU" into "MI."

The question

Now, as we have the rules, it should be quite easy to request a simple transformation.
ChatGPT Prompt: Question: Now use above rules to transform "MI" into "MIUIU"; Answer: To transform "MI" into "MIUIU" using the given set of rules, we can follow these steps: Start with "MI" (axiom). Apply Rule one: Append "U" to the end of the string. Result: "MIU" Apply Rule two: Replace "III" with "U." Result: "MU" Apply Rule four: Double the remaining part of the string. Result: "MUU" Apply Rule one: Append "U" to the end of the string. Result: "MUUU" Apply Rule one: Append "U" to the end of the string. Result: "MUUUU" Apply Rule two: Replace "III" with "U." Result: "MUU" Apply Rule four: Double the remaining part of the string. Result: "MUUUU" Apply Rule one: Append "U" to the end of the string. Result: "MUUUUU" Now the string "MI" has been transformed into "MIUIU" using the given set of rules in the MIU system.
I mean, just in that answer, we can observe quite a number of "misunderstandings" but let's take it step by step:
  1. correct
  2. correct
  3. where does it find "III"?
  4. correct
  5. not correct - only possible if the last letter is "I"
  6. see 5
  7. no "III"
  8. correct
  9. not correct
  10. Result: apparently, MI == MUUUUU.
If you try it yourself you will observe:
  • mostly, you get correct results for simple, one-step transformations - sometimes even that fails
  • everything else is subject to creative machine-"thinking"

Conclusion (for me)

This is a very illustrative examples that ChatGPT (or other LLMs) cannot "think", that they apply rules without "understanding" and produces false output all continuously. Even simple tasks like comparing strings (MI == MUUUUU) seem to be a problem.

The restriction to only four simple rules makes it obvious: all answer do somehow "sound" correct but it can easily be shown in that simple system that the output is rather messy. 
Quite likely, this is also the case for more complex problems with way more rules - just that it is less obvious how to prove that we got presented questionable results.

Disclaimer

I've been using ChatGPT May 24 version. I'm not AI expert, ML expert, ... so I'm happy to learn about mistakes in my conclusion but what I'd really like to know: is it possible at all for an LLM like ChatGPT to "learn" the real rules of such formal systems?


Comments

Post a Comment