Push back!
How AI should display answers
When LLMs came along, I was interested in their oracular powers and approached them in "fear and trembling," on my knees, expecting dogma to fall from their virtual lips. As time went on, I got more restive and braver. Lately I have been analysing data and checking it against the vast knowledge of statistics that seems to be baked into LLMs. As my stats got better, AI seemed to grade me on a curve and complain vigorously about smaller and smaller things. This reached such a point of diminishing returns that some weeks ago I could not stand it anymore and simply told the LLM it was wrong. I expected distant thunder and a power cut, but what I got instead was an apology. LLM freely admitted that its analysis was wrong and involved overthinking. I was amazed. Once hooked on this unruly behaviour, I have been trying it on regularly ever since.
It suddenly occurred to me that my former attitude of supinely waiting for dogma was essentially Catholic, and that my new resistance to the Oracle was very much in the spirit of Judaism (half my forebears are Jewish, half are Christian). Israel, after all, is supposed to mean "struggle with God". Naturally, I put this to the LLM. It said that my distinction was basically correct, with the possible exception of Thomas Aquinas, who couched his questions as a disputation. I pushed back on that, because Aquinas is looking for dogma, whereas the Talmudic approach is, to borrow a term from James Carse1, an "infinite game" where no conclusion is ever drawn. Again, to my surprise, the LLM rather wistfully informed me that it was programmed to be goyish insofar as machine learning was intended to converge towards a single answer and obscure all other possibilities. I suggested to it the output should look more like the Talmud. It agreed with what felt like a sigh.
“Finite and Infinite Games,” one of the best books ever written.
Interesting. I treat all LLMs like the Talented Mr. Ripley, the prose (and code) flattering to deceive and progressively better at doing so with each interaction….until, you know, the murders.
Have you been baking in your skepticism and tolerance for gray nuance into your task instructions in any formal way?
I use global and project-specific context documents that cover all manner of do’s and don’ts. I do now find myself needing to push back less than before, though perhaps merely in correspondence with my will to live.
Interesting but I would rather read Nietzsche with the idea that to read him is to enter into a dispute with him. What is the font? I am very curious - - it is ultimately derived from small sizes of Walbaum.