I was curious if a niche blog post of mine had been slurped up by #ChatGPT so I asked a leading question—what I discovered is much worse. So far, it has told me:
• use apt-get on Endless OS
• preview a Jekyll site locally by opening files w/a web browser (w/o building)
• install several non-existent #Flatpak “packages” & extensions
It feels exactly like chatting w/someone talking out of their ass but trying to sound authoritative. #LLMs need to learn to say, “I don’t know.”
@cassidy "#LLMs need to learn to say, 'I don’t know.'"
Doing that properly might require... something that isn't an LLM. I'd say the LLM generates something that (statistically) looks like an answer, because that's what its trained to do.
Actually modeling some understanding of truth and knowledge might be a different and more difficult task than modeling language.
@ids1024 yeah, fair point. Which is why I try to constantly use “LLM” instead of “AI,” because people seem to miss the “artificial” part of artificial intelligence. It’s artificial in that it is not intelligent!
This race to use LLMs for everything is so misguided; LLMs can be super cool for very specific things like summarizing a long text, typing suggestions, describing images, etc. but I genuinely think that chat model is just a terrible idea that needs to die.
@cassidy I consider all LLMs to be like a new, enthusiastic intern on the team. I'm unlikely take anything they say verbatim, but they certainly may know more about some things than I do!
I guess you tested on ChatGPT3.x (the free one) because the 4.x (paid) ones don't appear to exhibit those particular lies - but may produce other lies that are more or less interesting. 🤷
4o in fact is pretty good with how to install software on Endless. I guess iterating on the model and newer data to blame
Each time I would correct it or provide the error I would get when running obviously-incorrect commands, it would output something like, “It seems there might have been a misunderstanding…” and then gaslight me into thinking I did something wrong, instead of ChatGPT just outputting pure bullshit.
Again, exactly like chatting with someone on the Internet who has no idea what they’re talking about but is trying to hide it.
Add comment