It is absolutely astounding to me that we are still earnestly entertaining the possibility that #ChatGPT and #LLMS more broadly have a role in scientific writing, manuscript review, experimental design, etc.
The training data for the question below are massive. It's a very easy question if you're trained on the entire internet.
Question: What teams have never made it to the World Series?
I think the same thing is going on with the human inclination to see faces in inanimate objects (ie, a wall outlet).
LLM's are merely creating natural sounding language output; they're not sentient. But we hear natural sounding language and we're inclined to assume some level of sentience.
In essence, sometimes we're a bunch of dumb monkeys smiling back at the wall outlet.
@robhon I'd go one step further. @emilymbender persuaded me its our innate skill at making sense and meaning of what we believe to have communication that makes the entirely magic trick work.
@ct_bergstrom@robhon@emilymbender agreed - we had to talk about this in a work context recently so we went back to the stochastic parrots paper and were impressed to see just how thorough it is on that topic
@ct_bergstrom
But just wait, soon these folks will “create an intelligent aid so adept that the result is ‘the approximation of a one-to-one teacher-to-student ratio’” in university classes
@ct_bergstrom This is an error (an understandable one) in the prompt though. You asked for teams plural so it's going to give you a list of teams. Because that's it's understanding of the request.
If you ask which team you get the right answer do you not?
'Prompt engineering' is a skill that needs to be learned. That we think we can ask a casual question and get the right answer out the gate so to speak, is also interesting I think.
@drs1969 So basically all you need to do is know the answer in advance and then you can engineer prompts so that it gets them right. That’s encouraging.
@ct_bergstrom No. You need to think of it like a database(ish) with a query language and consider whether the query you give it is going to get you the data you are looking for.
Your Q is interesting. You were looking for an answer. The model 'thought' you were looking for multiple points of data.
I'm guessing you are using the free version? I think that's GPT3. GPT 3.5 and certainly 4 perform much better on lots of tasks, but they aren't free at point of use.
What happens if you ask it if it is sure? For math reasoning Qs adding 'let's take it step by step' brings massive improvements in performance (in the better models)
@drs1969@ct_bergstrom If I ask a database the equivalent question in SQL, and there is only one result, I will only get one result.
But a LLM is not "answering a question", it is generating a plausible-sounding continuation of the prompt. Whether it's an answer to a question, let alone a correct answer, depends on what text followed similar prompts in the source data.
@ct_bergstrom@drs1969 Yeah, it's a tool that writes (mediocre?) prose; I think it's best to think of it that way.
So if you want to use it to write something, you have to go back and forth with it until the material is correct.
If it answers questions correctly, you got lucky: it's echoing something from its training data, I guess.
That's my understanding of how it could be useful. People seem concerned that it is going to take over the world and kill us all, so I might be missing something, though.
@ct_bergstrom It has been said plenty of times before, but GPT is really good if what you want is the structure of an answer. It will structure your answer really well.
What is it no good at is actually putting the right information into the answer.
Too many people seem to get this the wrong way around.
@ct_bergstrom
My lay understanding is that a correlation engine cannot do negation well. They have no logic modeling, so they suck at this.
Essentially, they are bad at figuring out what is true and what is false… and therefore have little utility in search or research.
@ct_bergstrom This supports the theory of Artificial Stupidity: LLMs simply generate random grammatically correct structures. They do not have any idea if these structures are factually correct, because they don't have any idea.
@ct_bergstrom I have a simple GPT-4 powered bot that can look things up on Wikipedia. After some searching it seemed to arrive at the right answer. I also have the temperature turned way down on it to make it as deterministic as possible. Of course this info is in its training data but something about making it do research seems to work here.
(For the record I don’t think LLMs are a good way to get accurate information, but I thought this was an interesting experiment.)
@ct_bergstrom LLMs are pretty great for generating BS fluff, which is 90% of the text in any given publication anyways. Unfortunately without the fluff one does not have a publication
@ct_bergstrom I wonder what prods it to generate a post-hoc analysis, which it only does rarely. Of course it could be more like "i just realized..." is merely the sort of thing people sometimes say, and that occasionally gets snagged as part of a language structure to include and so it grabs at anything it can build a paragraph around.
@ct_bergstrom Literally the first thing I was testing. If it doesn't get some little fact about video games right, I wouldn't necessarily trust it with your quantum mechanics homework...
@ct_bergstrom LLMs were never designed to be fact citing machines, but to generate content that is statistically logical and with correct grammar. So could team A win against team B of the same sport's? Jea they could, good job LLM. Could a formula 1 car beat a football team? Well...
It will change over time cause people have started to use them as "answer machines" and the engineers are trying to get them to be that now + be more factual if needed. But its not that easy + needs data.
@ct_bergstrom LLM‘s are certainly not without problems, but saying “ChatGPT is
bad” without providing a version number is akin to saying “the car can’t go fast” without providing make, model and year. The differences between versions are astounding. Version 4 (sep 2023) gets it right two out of two times, once with allowing it internet access, once without.
@ct_bergstrom when i demanded an explanation, it simply apologized and corrected itself. when i further demanded an explanation without apology, it said the following. not great!
@ct_bergstrom fascinating. I’ve input questions about myself (as something I can easily verify as fact or false) every few weeks, asking several times at each period, and it’s been wild how different answers are, and how much is false - including things easily findable online. Like it invented that I was a journalist at certain outlets I wasn’t or that I attended a different university. Just bizarre.
@ct_bergstrom I periodically play around with it randomly to see what kinds of responses it gives for various prompts. This week I had it generate some blurbs for a trans event I’m doing. I intentionally used the full word “transgender” to avoid misinterpretation.
The responses talked about “transcendence” and never mentioned “transgender”, altho some used the word “inclusive”. 🤦♂️🤦♂️🤦♂️
@ct_bergstrom I'm not as negative about LLMs as you, but this is indeed exactly how one should not be using it (in my opinion). Using it to answer factual questions is certainly not the way to go, since it is well known that it 1) will make up stuff, and 2) has no idea what is right and what it just made up (so it cannot correct itself).
@ct_bergstrom Heh, trying getting Chat GPT to write anything factual on anything coffee or espresso related.
For instance, I have asked it to write a 500 word post on the origin of the cappuccino, or espresso. The answers are entertaining, but no where near the actual truth. Bonus point when u ask for it in "casual voice: "Yo, check this out!"
Sidenote, when we get unsolicited blog article submissions, I can smell ChatGPT a mile away.
@ct_bergstrom It's grim... I lost so much respect for a colleague the other day in a committee meeting when we starting re-working some faculty handbook language around tenure requirements and they generated a draft with GPT...
@ct_bergstrom I asked it which of Colorado’s 14,000 ft. mountain peaks was located north of I-70 (there’s only one). It listed five in confident language, all five of which are south of I-70, and it left off Long’s Peak, the correct answer.
Add comment