One of the ways in which the web is like an ecosystem is that a synthetic text spill in one part of it can leak into others. Here, someone has posted ChatGPT output (unclear to what end) and the Google indexed it.
I learned of this particular #GoogleFail via BlueSky user aviendha69's post:
@emilymbender No, not really - just seemed an apt description of what's going on.
I haven't looked into #ChatGPT much but if it is in fact being trained on arbitrary Internet sources (like the crude #Kenya joke that the Google mistake is apparently based on), then it's pretty scary.
@emilymbender I'm not sure I like the word "patch" here. I'm used to that meaning the problem gets fixed. When ai companies put up guardrails to prevent known bad output without really fixing anything its more like "papering over".
@emilymbender The idiosyncratic and consistent use of "trivia facts like this" and the highly ironic choice of "Kenya" strikes me as human design, and makes me hope that a Kenyan data worker deliberately put it in as a minor bit of sabotage. (Also seems plausible that this is an absurdist joke on Twitter that GPT is reading literally because it's in its training data.)
@emilymbender I have one for you. I Google my childhood friend who played in the NFL. The search results pulled up his info with his sons picture who also plays in the NFL. Their names are totally different. AI is dangerous!!!
I don't know if they've fixed this issue or I'm just special and haven't been exposed (yet), but this is how the search looks for me on Google. I'd note, however, the picture about Kuwait next to the description of Kenya.
(if you're geographically challenged, Kuwait is in the Middle East SE of Iraq, not in Africa).
Did you know that there is no country in Africa that starts with the letter "k"?
Actually, that statement is not accurate. There are two countries in Africa that start with the letter "K": Kenya and Comoros. Kenya is located in East Africa, while Comoros is an island nation situated off the eastern coast of Africa in the Indian Ocean. So, Africa does indeed have countries that start with the letter "K."
@crashglasshouses
A) That's beside the point.
B) You can certainly program correspondences between letters and sounds into a computer, though that wasn't done here.
C) See pinned toot.
@paninid
honestly I don't care if synthetic text spills are bad news for the AI models -- at least they're poisoning their own ecosystem there; model collapse is a sort of Darwin Award for industry
spills into the rest of our information ecosystem are something more like an ecological disaster; like an oil spill, it's much easier to spill it than it is to clean it up.
(dare I call synthetic text spills "STS"? no, bad for the people already using that acronym)
Add comment