#AI#GenerativeAI#Hallucinations: "I want to be very clear: I am a cis woman and do not have a beard. But if I type “show me a picture of Alex Cranz” into the prompt window, Meta AI inevitably returns images of very pretty dark-haired men with beards. I am only some of those things!
Meta AI isn’t the only one to struggle with the minutiae of The Verge’s masthead. ChatGPT told me yesterday I don’t work at The Verge. Google’s Gemini didn’t know who I was (fair), but after telling me Nilay Patel was a founder of The Verge, it then apologized and corrected itself, saying he was not. (I assure you he was.)
When you ask these bots about things that actually matter they mess up, too. Meta’s 2022 launch of Galactica was so bad the company took the AI down after three days. Earlier this year, ChatGPT had a spell and started spouting absolute nonsense, but it also regularly makes up case law, leading to multiple lawyers getting into hot water with the courts.
The AI keeps screwing up because these computers are stupid. Extraordinary in their abilities and astonishing in their dimwittedness. I cannot get excited about the next turn in the AI revolution because that turn is into a place where computers cannot consistently maintain accuracy about even minor things."
#AI#GenerativeAI#LLMs#AITraining#Hallucinations#AITraining: "Models like ChatGPT and Claude are deeply dependent on training data to improve their outputs, and their very existence is actively impeding the creation of the very thing they need to survive. While publishers like Axel Springer have cut deals to license their companies' data to ChatGPT for training purposes, this money isn't flowing to the writers that create the content that OpenAI and Anthropic need to grow their models much further. It's also worth considering that these AI companies may already have already trained on this data. The Times sued OpenAI late last year for training itself on "millions" of articles, and I'd bet money that ChatGPT was trained on multiple Axel Springer publications along with anything else it could find publicly-available on the web.
This is one of many near-impossible challenges for an AI industry that's yet to prove its necessity. While one could theoretically make bigger, more powerful chips (I'll get to that later), AI companies face a kafkaesque bind where they can't improve a tool for automating the creation of content without human beings creating more content than they've ever created before. Paying publishers to license their content doesn't actually fix the problem, because it doesn't increase the amount of content that they create, but rather helps line the pockets of executives and shareholders. Ironically, OpenAI's best hope for survival would be to fund as many news outlets as possible and directly incentivize them to do in-depth reporting, rather than proliferating a tech that unquestionably harms the media industry." https://www.wheresyoured.at/bubble-trouble/
#AI#GenerativeAI#LLMs#Automation#Hallucinations: "The only reason bosses want to buy robots is to fire humans and lower their costs. That's why "AI art" is such a pisser. There are plenty of harmless ways to automate art production with software – everything from a "healing brush" in Photoshop to deepfake tools that let a video-editor alter the eye-lines of all the extras in a scene to shift the focus. A graphic novelist who models a room in The Sims and then moves the camera around to get traceable geometry for different angles is a centaur – they are genuinely offloading some finicky drudgework onto a robot that is perfectly attentive and vigilant.
But the pitch from "AI art" companies is "fire your graphic artists and replace them with botshit." They're pitching a world where the robots get to do all the creative stuff (badly) and humans have to work at a robotic pace, with robotic vigilance, in order to catch the mistakes that the robots make at superhuman speed.
"In a world of digital creation, I sing my song of light
But lurking in the shadows, a tale of endless night
Generative AIs, they steal from artists' hearts
Their creativity taken, ripped apart"
"In a pixelated world, where bits collide
Hallucinations dance in 8-bit lullabies
AI models leaping, their guard rails untried
Spewing hate speech, casting shadows in the skies"
#AI#GenerativeAI#Search#SearchEngines#LLMs#Hallucinations: "A couple of days ago, Wharton professor Ethan Mollick, who studies the effects of AI and often writes about his own uses of it, summarized (on X) something that has become clear over the past year: “To most users, it isn't clear that LLMs don't work like search engines. This can lead to real issues when using them for vital, changing information. Frontier models make less mistakes, but they still make them. Companies need to do more to address users being misled by LLMs.”
#AI#GenerativeAI#Hallucinations#Disinformation#Misinformation#Politics#Elections: "From tech companies, we need more than just pledges to keep chatbot hallucinations away from our elections. Companies should be more transparent by publicly disclosing information about vulnerabilities in their products and sharing evidence of how they are doing so by performing regular testing.
Until then, our limited review suggests that voters should probably steer clear of AI models for voting information. Voters should instead turn to local and state elections offices for reliable information about how and where they can cast their ballots. Elections officials should follow the model of Michigan Secretary of State Jocelyn Benson who, ahead of that state’s Democratic primary election, warned that “misinformation and the ability for voters to be confused or lied to or fooled,” was the paramount threat this year.
With hundreds of AI companies sprouting up, let’s make them compete on the accuracy of their products, rather than just on hype. Our democracy depends on it."
Nice paper about the inevitability of hallucinations in LLMs with some nice and simple empirical experiments. For those TLDR people,
"All LLMs will hallucinate.
Without guardrail and fences" "LLMs cannot be used for critical decision making."
"Without human control, LLMs cannot be used automatically in any safety-critical decision-making."
The authors make the relevant point that this does not make LLMs worthless. https://arxiv.org/pdf/2401.11817.pdf #AI#LLM#hallucinations#generativeAI
“If #hallucinations aren’t fixable, #generativeAI probably isn’t going to make a trillion dollars a year. And if it probably isn’t going to make a trillion dollars a year, it probably isn’t going to have the impact people seem to be expecting. And if it isn’t going to have that impact, maybe we should not be building our world around the premise that it is” @garymarcus
And it’s not just a one-off, it’s constant. When the training set does not contain the inference needed, the system hallucinates it.
With recommendations, the only training parameter is engagement. With generic #LLM’s the parameters are found by the system independently, but the same problem generalises across.
So… People put stuff in their robots.txt to “prevent” malicious scraping of their data for machine learning purposes. I hope everybody understands that this is just a “please don’t take my data” sign on the front lawn. We should be creating heaps of adversarial data instead. Data suitable to taint those datasets.
Do you know it’s in the public domain now, but you didn’t know it a few seconds ago?
Why do I sound like the most obnoxious defense attorney on Law & Order?
That quote feels a little more promising – especially “it is always a contemporary emotion that we experience” and also “a snapshot that has survived and which we had not suspected of having taken”.
It’s weird that my memory feels so vague, and I have no idea where in the book that could have been, even though I reread Vol 2 in June.
Why is the phrase “c’est toujours une émotion contemporaine que nous en éprouvons” getting zero hits on Google and on Google Books?
Why does À l’ombre have to be divided into three separate files in the free web edition?
#AI#GenerativeAI#ChatGPT#Hallucinations: "Although chatbots such as ChatGPT can facilitate cost-effective text generation and editing, factually incorrect responses (hallucinations) limit their utility. This study evaluates one particular type of hallucination: fabricated bibliographic citations that do not represent actual scholarly works. We used ChatGPT-3.5 and ChatGPT-4 to produce short literature reviews on 42 multidisciplinary topics, compiling data on the 636 bibliographic citations (references) found in the 84 papers. We then searched multiple databases and websites to determine the prevalence of fabricated citations, to identify errors in the citations to non-fabricated papers, and to evaluate adherence to APA citation format. Within this set of documents, 55% of the GPT-3.5 citations but just 18% of the GPT-4 citations are fabricated. Likewise, 43% of the real (non-fabricated) GPT-3.5 citations but just 24% of the real GPT-4 citations include substantive citation errors. Although GPT-4 is a major improvement over GPT-3.5, problems remain."
By and large, after many, many years of dreams during sleep, I have come to the conclusion that they are probably just artifacts from neural management functions (sorting, retrieval, merging and storing, garbage collection, etc.), and have no major significance in and of themselves.
Artificial neural networks connect disparate bits of information that could be plausibly connected, which we view as a hallucination.
While we’re dreaming, our organic neural network connects disparate bits of information which could be plausibly connected, but are not necessary or helpful, so it flushes them out via dreams, as sleeping hallucinations.
« If #ChatGPT is fabricating code libraries (packages), attackers could use these #hallucinations to spread malicious packages without using familiar techniques like typosquatting or masquerading.
Those techniques are suspicious and already detectable. But if an attacker can create a package to replace the “fake” packages recommended by ChatGPT, they might be able to get a victim to download and use it. »