it’s a little disingenuous to refer to #LLMs as #opensource because you can really only open source an LLM in roughly the same way you open source a microprocessor — RISCV is open source, the plans for it anyway, but it still costs millions to riff off it and make your own custom version, same with LLMs. that’s not exactly what open source was going for
Okay, now this is important. Drop everything you're doing, because Tom7 has released a new video. And oh boy, is it wonderful. 20 minutes won't save your life - and this video demands you sacrifice that time. Now!
I see that openai.com/gptbot is crawling my blog, top to bottom, side to side. I’m sure OpenAI has consulted the “Rights” link clearly displayed on every page, invoking a Creative Commons license that freely grants rights to reuse and remix but not for commercial purposes.
one thing i love about #LLMs is asking it “how tf do i do X” and it responds with 5 ideas, four of which are terrible but one is far better than anything i’d thought of. or their all terrible but one makes me realize i’ve been thinking about the problem wrong
"In [the counterfactual task] paradigm, models are evaluated on pairs of tasks that require the same types of abstraction and reasoning, but for each pair, the content of the first task is likely to be similar to training data, whereas the content of the second task (a “counterfactual task”) is designed to be unlikely to be similar to training data." -- #MelanieMitchell
Today, I took ChatGPT's Data Analyst for a spin. You've probably seen the fancy advertising videos: just drop in a dataset and AI does all the analysis for you?! Let's see ...
"the worst part is that when they [#LLMs] can’t complete a task confidently, they don’t give you an error or tell you they’re unable to finish. They make something up and serve you incorrect information.
…companies like #Google are pretending this isn’t a problem and pushing these systems toward taking over as our phones’ virtual assistants and the brains behind our online searches."
It's fashionable to criticize #LLMs, but can you think of another human invention that allows us to spend the energy budget of Tanzania to lift shitposts out of context and present them as if they were authoritative knowledge?
i’m very excited about the interpretability work that #anthropic has been doing with #LLMs.
in this paper, they used classical machine learning algorithms to discover concepts. if a concept like “golden gate bridge” is present in the text, then they discover the associated pattern of neuron activations.
this means that you can monitor LLM responses for concepts and behaviors, like “illicit behavior” or “fart jokes”
💭 Dreaming of #OpenWebSearch in Europe
👉 The German Science Journal „Spektrum.de“ writes about the OWS.EU project & the challenge of creating a European #OpenWebIndex as a foundation for #WebSearch, #LLMs & special interest applications.
„So far, 1.3 billion URLs in 185 languages, totaling 60 terabytes, have been crawled and indexed“ says project lead Michael Granitzer in the article.
Find out more about potential future applications & OWS.EU´s unique approach:
if i had more time, i'd love to investigate PII coming from #LLMs. i've seen it generate phone numbers and secrets, but i wonder if these are real or not. i imagine you could look at the logits to figure out if phone number digits were randomly chosen or if the sequence is meaningful to the LLM. anyone aware of researchers who have already done this?
So… Big Tech is allowed to blatantly steal the work, styles and therewith the job opportunities of thousands of artists and writers without being reprimanded, but it takes similarity to the voice of a famous actor to spark public outrage about AI. 🤔
I was curious if a niche blog post of mine had been slurped up by #ChatGPT so I asked a leading question—what I discovered is much worse. So far, it has told me:
• use apt-get on Endless OS
• preview a Jekyll site locally by opening files w/a web browser (w/o building)
• install several non-existent #Flatpak “packages” & extensions
It feels exactly like chatting w/someone talking out of their ass but trying to sound authoritative. #LLMs need to learn to say, “I don’t know.”
Saying "LLMs will eventually do every job" is a bit like:
Seeing Wifi wireless data
Then predicting "Wireless" Power saws (no electrical cord or battery) are just around the corner
It's a misapplication of the tech. You need to understand how #LLMs work and extrapolate that capability. It's all text people. Summarizing, collating, template matching. All fair game. But stray outside of that box and things get much harder.
Nice example of how important emphasis can be for language understanding. Depending on which word in the sentence below is emphasized, it completely changes its meaning.
For #LLMs (and for our #ise2024 lecture) this means that learning to understand language purely from written text is probably not an "easy" task....