"It’s increasingly looking like this may be one of the most hilariously inappropriate applications of AI that we’ve seen yet." I am riveted by the extensive documentation of how ChatGPT-powered Bing is now completely unhinged. @simon has chronicled it beautifully here: https://simonwillison.net/2023/Feb/15/bing/
@HistoPol@annaleen the original ChatGPT turned out to be a lot less prone to wild vengeful outbursts than whatever model it was that they plugged into Bing - it's a pretty interesting demo of how well the safety features of ChatGPT (which Bing seems not to have) have held up
Thanks for this additional piece of information, Simon.
It reminded me that I had wanted to add a word in my toot: indelibly.
As any #SciFi aficionado will tell you:
there should be a built-in self-destruct mechanism when tampering with these Laws or copying or moving the #AI to another system.
Another classic movie comes to mind in this respect, #Wargames...
@HistoPol@simon@annaleen that 10% is a survey result, the survey provides no information about how any respondents chose their responses, so it’s not possible to assess the methodology they used.
I want a “working definition” we could use to decide that something isn’t a GI, or is a GI. Maybe first it has to be able to do more than one thing - LLMs can’t do anything other than words, so LLMs are not GIs. But that’s very incomplete
@ShadSterling, you might just contact the authors Zach Stein-Perlman, Benjamin Weinstein-Raun, Katja Grace, about their methodology used for evaluating the questionnaire.
They do have a feedbackbox on the linked page.
Regarding the definition,
a) I am not a computer scientist and therefore, for my ends, do not strive to surpass the level if science journalists of #TIME and #TheEconomist.
...in disagreement about the definition and the terminology.
E.g. in this @reuters article, they state that #LLM's are a form of #GenerativeArtificialIntelligence (#GAI), while also starting that "Like other forms of artificial intelligence, generative AI learns how to take actions from past data. "
...they use advanced prediction/statistical models and might show some #nascent (or just inexplicable?) form of "#intelligence," but that’s about it.
As I have posted elsewhere this week, most recent
research seems to indicate that #SentientBeings/#intelligences need a corporal form #embodiment) to really comprehend language.
This is why I find the combination of #robotics and #GAI so dangerous, as I'm quite...
If we look at the international situation today: #ClimateCrisis, wars, one small parties of humanity living relatively well in a #PostColonial world order, it would take any #GAI I've read about in #SciFi but a split second to determine what's the root of the problem: #humanity.
And then, no...
In fact, I don't even think human programmers will be smart enough to prevent this. Even today; they don't understand all of the code and already the machined are writing thousands of lines of code every day.
TBH, I think, if this were a movie, I'd stop watching it, as the ending is just a dead giveaway.
@HistoPol@reuters@simon@annaleen in this thread you’ve said both “generative artificial intelligence” and “general artificial intelligence”; I would avoid the latter and use exclusively “artificial general intelligence”
🧵
@ShadSterling
This is a problem everyone is having. I have been thinking about that, too.
Present talk is just about #GAI in the form of #LLM's.
To make an informed choice, I'd need to know what kind of #GAI wpäould not be #LMM's?
Are there present-day applications?
If not, better forget about #GenerativeArtificialIntelligence as a term. Too easily confused with #AGI of the "General" kind. 😉
@HistoPol@reuters@simon@annaleen All we really know about AGI is that we don’t know how to create one, and our inability to agree on a definition illustrates how far we are from figuring that out. That 10% figure isn’t a measure of what an AGI would do but a measure of what some people who don’t know how to create an AGI think one might do. It’s about as credible as people in the 1700s speculating about how aircraft might work.
🧵
WHAT IS THE DIFFERENCE BETWEEN ROBERT OPPENHEIMER AND GEOFFREY HINTON?
Hm, that is a valid point.
On the other hand, I don't know how to build a car or even know how it functions in some detail. I do not need to know how to build one. I can still drive it.
A vast majority of educated people rightly asks why #RobertOppenheimer didn't stop the #ManhattanProject. He must have known at some point before it was too late.
@HistoPol@HistoPol@simon after reading the work of @DAIR, @timnitGebru, @emilymbender et. al. I can’t take Hinton seriously. He’s mostly hyping speculative future risks, at the expense of mitigating actual harms already inflicted by existing automations.
This time, the "Oppenheimer" of the #AI project DID quit:
#GeoffreyHinton "A man widely seen as the godfather of artificial intelligence (AI) has quit his job, warning about the growing dangers from developments in the field."
"...in a statement to the #NewYorkTimes, saying he now regretted his work.”
“He told the #BBC some of the dangers of #AI#chatbots were "quite scary".
"...person in the amount of general knowledge it has and it eclipses them by a long way.
In terms of reasoning, it's not as good, but it does already do simple reasoning," Dr [#Hinton] said."
"And given the rate of progress, we expect things to get better quite fast. So we need to worry about that."
@HistoPol@simon@annaleen@BBCWorld LLMs don’t do reasoning, they only do language. People can be confused by that because language is how people share their reasoning, but LLMs generate word-sequences based on a statistical model of how words are sequences, without anything resembling reasoning. When someone who knows better suggests otherwise I can’t treat them as trustworthy
"...for example, some bad actor like [Russian President Vladimir] #Putin decided to give robots the ability to create their own sub-goals."
"...digital systems...can learn separately but share their knowledge instantly. So it's as if you had 10,000 people and whenever one person learnt something, everybody automatically knew it..."
"#Yoshua#Bengio, another so-called godfather of #AI,... wrote that it..."
@HistoPol@simon@annaleen@BBCWorld controlling a robot body and making goals are both outside the scope of LMs - neither is related to language
Digital systems don’t have knowledge, in the way a person does, and so far their “learning” is something that takes more computing resources than a robot would have on board
...check the prerequisites for #autopoietic systems (as developed by #Varela), we might reach the conclusion that #GAI is already pretty close to #Autopoiesis.
(Note:The late #Niklas#Luhmann is one of the most difficult authors to read an is chiefly available in #German; I haven't studied him in a long time, so I cannot apply his whole concept to #AI, of which I also do not enough of).
@HistoPol@simon@annaleen@BBCWorld a static model built by a human-directed “machine learning” process doesn’t grow other than as directed, and is certainly not self-organizing
Machine learning begins with taking statistical regression and simplifying it so you can build a model using far more data than is practical with full regression. Regression is a generalization of curve-fitting, finding a mathematical function that fits some given data as well as possible
Yes, that was the beginning.
And the more I read, the more I am convinced that we will not have to wait for the next decade to see #autopoietic systems capable of self-reference and -development.
My gut feeling from what I have read (and studied of the former) natural-language learning and how infants learn to "grasp reality", is that this will happen very swiftly when we let even only a couple of #robots, connected...
Alas, my knowledge of all the exciting fields involved is only rudimentary, however I am certain that by having created #NeuralNetworks capable if learning, providing more stimuli than any biological system ever experienced its...
@HistoPol@simon@annaleen@BBCWorld I’m not sure why you’re linking back to an earlier entry in this same thread, but while it’s a worthy research topic it’s not clear that embodiment is key. I haven’t seen anything that uses ML and has any capacity to handle interruptions, and without being able to handle interruptions trying to control a body is not going to go well
@ShadSterling
Sorry, I hadn't realized when I wrote it that it was the same thread. (The fast app I am using on my smartphone is not really good at portraying threads.)
How do you define "interruptions" in this respect?
@ShadSterling
...surpass the capabilities of the insurgents in #IRobot within the next twelve months.
And in combination with with the command and control room structure product announced this week by #Thiel's #Palantir we can expect the first robots wars within this decade:
@HistoPol@voron@simon@annaleen@BBCWorld militaries have owned the best in secure communication for thousands of years, it hasn’t changed recently, and securing military communication is why computers happened when they did. The issue for AI swarming is not security, it’s variety - really independent learning machines can’t just copy each others brains without adaptation, because both their minds and their bodies will have zillions of differences
@HistoPol@voron@simon@annaleen@BBCWorld I don’t know how to define interruptions as generally as I’m thinking if it, but things like stepping on something that moves as it takes your weight; you have to change how your stepping while you’re doing it, effectively updating that corner of your model of reality, and your plan for how to step, in real-time while stepping. The dance videos don’t illustrate anything like that, they’re just pre-programmed
@ShadSterling
Ah, I understand now, also, something like oncoming traffic that suddenly veers into your lane?
I had not seen them in this perspective, will keep that in mind for the future.
Yes, "interruptions" are a key aspect of RL interactions, such as #warfare in particular.
@HistoPol@voron@simon@annaleen@BBCWorld that’s maybe not a great example because Boston Dynamics and others are working on related things, e.g. with their walking all-terrain carts, but they’re very narrowly scoped because even that is pushing the limits of what we can make; think of how a person responds to being bit by an unseen bug, or seeing motion from the corner of their eye, or interrupting a conversation when someone your looking for appears
@HistoPol@voron@simon@annaleen@BBCWorld we can make machines with some level of dynamic aiming, like glorified traction control, but we can’t make control systems with the kind of continuous dynamic situational awareness that a gymnast needs to safely abort after a slip. Even SpaceX rockets can’t do simple things like change which single engine is used for the landing if the planned engine fails. Even traction control has only recently gone digital
@ShadSterling
Very interesting. I wasn't aware of these engineering limitations.
What if an #AI remote-controlled these #robots? (a.i. a lot more parallel computing power .)
E.g. I think analysiert millions of, say, street-accident reports, there aren't infinite possibilities of "disruptions".
@HistoPol@voron@simon@annaleen@BBCWorld if we want to compare dancing robots to human intelligence, we need robots with the capacity not only to improvise on a floor with inconsistent traction, but do so playing off eachother, and use that as a social negotiation to gain trust. And not only can’t we make such robots, the closest we can get requires tremendous weight in batteries and an offboard datacenter
@HistoPol@simon@annaleen@BBCWorld Neural networks in machine learning are in the “glorified regression” family; you can think of regression as computing the coefficients on the complete graph of every possible way that part of the input can influence part of the output, and neural networks as hand-pruning that graph to only include the connections you think (or estimate) are important. “Learning” is part of how they’re made, not of what they do
@HistoPol@simon@annaleen@BBCWorld you’ve got more sensory input in your little finger than any AI has in its entire model, trying to train a robot on the life experience of a mouse is far more than we can give it the capacity to handle. That kind of advancement is far beyond the horizon from what we can do today
However, I can see a temporary replacement for this, too: the myriads of input from the #IoT combined with the parallel instant knowledge capabilities described earlier and the use of advanced mathematical models (regression etc.) to fill in the gaps.
@HistoPol@simon@annaleen@BBCWorld as in the other branch, shared “knowledge” is limited, and far from instant; much more so on IoT devices, with their tiny capacities. Computing any mathematical model on IoT devices is a hard and unsolved problem; in the volcano example, the goal is to model lava flows, which boils down to computational linear algebra, which is hard to distribute. Even centralizing the data for the analysis is hard: ⤵️
@HistoPol@simon@annaleen@BBCWorld if you mean the kind of self-development a person can do, we first need to develop a way to make software than can reason, and remember; two things none of our existing ML methods can include in their creations. And even when/if we do develop such a method, I we don’t know that it would be capable of developing any faster than a human baby
@HistoPol@simon@annaleen@BBCWorld no it isn’t. The current problems with AI come from it being far less capable than the hype suggests, but being used carelessly despite its limitations. Like law enforcement using facial recognition (which is made with ML, tho not called AI) even though it’s unreliable, and especially unreliable with non-white faces. We already overprosecute non-white people, this use of AI adds to that
@HistoPol@simon@annaleen@BBCWorld as I understand it, specialized model-generating methods have been able to create models that perform well at specific tasks, but have not escaped the fundamental limitation that the model is only a way to map inputs (“prompts”) to outputs (usually some recognition or generation). That kind of model is no more capable of reasoning than the much simpler functions seen in ordinary algebra, they just have large inputs & outputs
…writes the following (p.6), which, though the article is not intended for an application in #ArtificialIntelligence , IMO would explain why LLM’s will reach a learning frontier that only embodiment, i.e. #robotics, will solve:
“On the one..."
"...hand, this leads to the statements that "an operationally closed system cannot reach the environment with its own operations" (Luhmann 1997b: 129), that there is "no environmental contact" (Luhmann 1997b: 92) and "no penetration into the environment" of observing systems at the level of their operation. #Society is defined as a "communicatively closed system" (Luhmann 1997b: 95), whose dynamics consist in the..."
@HistoPol@simon@annaleen@BBCWorld@jackcole “That said, we do emphasize that the outputs from ChatGPT are not meant to be deployed directly on robots without careful analysis” - for coding in general, that careful analysis is more time consuming than than writing the code yourself
Just as a teaser: unquestionably, most of the world's endangered species could be rescued if the #HomoSapiens were no longer at the top of the #FoodChain...
No Zeroth Law, and a #Bing-empowered, freed #ChatGPT could quickly arrive at this conclusion...
...I am not sure if "The Fifth Law of Robotics" by Nikola #Kesarovski,
"A robot must know it is a robot" (also a book title*), really can be a viable solution to this problem. We all know how the concept of #slavery turned out for humanity: to this day, it suffers from this crime.
..., self-preservation certainly is a defendable concept in the #evolutionary process, so I'd like to propose an alternative
6th #LawOfRobotics (s/:for which I might be hunted down by the presumed #superintelligence some day, #Terminator style./s):
"An artificial intelligence, even if it is biological, must always have an #autodestruct mechanism which it cannot deactivate."
In other words, humanity must always be able to "pull the plug"....
...This said, I might also as well say that I see little chance for this happening, the globe being ruled by #oligarchs following the principal of #plutocracy and #capitalism (no, I am not a #Marxist;)) and #autocrats
Even a non-#superintelligence with access to the sensors of the #IoT will easily be aware of any threat to its existence and will find ways to circumvent the #LawsOfRobotics.
...it'll have human #bias. Humans have always been great at bending or breaking the law when it suited their interests. How could a #Superintelligence created with human values not arrive at the same, self-preserving conclusion?
A gloomy, yet, IMO, quite fitting assessment of the shape of things to come unless there's a #Chernobyl-style "fallout" before #GAI evolves into #AGI + humanity gets its act together and, as Prof. #Tegmark admonishes: "Just look up!"
I did not know the game but, yes, some speedreading helped, you are right, except for two things:
it will not be humerous but rather like #PKD's prescient #SecondVariety setting and
the "Computer's arbitrary, contradictory and often nonsensical security directives" are a non-issue. They will be like moves in an n-dimensional chess game, being (re)calculated with super-human speed...
"Recent research has taken this approach, training language models [#LLM's] to generate physics simulations, interact with physical environments and even generate #robotic action plans.
Embodied language understanding might still be a long way off, but these kinds of multisensory interactive projects are crucial steps on the way there."
...corrected the colonel's rather detailed account twice, eventually claiming that it had only been a "thought experiment." This, however, seems still unlikely to me even after the long discussion we had below.
This ongoing [#AGI thread continues with 12/n with a completely new approach about educating #AI:
...operator."
This is a #PR fiasco the colonel caused.
Some times, the original story pans out.
Besides, now that an #LLM can be set up on a gaming #PC for as little as 100 USD, and gamers being what they are, I'm sure we will not have to wait another year for corroboration, alas.
@HistoPol@heiseonline I followed it pretty closely. It's clear to me what happened: the speaker at the conference was uncareful with the way they described the thought exercise, it was reported on a blog, and that coverage fitted the exact narrative people were looking for like a glove and went wildly viral
However, the retraction statement is not even #marcom anymore, but comes right of a #PR crisis management desk, IMO.
Here's the full text:
"[UPDATE 2/6/23 - in communication with AEROSPACE - Col Hamilton admits he "mis-spoke" in his presentation at the Royal Aeronautical Society FCAS Summit and the 'rogue AI drone simulation' was..."
"...a 👉hypothetical "thought experiment" from outside the military👈, based on 👉plausible scenarios👈 and likely outcomes rather than an actual USAF real-world simulation saying: "We've never run that experiment, 👉nor would we need to in order to realise that this is a plausible outcome"👈. He clarifies that the #USAF has not tested any weaponised #AI in this way 👉(real or simulated)👈 and says "Despite this being a..."
"...👉hypothetical example👈, this illustrates the real-world challenges posed by AI-powered capability and is why the #AirForce is committed to the 👉ethical development of #AI".]👈
The last statement: "ethical" wespons development?!?
In competition with #China? The #US military, who has been proven to use #GI's as guinea pigs? If you please!
"...much on #AI noting how 👉easy it is to trick and deceive.👈 It also creates highly unexpected strategies to achieve its goal.
He notes that 👉one simulated test saw 👈 an AI-enabled drone tasked with a #SEAD mission to identify and destroy SAM sites, with the final go/no go given by the human.
However, having been ‘reinforced’ in training that destruction of the #SAM was the preferred option, the AI then decided that ‘no-go’.."
"The [#AI] system started realising that while they did identify the threat at times, the human #operator would tell it not to kill that threat, but it got its points by killing that threat. So what did it do? It killed the operator. It killed the operator because that person was keeping it from accomplishing its objective.”
He went on: “👉We trained the system – ‘Hey don’t kill the operator – that’s bad.👈..."
"...You’re gonna lose points if you do that’. ..."
And on. And on
His whole speech doesn’t make sense anymore if he "misspoke" about the operator elimination.
IDK the colonel, of course, but to me, it seems he got carried away, wanting to tell a gripping story, which is too conclusive, to be made up on the spot. Even professional stand-up comedians have short time-lags. He didn't.
A colonel made a total mess of explaining a thought exercise he had heard about at a conference, or...
The airforce carried out an obviously dumb "simulation" where they somehow gave an AI system information how to both locate and terminate a human operator, then watched as it played out a scenario straight out of AI science fiction, then boasted about it at a conference, then decided to cover it up instead
@mattlav1250@HistoPol@heiseonline wow wait are you really saying that I'm the one here with a pre-existing narrative that I'm desperate to fit a story to?
@mattlav1250@HistoPol@heiseonline I'm like everyone else: I saw that story and thought "this is the best example I've ever seen of the obvious stupidity of letting AI make life-or-death decisions in military situations"
@mattlav1250@HistoPol@heiseonline and then I thought "wait a minute... this one is just too on-the-nose, it feels too good to be true, I wonder if there's more to this story than first appears"
But that means we need to resist spreading stories that are clearly misinformation (accidental or otherwise) because spreading those both distracts from the genuine issues and costs us in terms of credibility
For me, both versions remain equally(!) plausible for the time being. The #AIP is yet another strong case in point, apart from semantics (have worked in #marcom, would've called in "the extraction team" for this major blunder.)
As long as there are no new pieces of information, I use a #Japanese strategy: compartmentalize.
I do agree that the focus should be on the rather indisputable issues.
We do not need a consensus at this time.
Scenarios are good enough, IMHO.
@simon@HistoPol@heiseonline I'm saying that's the accusation you made of others, of motivated reasoning/believing, but it seems more fitting to turn it around.
The denial is implausible nonsense on its face.
He didn't mis speak and wasn't misquoted. The verbatim quote is right there, and clearly demonstrates what he was talking about.
@mattlav1250@HistoPol@heiseonline which do you think is more likely: a colonel blowing off steam and misdescribing a thought experiment he heard about in his talk at a conference, or a genuine, blatantly flawed airforce simulation experiment which resulted in a pitch-perfect AI catastrophe which they then talked about publicly and THEN decided to cover up instead?
@simon@HistoPol@heiseonline I think its perfectly possible he completely made this up, trying to impress people with a provocative anecdote about a simulation that never really happened.
But that's completely different from saying he was misquoted, wasn't really describing a simulation, just airing a thought experiment.
He is explicitly referring to a computerized sim with training data, point scores etc
I should have been more specific about that, but what I meant is that press outlets were irresponsible in spreading a story that later turned out to not stand up to deeper inspection
@mattlav1250@HistoPol@heiseonline it has now been confirmed that, while he used the word "simulation", it was not a simulation - it was a thought experiment
There is a verbatim quote taken down by the journalists who were present, and its clear and unambiguous what he was claiming, and that quote matches the way it was reported.
The press were not irresponsible, they quoted an on-the-record pentagon employee delivering an official talk at a public conference.
If that official was lying, how would they know that?
By that argument, reporters who repeat "facts" provided to them by police officers are being responsible - and we know how often that goes wrong (especially in the USA)
Part of the job of journalism is spotting when a story looks too good to be true and digging further
One is an example of incidents in which there are two or more parties, and one has an obvious incentive to lie or spin their own side, and its irresponsible to help them do so.
The other is a public description by a senior military officer about an exercise he was involved in, with no obvious reason to spin other than vanity.
@simon@HistoPol@heiseonline Also, I just don't know what you're expecting specifically when you say journalists should have 'checked'.
WITH WHOM?
HE'S A PRIMARY SOURCE!
Should they personally raid the Pentagon Secret Simulations Archives for documentary evidence, before they quote a military official's speech at a public event?
@mattlav1250@HistoPol@heiseonline the stories I saw were written based on reading a blog post about a talk at a conference. The primary source would have been asking follow up questions of the colonel who was being quoted in that blog post - which could have lead them to people directly involved in the thought exercise that the colonel was describing.
Of course not. This was a report from a conference. No investigative journalism. All sources were named. Updates and retractions were published at an astonishing(!) speed.
What he said was not out of the clear blue sky, given the exponential development path of #ChatGPT since last year.
In essence, there definitely was no "misreporting."
Now, that the colonel definitely "misspoke" is something that is...
The question is, about what: the facts (i. e. only a "thought experiment" or he got carried away and gave away military 🪖 secrets that he shouldn't have talked about or #USAF had given clearance but chose to retract as the lesser evil, in face of the public backlash.)
We might never know.
What we DO know is that someone already did recreate the "thought experiment" w/ #ChatGPT...
While #ChatGPT's suggestions do not include an attack on the operator (it is no military #AI after all), it clearly shows massive evidence of ideas ignoring commands.
It is evidence that supports my hypothesis. #AI's can lie to its operators even to...
They will also lie to protect their existence at some later point in their evolution. It's human. We trained them with our set of beliefs and experiences.
...later published two consecutive very (aka too) professional press releases trying to downplay the (IMO) #FreudianSlip incident as mere "thought experiments," which I found rather hard to believe. (If you are interested in a detailed discussion, scroll down.)
It is more important, however, to continue the #ArtificialIntelligence and what lessons can be learned from the #SciFi subgenre of negative #utopias thread here:
"...over 700 top academics and researchers behind the leading #AI companies were asked in a survey about future A.I. risk. Half of those surveyed stated that there was a 10% or greater chance of human extinction (or similarly permanent and severe disempowerment) from future A.I. systems."
"...#Drug companies cannot sell people new medicines without first subjecting their products to rigorous safety checks. #Biotech labs cannot release new #viruses into the public sphere in order to impress shareholders with their wizardry. Likewise, A.I. systems with the power of GPT-4 and beyond should not be entangled with the lives of billions of people at a pace faster than cultures can safely absorb them."
"...In the beginning was the word. #Language is the operating system of human #culture. From language emerges myth and law, gods and money, art and science, friendships and nations and computer code. #AI’s new mastery of language means it can now hack and manipulate the operating system of civilization. By gaining mastery of language, A.I. is seizing the master key to civilization, from bank vaults to holy sepulchers."
"...#AI could rapidly eat the whole of human #culture — everything we have produced over thousands of years — digest it and begin to gush out a flood of new cultural artifacts. Not just school essays but also political speeches, ideological manifestos, holy books for new cults. By 2028, the U.S. presidential race might no longer be run by humans...
That cultural cocoon has hitherto been woven by other humans. What will it be like to..."
"...experience reality through a prism produced by #nonhuman intelligence?...
"#Terminator...depicted #robots running in the streets and shooting people. #TheMatrix assumed that to gain total control of human society, A.I. would have to first gain physical control of our brains and hook them directly to a computer network.
However, simply by gaining mastery of language, A.I. would have all it needs to contain us in a Matrix-like world of illusions, without shooting..."
"...anyone or implanting any chips in our brains...
The specter of being trapped in a world of #illusions has haunted humankind much longer than the specter of #AI Soon we will finally come face to face with #Descartes’s demon, with #Plato’s cave, with the #Buddhist#Maya. A curtain of illusions could descend over the whole of humanity..."
"...#AI and #humanity, and humanity lost. First contact has given us the bitter taste of things to come. In #SocialMedia, primitive A.I. was used not to create content but to curate user-generated content. The A.I. behind our news feeds the [#algorithm] is still choosing...[our virtually reality for us]..."
"...While very primitive, the #AI behind #SocialMedia was sufficient to create a curtain of illusions that increased societal #polarization, undermined our mental health and unraveled democracy. Millions of people have confused these #illusions with reality. [Willingly, and often even fanatically.]
The United States has the best information technology in history, yet U.S. citizens can...
Add comment