#AI#GenerativeAI#OpenAI#AISafety#AIEthics: "For months, OpenAI has been losing employees who care deeply about making sure AI is safe. Now, the company is positively hemorrhaging them.
Ilya Sutskever and Jan Leike announced their departures from OpenAI, the maker of ChatGPT, on Tuesday. They were the leaders of the company’s superalignment team — the team tasked with ensuring that AI stays aligned with the goals of its makers, rather than acting unpredictably and harming humanity.
They’re not the only ones who’ve left. Since last November — when OpenAI’s board tried to fire CEO Sam Altman only to see him quickly claw his way back to power — at least five more of the company’s most safety-conscious employees have either quit or been pushed out."
#AI#AISafety#Cybersecurity: "With a few exceptions, AI safety questions cannot be asked and answered at the levels of models alone. Safety depends to a large extent on the context and the environment in which the AI model or AI system is deployed. We have to specify a particular context before we can even meaningfully ask an AI safety question.
As a corollary, fixing AI safety at the model level alone is unlikely to be fruitful. Even if models themselves can somehow be made “safe”, they can easily be used for malicious purposes. That’s because an adversary can deploy a model without giving it access to the details of the context in which it is deployed. Therefore we cannot delegate safety questions to models — especially questions about misuse. The model will lack information that is necessary to make a correct decision.
"AI risks are exploits on pools of technological power. Guarding those pools prevents disasters from exploitation by hostile people or institutions as well. That makes the effort well-spent even if Scary AI never happens. This may be more appealing to publics, or governments, if they are skeptical of AI doom."
I just discovered that one of my favourite philosophical writers, former AI researcher #DavidChapman, has published a book on the current state and future risks of AI;
I've read standalone essays David wrote years ago exploring the philosophical underbelly of AI development, and I'm confident this timely book will be just as insightful.
"This is the domain of #AISafety, where systems are often imagined as moral... agents... AIs should align to human values, ideally by understanding and acting according to them, or at minimum by reliably recognizing and intending to respect them.
Attempts to specify what abstract values we want an #AI to respect fail because we don’t have those. That’s not how human motivation works, nor are “values” a workable basis for an accurate ethical framework."
“…a deep truth about AI: that the story of AI being managed by a ‘human in the loop’ is a fantasy, because humans are neurologically incapable of maintaining vigilance in watching for rare occurrences.”
I'm a PhD biologist and I read @OpenAI's threat preparedness assessment plan for CBRN threats. It appears to be total nonsense designed without any input from a scientist. Here's why: #ai#artificialintelligence#airisk#aisafety
When people fret that A.I.s will achieve superhuman general intelligence and take over the planet, they neglect the physical limits on these systems. This essay by Dan Roberts is a useful reality check. A.I. models are already resource-intensive and will probably top out at GPT-7. Roberts is one of the physicists I feature in my new book about physics, A.I., and neuroscience. #AIrisk#AIsafety#Singularity@danintheoryhttps://www.sequoiacap.com/article/black-holes-perspective/
It is crucial that they know those models are safe and that they are not exposing themselves to unacceptable levels of regulatory and liability risk.
If a European startup has to meet safety standards of general-purpose AI models under the AI Act, they will only want to buy models from companies that can assure them that the final product will be safe.
But the information and guarantees that they need are not being offered.
All of this means that European startups have unsafe services that they will be asked to make safe under the AI Act, with limited resources to do so."
> The fixation on speculative harms is “almost like a caricature of the reality that we’re experiencing,” said Deborah Raji, an AI researcher at the University of California, Berkeley. She worries that the focus on existential dangers will steer lawmakers away from risks that AI systems already pose, including their tendency to inject bias, spread misinformation, threaten copyright protections and weaken personal privacy.
A group of researchers managed to get both Stability AI’s Stable Diffusion and OpenAI’s DALL-E 2 text-to-image models to disregard their policies and create images of naked people, dismembered bodies, and other violent and sexual scenarios.
Their work, which they will present at the IEEE Symposium on Security and Privacy in May next year, shines a light on how easy it is to force generative AI models into disregarding their own guardrails and policies, known as “jailbreaking.” It also demonstrates how difficult it is to prevent these models from generating such content, as it’s included in the vast troves of data they’ve been trained on, says Zico Kolter, an associate professor at Carnegie Mellon University. He demonstrated a similar form of jailbreaking on ChatGPT earlier this year but was not involved in this research."
#AI#AISafety#AIEthics: "The emerging field of "AI safety" has attracted public attention and large infusions of capital to support its implied promise: the ability to deploy advanced artificial intelligence (AI) while reducing its gravest risks. Ideas from effective altruism, longtermism, and the study of existential risk are foundational to this new field. In this paper, we contend that overlapping communities interested in these ideas have merged into what we refer to as the broader "AI safety epistemic community," which is sustained through its mutually reinforcing community-building and knowledge production practices. We support this assertion through an analysis of four core sites in this community’s epistemic culture: 1) online community-building through web forums and career advising; 2) AI forecasting; 3) AI safety research; and 4) prize competitions. The dispersal of this epistemic community’s members throughout the tech industry, academia, and policy organizations ensures their continued input into global discourse about AI. Understanding the epistemic culture that fuses their moral convictions and knowledge claims is crucial to evaluating these claims, which are gaining influence in critical, rapidly changing debates about the harms of AI and how to mitigate them."