i think something people don't understand about #ai models like #largelanguagemodels is that they're fixed. they're deterministic. the same input results in the same output. in the whole #copyright discorse recently, people talk like the ai has some agency; that you're just "telling it what to do". only in the same way you tell photoshop what to do. it's just the type of input is different. they're complex. they're not magic. there's no ghost in the machine (yet).
@wagesj45 Some people understand this, which is why there are discussions about how much these are “plagiarism machines”.
Also, technically they are probabilistic, not deterministic. There is a “ghost” in the machine that tricks a lot of people, but it’s the same ghost that makes casinos money and powers the Gambler’s Fallacy. People are really bad at statistics and their eyes tend to light up at slot machines they can’t win because they are shiny.
@max if i take a Stable Diffusion model, give it the same input, same seed, same everything, i get identical output every time. that seems to fit the definition of deterministic to me. gpt has that concept of "temperature" which adds randomness, but that is an application feature, not part of the model. no different than any other algorithm that adds a random number generator as part of its process.
@wagesj45 Seed implies a deterministic pseudo-random number generator under the hood, sure, but the fact that a random number source is involved at all is proof that the core algorithms are in the category of probabilistic even if we mostly run them deterministically (and pseudo-random rather than true random).
Computers are bad at true random so we run a lot of probabilistic algorithms deterministically. Doesn’t mean they aren’t probabilistic as a class of algorithm.
@max maybe I'm not fully grasping this, but isn't that "probability" pre-computed? to me that signals that use of that model is then deterministic. even training the model, if you use the same seed values and training data, you should receive an identical model at the end.
@wagesj45 Well yeah, if you fix the seed value and PRNG algorithm you get the same results every time. There are useful properties for fixing probabilistic algorithms to deterministic runtimes.
But the algorithm itself is designed to not use fixed seeds and “wants” a true random data/entropy source, even if we don’t run it that way most of the time (because determinism is easier to debug, because true random is harder for silicon digital computers).
Add comment