“A fox crossing a residential street. The fox has a human face. There are autumn leaves on the ground, terraced houses in the the background, and a slight mist.”
It's just ignoring most of my prompt (as well as really struggling with what foxes look like). I've tried many iterations and variations, they're all like this.
@john The thing about Stable Diffusion is that it is not a tool for following a logical set of instructions. It is about associations and links into features of existing images on the internet. In the case of your prompt, the associations attached to 'autumn' and 'leaves' override the associations of 'terraced houses' and 'residential street'. Also, pictures on the internet of foxes with human faces are, as you say above, not common, so SD doesn't have a lot to work from.
@john Another thing about SD is that it can be sensitive to your prompt details. There may be few foxes with human faces on the internet, but there are plenty of foxes so it should be able to do a lot better than your outputs. The challenge of using SD as a tool can often be in getting the feel for how it likes to work and going with it - more like sailing than motoring. Then if you get a picture you like with a fox, SD lets you experiment with infilling the face area with a new prompt.
@BlackPhi I guess I'm a little bit surprised that SD doesn't get which bit is the face and composite, like it can do with subject and background. But the experimenting I’ve seen today leads me to think that although AIs have a strong bias toward the conventional, some can be coaxed to do such things. Dall-E was certainly resistant, but can do it.
@bhawthorne Yes, I’ve actually thought about that. The LLM would need to know what it is aiming at, so you’d need to hook in an image description AI to describe a model image, and then tell it to turn that into an effective prompt. I don't know whether they know what makes good prompts though, because they avoid training on AI generated stuff I believe.
@mike I feel like they're more at the DALL-E level for a lot of stuff. Mostly correct but with small or subtle errors, and gaps in capability.
I don't care much about the current error rate for stuff I do. It's adequate. I care more about prompt comprehension, which is also hit and miss (but it's probably as good as people on average).
@Zeugs Well, my experience of Dall-e even a while ago was considerably better. Here's what it does with the prompt now. It's ignoring the human face part, but obviously these are light-years better.
@john@PeterFalkingham@miekeroth
This turns into: lets Google that for me with Stable diffusion. It just takes at least 60seconds longer than googling. First shot. Fox running no filters got the legs problem. The shape fits better than on your SD stuff.
@john@PeterFalkingham@miekeroth
In the end the hard truth about AI generation is that you should kill your darlings early. If the model doesn't hook it's pointless.
But look at residential cyberfox he also has back legs but here the model can compensate. It relaxes.
@Zeugs@PeterFalkingham@miekeroth I'm not sure exactly what you're arguing, to be honest, but my arguments pretty simple, SD seems a long way behind. I need dozens of attempts to get things like a fox without egregious errors, whereas DALL-E doesn't even seem to make mistakes (although it's interesting it still ignores important components of prompts and absolute will not make a fox with a human face!)
Quality is not particularly for what I’m vaguely planning to do, so oh well I guess.
@etchlings Yeah, essentially. I don't think artists should be ignoring AI, or pinning their hopes on some sort of copyright salvation, so I'm diving in.
@john it’s a worthy effort. My vague understanding was that most of the “effective” visual ai was being paywalls, but that’s not something I looked deep at.
@john every aspect of this is so much better than even the DALL-E stuff @PeterFalkingham posted; the fox body is more accurate, the leaves look real, the street looks better, the houses look better, and so on.
Add comment