Compositing and inpainting photos with generative AI

---

Ordering an AI to do something is like negotiating with a Djinn. It will do its best to give you what you request in your prompt (instructions), but you have to be very careful. If you ask a Djinn to make you live forever, you had better specify, “as a young, healthy woman of 30” or you’ll be stuck in a creaky, 68-year old body for all eternity. Even when you provide very precise instructions, AI can't know what you really want.

Every day, there are more free tools online for generating text and images, even logos, layouts, and videos. Have fun and play! But plan to edit, re-write, or re-paint, because the AI will make hilarious mistakes. It will certainly surprise you. Be prepared to take advantage of those surprises, and use AI to try out new concepts, or as a sketching tool. You can generate enough ideas quickly enough to speed up concept development or focus on a new direction for a project. If you have strong writing, editing, digital painting, drawing, photo editing and graphic design skills, you’ll be intrigued rather than frustrated. 

About the ethics—although an AI can synthesize new images from billions worldwide, it leans toward imagery and styles from popular culture. An AI also tends to produce commonly expected images, unless trained . Towns usually look like either Tokyo or New York, with skyscrapers. If you request "a female warrior" you'll get characters that resemble Disney's Mulan, unless your prompt is very specific. If you request a “mermaid,” you may get characters that strongly resemble Disney’s Little Mermaid.  (You really want to avoid getting a "cease and desist" letter from Disney's lawyers.) To avoid violating copyrights all over the world, you need to provide your own work as a model for the AI. You could start with a composite of images that are copyright-free from an image bank, such as Unsplash or Pexels.

I used Stable Diffusion's AI and Adobe Photoshop's Generative AI plug-in to generate the mermaid images below, while trying to train the AI.   But I discovered that to get predictable and reproducible results, you have to download a version of an AI to your own computer, and then train your own AI, and that takes a whole different level of knowledge. So you have to train yourself first. And that takes time. A lot of time. All of this goes more smoothly if you have support, such as a club, or take a class. I will post some free classes and tutorials soon. In the meantime, check out my experiments, below.

TIP #1. An image-generating AI may not understand similes. In your prompt, you can’t request “skin like a fish.” You should write, “skin with small orange scales.”  To create original mermaids, I had to describe each major body part, choose the best of dozens of results, and with each iteration, adjust the description further, as the AI learned. 

For example:

“Female human face, orange hair, red eyes; human torso with slim hips merging with fish tail, orange scales on skin, orange fins. Full figure, swimming, side view. In the background is a kelp forest in blue water, with rays of light. In the foreground are pink shells and red coral. Full image.” 

And then you need to go through a hundred iterations (see Mer-people, below).

TIP #2.  Unless trained out of it, AI adds full lips, sultry eyes, and very large curves to every female character, and huge muscles with lots of armor and big weapons to every male character. Perhaps AI is being raised by programmers with the fantasies of 12 year-old boys, or heavily influenced by popular culture. With non-human characters, the AI can become very confused. I have seen eye shadow and lipstick on monsters, bugs and reptiles. (As well as claws, spikes, bat wings, dripping saliva, and fangs, of course.) 

Therefore, in Stable Diffusion I must always specify these negatives: 

“No legs, no teeth, no fangs, no claws, no tongue, no armor, no breasts, no big butts, no clothes, no weapons, beautiful, not scary.“ 

And yet, Photoshop's AI was more cautious about sexualizing images. When I specified "no clothes," I had to check a box to confirm that I'm over 18. When I requested a golden "breast plate" on a female warrior, I got an error message and had to change it to "chest plate!"

TIP #3. Stable Diffusion and Photoshop's AI both got confused about the number of limbs, even when creating a human. See the astronaut, below, and the arthropods. You may get an extra leg. Or three. You may also get extra eyes or fingers, or none. Photoshop's AI has a lot of trouble with hands and faces. If you want your character's hands to clasp the hilt of a sword, better be prepared to re-paint the fingers, if you have that skill. If your character's face is lop-sided, re-paint it. Or keep refining and training.

Nevertheless, this technology is amazing! As you might imagine, I am most interested in creating my own images. So I'm uploading original art work to an AI, to guide it in building characters and environments. Eventually I'll be able to rotate, apply textures, add lighting, and more. More to come.