The AI that draws pictures based on verbal instructions

“Four years ago, no one even believed such a thing could be done,” says Xiaodong He, the lead researcher on the project. Indeed, for the past five years, He has been researching the relationship of images and words, training AIs to do all sorts of compelling tasks. First, he created an AI called CaptionBot that could use words to describe a photo... Then, he pushed that research further, creating an AI that could answer specific questions you might ask about photos. Now, with AttnGAN, he’s “closed the loop.” In other words, Microsoft’s AIs can create images from mere words, that another AI can then caption.

Also, I strongly urge He to start talking in a more biblical style.

These photos are often realistic, though relatively low resolution–and on top of the realism, they’re also highly specific in their customized detail. This is the “attention” part of “AttnGAN,” as the AI fine-tunes very small regions of each image to the verbal specifications. That means a bird, for instance, can have extremely specific features, like a blue beak, a yellow beak, a long beak, or a short beak.

But ask AttnGan take these objects out of their context, and mix it with some other objects, and things go wonky. “If there are complicated attributes or relationships of objects in the system, then the machine gets confused and draws something not as good as we hope,” says He.

In another case, researchers asked for “an image of a girl eating a large slice of pizza.” The shape of the girl is actually excellent. [Uhh, no, unless you think that Picasso was just painting what he saw] But just about everything else is off in this invented portrait. It looks borderline cubist in its strange rendering.

Even still, He isn’t deterred. In just a couple years, he insists these AI models will vastly improve, and with faster computers loaded with more memory, researchers will be able to make the final images larger and more detailed, too. Given his last half decade of progress, it’s hard to disagree.

https://www.fastcodesign.com/90158102/microsofts-new-ai-creates-fake-photos-from-your-words?utm_content=buffer50b71&utm_medium=social&utm_source=facebook.com&utm_campaign=buffer