On Tuesday, May 23, Adobe launched a brand new Photoshop tool called “Generative Fill”, which allows users to directly edit their images via text prompts or extend the background of a visual. The tool, powered by the company’s new AI software called Firefly, can be accessed during its beta testing through the desktop app. Online, some art history buffs have fun stretching their favorite works beyond the seemingly limitless frame – while others have fun lamenting the ways the tool can distort an artist’s original vision.
A key feature worth noting is that Adobe Firefly is trained solely on the Adobe Stock library, rather than a dataset of all sorts of scraped content available online, so apparently escape the eminent reviews of art theft and copyright infringement this tailgate other generative image software like Dall-E 2 and Stable Diffusion. But in the short time I’ve spent playing around with the tool, I can safely say that there are a few issues that need to be addressed before our dear photo editor friends panic.
Fortunately, getting started with Generative Fill is quite simple. Once you’ve downloaded the app, all you have to do is open the image you’d like to play with and start selecting the areas you’d like to work on. Getting the desired output is actually the hardest part! For the first pass, I extended the size of the canvas in Photoshop to create more space around the image for the Generative Fill tool to complete. Inspired by this tweet containing an extended field of Vincent van Gogh’s “Starry Night” (1889), I wanted to see if the software could “imagine” what lies beyond the four edges of several famous works of art.
Like Midjourney, Dall-E 2 and Stable Diffusion, Generative Fill can work on text prompts. However, if you simply select the blank area of the enlarged canvas surrounding your original image and click the Generate button, the tool will do its best to extend the original image based on the available information. You can see how he had more success with a lithograph by MC Escher compared to Thomas Eakins’ “Portrait of Dr. Samuel D Gross (The Gross Clinic)” (1875) – in the latter a chilling figure by Francis Bacon- esque appears in the lower left corner.
This software clearly struggles with portrait and body. On the other hand, the tool did a fairly reasonable job extending the background of Henri Rousseau’s “Tiger in a Tropical Storm” (1891), although I had to remove Rousseau’s signature, because text tends to mess with the generative output of most AIs. software. The tool performed even better with Wayne Thiebaud’s “Flatland River” (1997).
Take a sheet of a Twitter feed who used the Adobe tool to reinvent popular album art, I had to add the cover art for Lorde’s “Melodrama” and see what would happen if I put the painted indie-pop icon in space. I extended the size of the canvas and selected the empty space around the album cover and typed “outer space” into the text box and…it got kinda cool and funky! Cool in that the generative part felt like a giant Lorde was snuggling into a planet, but funky in that Lorde’s body wasn’t really a body at all.
The Generative Fill tool provides three different iterations in a single pass, and the generative quality reminded me of the open source AI image generation tool Crayon (formerly known as the Dall-E Mini) which rose to popularity last summer for its absurdly distorted productions and bolt-on body parts.
Having taken care of simply extending the images beyond their compositions, I tried to recontextualize the existing backgrounds in these works using the “Magic Wand” selection tool. I spent more time than I’m willing to admit struggling with the selection tool to highlight the areas I wanted to change, so my assessment of Generative Fill’s need for adjustment could very well be n be just a glaring learning curve on my part. I selected the background from Edward Hopper’s “Nighthawks” (1942) and remixed it with a “zombie apocalypse” text prompt and was actually quite pleased with the results. However, I specified “Zombie apocalypse Edward Hopper style painting” in the text box and received something more along the lines of Yves Tanguy associated to Simon Stalenhag instead… Not that I’m complaining!
Either way, the tool produces blended images with a dreamy, eerie feeling similar to what Craiyon was sending into the ether about a year ago. Generative Fill brings us closer to achieving what we thought was confined to our minds eye, but it certainly requires more editing and cleaning on top of what it provides to create a well balanced image with manipulations indistinguishable.
In a weird way, Photoshop might end up inadvertently deleting itself thanks to this update that lets us use our words instead of forcing us to memorize tools and settings or rewatch the same YouTube tutorial four times from following. Still, I think we’d all rather see researchers spending more time training the AI to properly transcribe what’s on our PDF resumes into the required job application text boxes instead of encroaching on and d potentially absorb an already vulnerable industry of exceptionally hard workers who already don’t get enough credit!
