Training my computer to draw birds

Of all the jobs that Artificial Intelligence (AI) will replace over the next few decades I never thought mine would be top of the list. I got my degree in illustration 20 years ago, since then I have picked up many more skills but I have always been most proud of my ability to draw. I thought it made me more ‘visually intelligent’ than other creatives because of the volume of data an illustrator has to generate. Over the last few months I have been absolutely blown away by three tools, DALL-E 2, Stable Diffusion and Midjourney. These AI tools are so much faster than me, they have more range, and in most cases are just better at drawing than me. If you want to see what they can do checkout this gallery.

The tools do have some limitations, the main one being the sizes of the images they can draw, but most of these will get solved with market demand. Of course the first thing I wanted to know is how good it was at drawing the things I love, New Zealand animals. My ego was quite pleased to see them fail miserably and in quite entertaining ways (go try kiwi here). Here are some examples of white-faced herons (I have chosen a very well photographed species on purpose).

This is mostly because the AI’s have not been trained how to draw these animals. In the above examples MidJourney and Stable Diffusion confuse our native heron (which can also be found in Australia) with a North American Great blue heron. Developers are working on multiple ways for users to be able to train tools to draw specific subjects. One of them (DreamBoth for Stable Diffusion) involves training a model based on 20 or so images. I happen to be a very organised photographer with 2,700 bird photos and 3,300 photos of invertebrates all cataloged by name, place and time. It took me some time to figure out how to do it and it takes a lot of computing power to train the models, here are the 20 photos I used to train my white-faced heron model.

And here are some of the results (good and bad):

You can see there are still some problems but its pretty good! I can easily fix them up to create future works.

I have been using the tools to create components for illustrations (photo bashing). Here is an example that would have taken me ages to draw from scratch.

James Cook and his men encounter a kahikatea forest in the Waihou River in 1769

I’m excited about the tools and think they will make my work better, faster and cheaper.