Adam Jensen - 3D versus AI

At least, that's my prediction based on several experiments I'd like to write about here. Actually, this step was supposed to come later in the production process, but a small test I did on the side caught my attention.

The experiment starts, just as an aside

For me, ChatGPT isn’t just a good conversation partner - it’s also an excellent image generator. Thanks to the ability to communicate iteratively, we can edit images together with remarkable precision. And during one such session, I simply gave the AI our Jensen Files cover image and asked it, just for fun, to turn it into a photorealistic portrait of Jensen. After just the second round of revisions, I had a result that really captivated me:

For someone who has suffered and felt alongside this character multiple times, traveling through a dystopian future for weeks on end, this sight is revelatory. It’s as if a final veil is being lifted. And with that, a technical question was answered - at least in terms of its feasibility: AI can definitely support us. First, we produce the film in 3D with a high level of detail. Then, the AI puts the finishing touches on the images.

A workflow is taking shape

This approach ensures that we can solve several problems that AI-generated films (in some cases, still) have. Local AI, in particular, is highly prone to errors in the following areas, and since we prefer local AI in the final process, these points are indeed important to us:

1) Our character remains consistent. All distinctive features are carried over into the final image. Adam, in particular, with his many details, is prone to hallucinations - whether added or imagined. No buttons, earrings, etc., disappear or appear.

2) We can control multiple characters on screen. Even the latest high-end models struggle with this, which is why AI-generated films often can only keep a single character on screen for any length of time.

3) We have full control over the scene's movement. It remains to be seen how much AI can help us make these movements look natural.

4) Thanks to these advantages, we can avoid a lot of trial and error in the still very time-consuming process of generating video footage. This is a positive development that will save us not only time but also a lot of electricity.

Adam Jensen in daylight

When looking at the two images, you’ll also notice subtle adjustments to the lighting and to Jensen’s line of sight. The area around the eyes, in particular, has undergone the most significant changes. I made these adjustments and also wanted to see how my 3D model would fare in a direct comparison with AI in daylight.

Here you can really see where the AI's strengths lie: delicate eyelashes, a more textured skin surface, the film of moisture on the eyes - which creates a realistic reflection - and the correction of the inner corners of the eyes.

In addition, the AI performs a subtle color correction, which, in my opinion, isn't very distracting. The overall impression remains intact; the correction simply refines what would have had to be done in post-production anyway.

Under daylight, the only noticeable difference between 3D and AI was minimal - a fact that can clearly be attributed to Blender’s outstanding capabilities. The image on the left was rendered in just 12.71 seconds using Cycles. There isn’t much left to do to convince the viewer, but this final step is still essential to maintaining the cinematic illusion.

If you'd like to try this out for yourself, I recommend making it clear in the AI prompt that the character's geometry should be preserved entirely and that only subtle details should be added. It also helps not to feed the AI an image with too high a resolution, so that there's enough leeway to achieve these details.

What's Next

Next up is the transition from still images to video. We already have several models in mind for this, but they still need to be tested. Before that, however, we’ll focus on AI-generated character animation. Here, too, there have been significant innovations with local models in recent months.

In closing, I have to say once again that this test really impressed us. Just a few years ago, results like these would have been impossible to achieve. When you think about it, we now essentially have a real person stored on our hard drive - one I can capture in a photo in any pose imaginable without even leaving my desk. And it’s accurate down to the little finger.

This will certainly help many photographers produce high-quality photos more quickly and accurately, but at the same time, it will also render photo models obsolete to some extent. I'm not sure myself whether I think that's a good thing. The fact is, however, that AI is here to stay.

Best regards, Winni & Doreen.