AI Algorithm Puts the Color Back in Black and White Films

Summary: A newly developed integrated software application combines interactive and automated coloring techniques with deep learning technology. The application is able to generate color to black and white movies automatically.

Source: Graz University of Technology

Countless historical photographs are stored in black and white in the world’s archives. Today, these cultural assets and contemporary documents are conserved by means of digitization and (partially) improved through digital restoration. Efforts have been made since the 1970s to colorize this film footage.

However, there was no real progress – the effort and the associated costs for manual or semi-automatic coloring techniques were too high. It is possible in principle to color films fully automatically. However, this has the disadvantage that the colors may be nice to look at, but they are not true to reality.

This is exactly where the project “RE:Color: Efficient coloring of films in cinema quality based on novel machine learning methods” at Graz University of Technology (TU Graz) came in.

Computer scientists led by Thomas Pock from the Institute of Computer Graphics and Vision together with the Graz-based company specializing in the restoration of historical films HS-Art have developed an integrated software application that combines interactive and automated coloring techniques with deep learning technologies.

The result is an algorithm for a predominantly automatic, yet fully user-controlled coloring process.

Realistic coloring

According to Pock, it is essential that humans can influence the coloring process: “You always need someone who is familiar with the historical traditions who can say what the clothes, the facades, etc. looked like back then. Was the soldier’s uniform green or blue? No algorithm can decide that, but it can learn from it.”

The algorithm must therefore be fed with a sufficiently large collection of training samples in order to then automatically take over the coloring of historical films.

“It’s about coloring the films as efficiently as possible with as little user input as possible. This can mean, for instance, that a person specifies the coloring for a film frame and the software then takes over the coloring of further frames,” explains Pock.

This central requirement of user-guided control is only fulfilled thanks to pre-trained neural networks that can be dynamically influenced by user interaction.

Credit: Hsartdigital

For this purpose, the researchers have developed different novel approaches in the field of automated coloring based on artificial intelligence (AI).

Together with the developers of HS-Art, they then implemented the most efficient approach in a prototype application and generated a sufficiently powerful collection of training samples.

Then the implementation of human-guided control took place to obtain authentic and appropriate color schemes.

Authentic image noise

With the algorithms developed, the films can be restored extremely cleanly and also coloured – but this is not always necessarily desirable.

This shows a black and white image from an old movie and a colorized version
Researchers at TU Graz have developed an AI-based algorithm that is now the basis of software for realistically coloring historical black-and-white films. Credit: HS-ART

Pock: “With historical footage and cinema films in general, you need a certain amount of noise, so-called “film grain”, otherwise it doesn’t look authentic to the audience. For this reason, the software can also artificially generate and add this noise again after restoration and coloring.”

The core algorithm was published at a major international symposium, and the source code is freely available. For efficient use, however, software based on this, which was developed by the project partner HS-Art and is in their product portfolio, is necessary.

The “Diamant film colorizer” was used, for example, in the ZDFzeit documentary series “Hitler’s Power” to color historical footage true to the original.

About this AI research news

Author: Susanne Filzwieser
Source: Graz University of Technology
Contact: Susanne Filzwieser – Graz University of Technology
Image: The image is credited to HS Art

Join our Newsletter
I agree to have my personal information transferred to AWeber for Neuroscience Newsletter ( more information )
Sign up to receive our recent neuroscience headlines and summaries sent to your email once a day, totally free.
We hate spam and only use your email to contact you about newsletters. You can cancel your subscription any time.