New Delhi: A new AI tool built by researchers at the Carnegie Mellon University in the US can now help art historians identify patterns in paintings that the naked eye cannot spot. This might allow them to find the creators of centuries-old paintings, by comparing them to the styles of masters like the Dutch artist Rembrandt van Rijn.
To begin with, the team trained the model on the images of Rembrandt and taught it to analyse the features of paintings such as their edges, colours, distribution of light, and even the brush strokes. The model classified his paintings with a 95.8 per cent accuracy, according to a study published on 26 October in the Proceedings of the National Academy of Sciences (PNAS). The researchers have stressed that the tool is not meant to replace human experts but rather help them.
The reason they picked Rembrandt for their initial trials is because attributing a painting to him is a complex affair. The artist had a long career where his style changed and he had several pupils and assistants working with him. The paintings too are likely to have been altered by several restorers over the years. To say that a painting has been created by Rembrandt, experts have to examine historical records, painting material, and even technical evidence including X-rays and infrared imaging. The team of researchers wanted to simplify this process using AI.
Also read: What are rad-hard materials? Why satellites and nuclear reactors need them
Training AI
Researchers first assembled a dataset of over 1,300 high resolution digital images, including 311 works that have been attributed to Rembrandt over the years. The AI tool first learnt to recognise patterns across the images and then used those patterns to analyse other images.
The tool does not simply give researchers a ‘yes’ or ‘no’ answer but explains what parts of the paintings led to its conclusion. The team at Carnegie Mellon referred to this as “explainable AI” which is designed in a way that experts can look into what is usually a “black box” behind the AI.
“Experts need to understand how the AI model/algorithm came to its decision. That is critical in successful human-AI collaboration, so we needed to make sure the algorithm could expose which areas of the painting influenced the algorithm’s conclusion,” co-author Jon Cagan, professor of mechanical engineering, said in a statement.
For example, if the tool flags that the hands in a painting do not resemble Rambrandt’s work but the rest of the painting does, it might help experts conclude that maybe the hands were painted by a student or an assistant.
For Andrés Arias-Rosales, an alumni of the institute involved in the project, a highlight of the research was being able to work with Arthur Wheelock Junior, a renowned Rembrandt expert.
“Working with him was a dream of mine. We wanted to understand Rembrandt through the eyes of a world-class expert, then understand it through the eyes of ‘the machine’, our AI approach, and bring those together,” Arias-Rosales said in a statement. Even before the team could begin training the AI, they first went to the National Gallery of Art in DC to study Rembrandt painting themselves and understand his style.
To test out their tool, the team applied it to a painting found in 2024, titled ‘Portrait of a Girl’. The painting, discovered in an attic in Maine, was sold for $1.41 million because of its possible connection to Rembrandt. The model was about 84 per cent confident that the painting was indeed created by the Dutch artist.
“These results aren’t an authentication, they are a statistical observation meant to be considered alongside historical research, material analysis, X-rays, and other tools with the final decision made by human experts,” co-author Phil LeDuc, mechanical engineering professor at CMU, in a statement.