Bilderatlas – Progress Post 1

Current progress

The first three weeks have not been very productive compared to the first milestone we had set last semester. The initial plan for the first three weeks was:

“Select tables and annotate them. Complete the first bot that loads images in the CNNs database.”

Briefly, while the tables have been chosen and the relative images retrieved on the internet, the annotation tool is still not available, nor is the API for the CNN server.

The first thing we have done was to select 4 tables on which we will perform our analyses:

  • Table 2. It is rather a conceptual table, with little or no visual similarity among the paintings. The theme of the Pegasus, the winged horse, is particularly interesting. It dates back to Ancient Greece, but in this table we find it together with other paintings from the Medieval Age. Through the use of CNN we would like to explore the theme of the Pegasus to see how it evolved in time and geographically. If we manage to get a wide result, the idea would be to create a map that evolves in time and shows how the Pegasus changed across the European territory. This research can be considered “conceptual” because we will not search for the exact same image of the Pegasus, but rather for an evolution of the theme.
Table 2 - The cosmos in Greece: the resemblance to humans.
Table 2 – The cosmos in Greece: the resemblance to humans. [1]
  • Table 45. This table contains several paintings of buildings that have a similar internal architecture. We will search for the theme of the dome. At the same time this table also contains a painting that apparently does not connect with the other elements of the table. It’s “Il sangue del redentore” by Giovanni Bellini. Just like for the Pegasus, we want to explore the theme of the bleeding Christ and the angel to see if we can find some pattern in time and geography.
Table 45 - Gestures to the superlative grade. From the grisaille to the painted reality.
Table 45 – Gestures to the superlative grade. From the grisaille to the painted reality. [1]
  • Table 46. This is the table on which the CNN must really be tested. The theme of the Nymph can clearly be observed by the human eye in several of these paintings, but we need to see if the CNN will be able to find these paintings starting from the woman in the “La nascita di San Giovanni Battista” by Domenico Ghirlandaio (top, right in the image below), and other paintings in the database that report the theme of the Nymph.
Table 46 - The Nymph-maid
Table 46 – The Nymph-maid. [1]
  • Table 25. It is a strictly visual table with stone and pillar reliefs. This query will likely be a simple task for the CNN.
    Table 25 – The Apollineus Ethos and the Dionisic pathos in the Malatestian cosmos. [1]

For each table we searched on the internet for the corresponding paintings or pictures, to find a color version of good quality. Naturally, we were not able to find all of them, but the found ones will presumably be enough for our purposes. If this is not the case, we can always try to use the pictures scanned from the book “L’Atlas Mnémosyne” on Aby Warburg’s work, even if their quality is not exceptional, keeping in mind that the fact that they are black and white will influence the CNN results.

Regarding the annotation step, we have not been able to start yet because the DHCanvas is still not available to us.

As far as the bot is concerned, we are currently in the process of developing it. It will load the images with their metadata in the CNN. Unfortunately, we have not been able to test it yet, because the server was not available. However, we expect to perform such tests next week.

The source code used for the bot is in Python. In particular, we have imported the “requests” and “json” libraries to help us, respectively, sending HTTP POSTs and handling JSON objects. In fact, the APIs are based on HTTP POST both for adding images and finding similar paintings.

We report in brief an example of the bot used for adding an image to the database:

The address of the HTTP POST will be: <web-server-url>/api/v1/database. An image will be characterized by its URL, its metadata (such as author, genre, title), its source and (if available) an URL where there is a description of the painting. Following is an example of code that adds an image from Wikiart:

{
'image_url': 'http://uploads6.wikiart.org/images/john-singleton-copley/governor-john-wentworth.jpg',
'metadata': {'author': 'John Singleton Copley',
'genre': 'portrait',
'style': 'Neoclassicism',
'tags': ['male-portraits'],
'title': ' Governor John Wentworth'},
'origin': 'wikiart',
'webpage_url': 'http://www.wikiart.org/en/john-singleton-copley/governor-john-wentworth'
}

 The result is:

{
 "id": "56a8af746be0fa5bd544c120"
}

We can therefore see that the result is contained in a JSON object and it consists of an identifier. This value could be used in the future to refer to this image in the queries. Anyway, every image is already indexed by the web URL at which they are accessible.

Further ideas using basic data analysis

During the preliminary work, we discussed about further ideas we will have from the usage of the CNN algorithm. As previously mentioned, the theme of the Pegasus could show a strange pattern in its appearance through the history of art. One idea can be to launch a query with the painting from Table 2, and then take all the results relatively close (with a sufficiently low distance, according to a tolerance given by us) and collect them in a data frame; we could analyze the number of appearances of the theme for each decade and then perform time series analysis; we can then see the spectrum of frequencies through time, if there is a trend or a seasonality in the pattern; maybe, with reliable data, forecasting appearances of the theme in the future could be also possible.

References

[1]. L’Atlas Mnémosyne, Aby Warburg, 2012L’écarquillé.