Recognizing copies of paintings – Blog post III

Introduction

This third blog post focuses on the concrete results we achieved so far and on the major issues that still affect the implemented algorithm and thus its performances. One of the most remarkable development has been the creation of the skeleton database. Many paintings have been processed thanks to the previously designed interface and coordinates of skeletons vertices have been gathered and saved in txt format files. At this point around 250 skeletons of people are available for our simulations. Since machine learning techniques need a much higher number of elements we are still improving it in order to make the algorithm more robust.

The rules for the definition of the skeletons have already been discussed in the previous blog posts, however it is noteworthy that the head and the abdomen vertices can not be missing since our normalization procedure is based on the distance between these two points.

K-means qualitative results

Our first tests based on the K-means algorithm have generated preliminary results which have been qualitatively analyzed and used to verify the reliability of the method. As shown in the pictures below the output of the program are the processed paintings with the corresponding skeletons identified by colored vertices and by a number. The vertices are those that have been selected by using the interface and their color, as well as the numerical index, represent the cluster the corresponding skeleton belongs to.

Since the database is still not so much substantial the results are not really trustworthy. For this reason only skeletons with an high number of vertices and which are identifying particular poses (i.e. Christ on the cross, dead Christ) can be properly detected and associated always to the same cluster whatever the picture they come from [Fig 1, 2, 3, 4].

However from these pictures we can note that people who are not assuming peculiar poses and have a low number of vertices represent a “noise” in our results. Consequently they identify clusters that gather skeletons which are frequently coming from people standing in different ways.

Dead_Christ_2
Fig. 1: Dead Christ 1
Dead_Christ_1
Fig. 2: Dead Christ 2

 

 

 

 

Christ_cross_2
Fig. 3: Christ on the Cross 1
Christ_cross_1
Fig. 4: Christ on the Cross 2

Encountered issues and possible solutions

The two main problems we run into so far are the number of cluster definition and how to represent the missing point of skeletons. These issues are directly related to the K-means because they both represent inputs parameters of the algorithm.

One of the possible solutions regarding the number of clusters could be a manual determination of it by looking at the pictures and estimating how many different poses should be recognized.

The missing points limitation is the most critical problem we inevitably have to deal with. The lack of just one point in a skeleton could completely falsify the attribution to the right cluster. Two different solutions are proposed. The first one relies on the substitution of the missing point with the same coordinates of the head (origin of the local reference frame) while the second one is based on the average of all the available corresponding coordinates coming from the other skeletons.

Further improvements

With respect to the last presented problem, the idea is to test both the solutions and afterwards keep the one that is performing better. In order to do that the following procedure will be applied: given two different poses A and B, our algorithm will process some complete skeletons of A together with a non-complete one belonging to the same group A and some representing B. To validate the proposed ideas the algorithm should classify the non-complete skeleton in the right cluster A. In addition by computing the training error matrix we can estimate the one that is more effective.

Another possible task for the future will be the study and implementation of different machine algorithms (i.e. spectral clustering) to compare their results with those generated by the K-means.