Progress report post 2

According to previous report, the case study which raised in that article was then tested. We firstly use the semantic query with query key ‘venice san marco basillica’. We then gain a 30 pictures database on this single tourist site

Part 1 Software

In this 30 pictures, we have 21 good pictures (daytime, sunny, high luminance), 5 bad pictures (night or evening, not sunny, low luminance) and 4 irrelevant pictures (other building). To know the robustness of the 3D reconstruction software, we designed several tests for it

figure11Figure 1.1 Example Raw Pictures after semantic query

Test1.  The accuracy of the software

In this test, we filtered bad and irrelevant pictures artificially and only input the good pictures. And we get this picture:

Screen Shot 2014-04-09 at 12.21.13 AM

Figure 1.2 Testing with pure good pictures


  • Pictures always from the same view site. We should try to gain more other view angles other than only the front view. However, this is a global issue for all the buildings or constructions we may focusing on later. The view angles are always limited due to the surrounding of the target.
  • We noticed the problem mentioned above, and we inset one picture of the right side (1 of 21). Not surprisingly, the 3D results shows no features on right side at all. Therefore, in the next test, we will add more side pictures (14 of 28, 50%).


Test2.  The accuracy of the software (Extended database)

Due to the problem on test1, we turn back to the semantic query. We chose a bigger database than before with 28 good pictures, 50% is front view and 50% is right side view. (Left side view isn’t accessible)


Figure1.3 Testing with pure good pictures(Extended pictures)


  • We can see a much better result on the reconstruction. Now we can have both the front and side view together.
  • Another thing we noticed is that background doesn’t influence on the 3D reconstruction a lot. Among those pictures, we have several cloudy pictures (see figure2.1 bottom right). This may indicates a loose threshold on the first-stage filtering by histogram matching.
  • Considering the error-tolerant rate of the classification process, we then added some more bad pictures (see figure 2.1 top ), the result is shown on later test.


Test3. The robustness of the software

In this test, we have more bad and irrelevant pictures, we have 32 pictures as database, 28 good ones as mentioned above, and then 2 night pictures and 2 irrelevant pictures.


Figure1.4 Testing with both good and bad pictures


  • No clear difference between the result with and without bad pictures.
  • A good construction can be achieved on some partial shape of the buildings


Part2 Histogram analysis

In this part we output histogram of four different pictures (Night, Evening, Sunny, Cloudy). Before the histogram output, we convert all the four color picture into gray-level 256.


Figure2.1 Different origin picture


Figure2.2 Histogram of different picture


  • We can easily see great difference among these different
  • When we convert picture into gray-level, the dominant character of the picture will become the luminance and we lost the information for different color.
  • The histogram can be easily influenced by very brightness. This will be used as a feature to filter those bad pictures automatically.

In order to do the classification, we are trying to give out a standard description on the histogram of those good pictures. Therefore, we compared the histogram on several good pictures.


Figure2.3 A comparison of good picture(sunny)


Figure 2.4 Histogram of the comparison pictures


  • According to the picture and also the histogram, we separate the pictures into two kinds, one is for sunny1 and sunny4, which is brighter and have a better sunshine. The other is sunny2 and sunny3, which are between the good sunny and cloudy we mentioned before.
  • For those brighter pictures with very blue sky, the histogram shows a much consistent shape with a good bell-shape and can be extract as a Gaussian Distribution. The distribution of the histogram is more concentrated.
  • Also, those brighter picture have a higher value for the concentrated zone.
  • The blue sky doesn’t necessarily means a good character for candidate picture of 3D construction. But because of the good weather in Venice, the pictures with blue sky are more common than other background. This can be an easy way to filter those bad pictures only by matching the histogram.



Part 3 Next step

As we have already seen the robustness of the 3D reconstruction picture, we may have several problems to be focused in next step:

  1. Setting a good threshold value to compromise the enough number of candidate pictures and less number of bad pictures.
  2. New features also need to be extracted from the information on good picture, then the clustering program will gain a better classification accuracy.
  3. As we mentioned in part1, we may need more methods to gain the pictures on as much view angles as possible. This procedure now is accomplished artificially, which is not efficient enough.