• Home
• Feasibility test
• Experiment 2
• Experiment 3
• Experiment 4
• Belga images copyright notice

Image annotation using clickthrough data

Theodora Tsikrika, Christos Diou, Arjen P. de Vries and Anastasios Delopoulos,
In Proceedings of the 8th ACM International Conference on Image and Video Retrieval, 8-10 July, Santorini, Greece, 2009

Reliability and Effectiveness of Clickthrough Data for Automatic Image Annotation

Theodora Tsikrika, Christos Diou, Arjen P. de Vries and Anastasios Delopoulos,
Multimedia Tools & Applications, Special issue on Image and Video Retrieval: Theory and Applications, 2010.

Abstract

Automatic image annotation using supervised learning is performed by concept classifiers trained on labelled example images. This work proposes the use of clickthrough data collected from search logs as a source for the automatic generation of concept training data, thus avoiding the expensive manual annotation effort. We investigate and evaluate this approach using a collection of 97,628 photographic images. The results indicate that the contribution of search log based training data is positive. In particular, the combination of manual and automatically generated training data outperforms the use of manual data alone. It is therefore possible to use clickthrough data to perform large-scale image annotation with little manual annotation effort or, depending on performance, using only the automatically generated training data.

Feasibility test: Training with search logs, evaluation on manual annotations

• Feasibility test:

  Training using searchlog generated data, evaluation on the manual annotations. Multiple runs to measure the effect of negative sample selection.

Experiment 2: Training with search logs, common evaluation set.

• Experiment 2:

  Training using only the searchlog generated data. The common evaluation set consists of the images not participating in the training set any of the experiments.

Experiment 3: Training with combination of search logs and manual annotations. Common evaluation set

• Experiment 3:

  Same as experiment 2, but the training data are now the union of the searh-log generated data and the manual annotations.

Experiment 4: Baseline experiment, Training with manual annotations. Common evaluation set

• Baseline:

  Baseline experiment, only manual annotations are used for training.