Affective and holistic approach at TRECVID 2010 task - Semantic Indexing (SIN)

This paper reports our experiments for TRECVID 2010 task: Semantic Indexing. We present two approaches namely, Affective and Holistic. In the first approach, we have used combination of affective features from image, video and audio trained with neural network algorithm. Image features employed are color histogram and face detection from the keyframe. The number of face is also used in one of the runs. Video features include the motion activity and shot duration. Additionally, the audio power is included as feature. For the second approach, color, texture and scene features are extracted from the whole keyframe image as well as its background and saliency regions. Genetic algorithm is used to find the weight of each feature for effective combination. Then, KNN is used to propagate the annotation. We have submitted 4 runs where we distinguish the first two as affective category and the the last two as holistic ones. The summary is as follows: • kmlabGITS1-color histogram, motion, rhythm, sound and face number trained using neural network • kmlabGITS2-color histogram, motion, rhythm, sound and without face number trained using neural network • kmlabGITS3-combination of 5 image features (hsv bg, gabor, haar, gist and lab bg) using Genetic Algorithm and KNN • kmlabGITS4-combination of 5 image features (hsv, hsv bg, haar, haar roi and gist) using Genetic Algorithm and KNN.