Fuzzy-GIST for Emotion Recognition in Natural Scene Images

  • 格式:pdf
  • 大小:494.37 KB
  • 文档页数:7

2009 IEEE 8th International Conference on Development and Learning 978-1-4244-4118-1/09/$25.00 ©2009 IEEE Abstract—Emotion modeling evoked by natural scenes is challenging issue. In this paper, we propose a novel scheme for analyzing the emotion reflected by a natural scene, considering the human emotional status. Based on the concept of original GIST, we developed the fuzzy-GIST to build the emotional feature space. According to the relationship between emotional factors and the characters of image, L*C*H* color and orientation information are chosen to study the relationship between human’s low level emotions and image characteristics. And it is realized that we need to analyze the visual features at semantic level, so we incorporate the fuzzy concept to extract features with semantic meanings. Moreover, we treat emotional electroencephalography (EEG) using the fuzzy logic based on possibility theory rather than widely used conventional probability theory to generate the semantic feature of the human emotions. Fuzzy-GIST consists of both semantic visual information and linguistic EEG feature, it is used to represent emotional gist of a natural scene in a semantic level. The emotion evoked by an image is predicted from fuzzy-GIST by using a support vector machine, and the mean opinion score (MOS) is used for performance evaluation for the proposed scheme. The experiments results show that positive and negative emotions can be recognized with high accuracy for a given dataset.

Index Terms—Emotion modeling, fuzzy-GIST, fuzzy C-means clustering, support vector machine

I. INTRODUCTION RTIFICIAL emotion research focuses on how to make a machine capable of recognizing and understanding the human emotions and expressing its own emotional status. It is an important facet to build the real humanoid robot which is intelligent, autonomous and interactive. Empirical research on emotion is characterized by a wide variety of methodologies. The presentation of speech signals with different emotional expressions is one of the common strategies. Recently, more and more researchers have been paying attention to the visual influences of images on human emotion, as in [1]-[4], because visual information plays a very important role to affect a subject’s emotional status. It can be positive or negative depending on whether a concern is advanced or impeded, respectively [5]. However, most of conventional image processing ignores emotional factors, which can help us to describe and simulate the human feedback

Q. Zhang is a Ph.D. candidate with the School of Electrical Engineering and Computer Science, Kyungpook National University, 1370 Sankyuk-Dong, Puk-Gu, Taegu 702-701, South Korea (email: zhangqing@ee.knu.ac.kr). M. Lee is a professor with the School of Electrical Engineering and Computer Science, Kyungpook National University, 1370 Sankyuk-Dong, Puk-Gu, Taegu 702-701, South Korea (corresponding author to provide phone: 82-53-950-6436; fax: 82-53-950-5505; e-mail: mholee@ knu.ac.kr)

of the natural scene images. Accordingly, a lot of research was done with the presentation of faces with different facial expressions. It is clear that emotions cannot be exhaustively apprehended using only facial expressions [6], but also can be affected by natural scenes. Understanding mechanism of emotion in natural scene must be developed for a robot to understand human emotion in various environments. Considering all of the above, the motivation for this study is to build a system that can analyze and understand the emotions in natural scene images under the supervision and interaction with human. One of the difficulties to build this human emotion understanding system is feature selection process. The features we need here should reflect the emotional factors of natural scene. Some of them can come from the subject’s response while others can be extracted from the scene image. In term of the subject’s response analysis, we need to start with the simple idea that we could envision a mind (or brain) as composed of many different “resources” [7]. Each one is responsible for certain specialized jobs. This can help us understand how a mind could make changes in its state, such as emotional states. For example, the state labeled “angry” could be what happens when people activate some resources that help them react with more speed and strength—while also suppressing some other resources that make them act prudently. Based on this concept, it is necessary to review some methods in brain computer interface (BCI) research, such as EEG observation, through which we can record the electrical activity of the brain. The best known correlates of emotionality found with EEG involve prefrontal asymmetry—that is, positive affect is associated with greater activity in the left prefrontal region than in the right side, and negative affect with the reverse [8]. When a human subject is stimulated by a natural scene, his or her EEG signals will give a pattern to indicate the emotion status and it will be used as one part to construct the emotional feature space. Besides, human cognition can be induced by some stimulative factors of the image [9]; therefore, some features related to the emotion reflected by the scene should also be considered as another part of emotional feature space. Here we take color and orientation information into consideration. And because of the smooth changing in term of human perception, L*C*H* space is selected. In order to construct emotional space in a semantic level, the system needs to incorporate human expertise. To do this, we use fuzzy C-means clustering (FCM) to assign one natural image into several different groups to a degree specified by a membership grade [10]. The FCM is used to cluster each component to get different emotional descriptors, and these descriptors are combined together to formalize the fuzzy-GIST