Duanxx的论文阅读:Behavior Recognition via Sparse Spatio-Temporal Feature

  • 格式:pdf
  • 大小:315.92 KB
  • 文档页数:6

Duanxx的论文阅读:Behavior Recognition via SparseSpatio-Temporal Features 基于稀疏时空特征点的运动识别——Duanxx——2015-04-241、InreoductionIn this work we develop a general framework for detecting and characterizing behavior from video sequences, making few underlying assumptions about the domain and subjects under observation. Consider some of the well known difficulties faced in behavior recognition. Subjects under observation can vary in posture, appearance and size. Occlusions and complex backgrounds can impede observation, and variations in the environment, such as in illumination, can further make observations difficult. Moreover, there are variations in the behaviors themselves.在面对运动识别的问题时,各种各样的困难需要考虑,比如说:被观察者的姿态、外观、大小可能会改变;遮挡以及复杂的背景也会妨碍我们的观察;像光照等环境因素也会让观察变得异常的棘手;最要命的是运动自身的变化也是多样的。

本文提出了一种针对视频的通用的运动检测及识别的框架,它只需要对被观察者做很少的一些前提假设就可以很好的应用。

The inspiration for our approach comes from approaches to object recognition that rely on sparsely detected features in a particular arrangement to characterize an object, e.g. [6, 1, 18]. Such approaches tend to be robust to pose, image clutter, occlusion, object variation, and the imprecise nature of the feature detectors. In short they can provide a robust descriptor for objects without relying on too many assumptions这个方法主要是基于以下三篇论文:1:S. Agarwal, A. Awan, and D. Roth. Learning to detect objects in images via a sparse, part-based representation. PAMI, 26(11):1475–1490, Nov 20042:A. Frome, D. Huber, R. Kolluri, T. Bulow, and J. Malik. Recognizing objects in range data using regional point descriptors. In ECCV, 2004.3:B. Leibe and B. Schiele. Scale invariant object categorization using a scale-adaptive mean-shift search. In DAGM, Aug. 2004.简而言之,它可以在不需要太多的前提假设下给出一个鲁棒性强的运动表述算子。

We propose to characterize behavior through the use of spatio-temporal feature points (see figure 1). A spatiotemporal feature is a short, local video sequence such as an eye opening or a knee bending, or for a mouse apaw rapidly moving back and forth. A behavior is then fully described in terms of the types and locations of feature points present.本文通过时空特征点(spatio-temporal feature points)来描述一个行为。

所谓时空特征点就是一些特别短的局部视频序列,比如眨眼睛、曲膝等。

而对一个行为的描述,就是对一些时空特征点的类型和位置的描述。

The motivation is that an eye opening can be characterized as such regardless of global appearance, posture, nearby motion or occlusion and so forth, for example, see figure 2. The complexity of discerning whether two behaviors are similar is shifted to the detection and description of a rich set of features.之所以能有上面的时空特征点的描述算法,是基于运动的局部性,比如在对眨眼睛这个动作进行识别时,我们根本不需要考虑到被观察者全部的外观、姿态、遮挡等等。

于是乎,判断两个运动是否相似的问题就转化为了对一个时空特征点序列的检测和描述的问题了。

3、Proposed Algorithm 算法描述这里详细的描述了本文所使用的算法,算法一共分为四个步骤:3.1. Feature Detection 特征提取关于特征提取的算法和论文是非常的多的,这里提供一篇特征提取的综述:C. Schmid, R. Mohr, and C. Bauckhage. Evaluation of interest point detectors. IJCV, 37(2):151–172, June 2000.其中比较流行的方法如下:●based on the detection of corners 基于角点的检测所谓角点就是局部梯度向量正交的区域而梯度向量又是通过计算已经平滑过的图像的一阶微分得到的:L (x, y, σ ) =I (x, y ) ∗ g (x, y, σ ),其中g是高斯平滑算子,σ控制的是角点的尺度。

The response strength at each point is then based on the rank of the covariance matrix of the gradient calculated in a local window.基于不同点局部的梯度的协方差矩阵的秩,每个点有不同的反应强度。

参考论文如下:1、W. Forstner and E. G¨ulch. A fast operator for detection and precise¨location of distinct points. In Intercommission Conf. on FastProcessing of Photogrammetric Data, pages 281–305, Switzerland, 1987.2、C. Harris and M. Stephens. A combined corner and edge detector. In Proc. Alvey Conf., pages 189–192, 1988●the Laplacian of Gaussian (LoG) and Harris detector LoG算子和Harris的检测参考论文:1、K. Mikolajczyk and C. Schmid. Indexing based on scale invariant interest points. In ICCV, pages I: 525–531, 2001●specific goal of detecting features显著点检测参考论文:1、T. Kadir and M. Brady. Saliency, scale and image description. IJCV, 45(2):83–105, Nov 2001.3.1.1. Extensions to the Spatio-Temporal Case 扩展到时空特征提取The general idea of interest point detection in the spatiotemporal case is similar to the spatial case. Instead of an image I(x, y), interest point detection must operate on a stack of images denoted by I(x, y, t). Localization must proceed not only along the spatial dimensions x and y but also the temporal dimension t. Likewise, detected features also have temporal extent.在时空域的感兴趣点提取和空间域上的提取方式相似,不过并不是仅仅是空间域上的I(x, y)描述,而是一个图像序列的描述:I(x, y, t)。

对时空域上的特征提取除了空间域上的x和y以外,还需要时间t作为其描述的一个维度。

现今已知的时空域的感兴趣点的描述子是3D harris特征提取(an extension of the Harris corner detect to the 3D case)。