Tsotsos. Hand Gesture Recognition within a Linguistics-Based Framework

A Real-time Continuous Gesture Recognition System for Sign Language

Rung-Huei Liang1, Ming Ouhyoung21Dept. of Information Management, Shih-Chien University, Taichih, Taipei 104, Taiwan, R.O.C.liang@.tw2Dept. of Computer Science and Information Engineering, National Taiwan University, Taipei 106Taiwan, R.O.C.; ming@.tw

AbstractIn this paper, a large vocabulary sign languageinterpreter is presented with real-time continuous gesturerecognition of sign language using a DataGloveTM. Themost critical problem, end-point detection in a stream ofgesture input is first solved and then statistical analysis isdone according to 4 parameters in a gesture : posture,position, orientation, and motion. We have implemented aprototype system with a lexicon of 250 vocabularies inTaiwanese Sign Language (TWL). This system uses hiddenMarkov models (HMMs) for 51 fundamental postures, 6orientations, and 8 motion primitives. In a signer-dependent way, a sentence of gestures based on thesevocabularies can be continuously recognized in real-timeand the average recognition rate is 80.4%.

ComputerEngineeringandApplications计算机工程与应用2018，54（9）

1引言当前，生物特征识别技术[1]由于其便捷性，是人们用来替代传统人工密码认证方式的主要途径。现有的研究人员对基于手部特征的识别技术已经做了大量的研究，主要包括指静脉识别[2]、掌静脉识别[3]、指纹识别[4]、掌纹识别[5]和手型识别[6]等。而针对指纹识别和掌纹识⦾图形图像处理⦾单幅近红外手掌图像掌静脉和掌纹多特征识别李俊林，王华彬，陶亮LIJunlin,WANGHuabin,TAOLiang安徽大学计算机科学与技术学院，合肥230601SchoolofComputerScienceandTechnology,AnhuiUniversity,Hefei230601,ChinaLIJunlin,WANGHuabin,puterEngineeringandApplications,2018,54（9）：156-164.Abstract：Thetraditionalpalmveinandpalmprintfusionrecognitionneedstoobtainthepalmveinimagesandthepalm-printimagesseparately.However,thepalmveinandpalmprintstructureinformationactuallyexistsinthesamenearinfraredpalmimage.Becausethepalmveinandpalmprintstructuresstayinthedifferentrangesofpixelvalues,andtheirlocaltextureinformationisalsodistinct,thepalmveinandthepalmprintcanbeseparatedfromasinglenearinfraredpalmimageandthenenhancedindividually.Firstly,animprovedself-guidedfilteralgorithmisproposedtoremovethepalmprintstructure,andtheanti-fuzzydetailenhancementmodelisdesignedtoenhancetheimageofthepalmveinstructure.Then,animprovedblockenhancementalgorithmisproposedtoenhancethepalmprintstructureandremovethepalmveinstructure.Besides,theSobeloperatorunsharpmaskingalgorithmisusedtohighlightthemainlinestructureofthepalmprint.Finally,arecognitionalgorithmispresentedbasedontheimagefusionofpalmveinandpalmprint.ExperimentsareimplementedonthemultispectralpalmprintdatabaseofHongKongPolytechnicUniversity,andtheexperimentalresultsshowthatthefusionrecognitionratehasreached99.63%.Comparedwiththeotherexistingalgorithms,theequalerrorrateoftheproposedalgorithmisreducedby0.66%inaverage,whichvalidatestheeffectivenessoftheproposedalgorithm.Keywords：palmvein;palmprint;fusionrecognition;guidedfilter摘要：传统的掌静脉和掌纹图像融合识别一般需分别采集掌静脉和掌纹两类图像，而单幅近红外手掌图像中实际上同时包含了掌静脉和掌纹结构信息。由于二者局部纹理细节差异较大，且像素值分布范围不同，因此，可以先分离再分别增强处理。首先，提出了改进的引导滤波算法以便去除掌纹结构，并设计了反模糊细节增强模型增强掌静脉结构图像；然后，提出了一种改进的分块增强算法，可以在增强掌纹结构图像的同时滤除掌静脉结构信息，再利用基于Sobel算子的反锐化掩模算法以便突出掌纹主线条结构信息；最后，对单幅近红外手掌图像中获取的掌静脉和掌纹图像进行融合识别。在香港理工大学近红外手掌数据库上进行了实验，结果表明：所提出的算法识别率达到了99.63%，与其他已有算法相比等误率平均降低了0.66%，验证了所提出算法的有效性。关键词：掌静脉；掌纹；融合识别；引导滤波文献标志码：A中图分类号：TP391.4doi：10.3778/j.issn.1002-8331.1612-0068

基于Leap Motion传感器的自适应动态手势识别

刘权;陈一民;高明柯;李启明;黄晨

【期刊名称】《计算机应用与软件》

【年(卷),期】2017(034)002

【摘 要】基于视觉的手势识别是实现新型人机交互的一项关键技术,针对手势普适性与识别率问题,在改进隐马尔可夫模型学习机制的基础上提出一种新的基于Leap

Motion传感器的自适应动态手势识别方法.该方法首先采用几何特征法识别静态手姿以确定动态手势起始点与结束点,然后基于角度对动态手势轨迹进行特征提取与分类,并引入修正的重估方法计算隐马尔可夫模型参数,最后在对非定义手势识别的基础上自动学习更新隐马尔可夫模型,以提高动态手势识别率,并最终实现对26个小写字母的动态手势识别.实验结果表明,所提出的动态手势识别方法具有良好的自适应性与精确性.%Gesture recognition based on vision is a key technology

to realize new human-computer interaction.To the problem of gestures

adaptability and recognition rate,a new method of adaptive dynamic

gesture recognition based on the Leap Motion is presented on the basis of

improving Hidden Markov Model (HMM).Firstly,by using the method of

geometric features to recognize the static hand posture,the start and end

point of dynamic gesture trajectory could be confirmed.After extracting

III

ABSTRACT

Dynamic hand gesture recognition is a crucial but challenging task in the pattern

recognition and computer vision communities. In this paper, we propose a novel feature

vector which is suitable for representing dynamic hand gestures, and present a

satisfactory solution to recognizing dynamic hand gestures with a Leap Motion

controller (LMC) only. These have not been reported in other papers. The feature vector

with depth information is computed and fed into the Hidden Conditional Neural Field

(HCNF) classifier to recognize dynamic hand gestures.

The systematic framework of the proposed method includes two main steps: feature

extraction and classification with the HCNF classifier. The proposed feature vector that

consists of single-finger features and double-finger features has two main benefits. First,