binocular stereo Vision

基于双目视觉机器人自定位与动态目标定位卢洪军【摘要】Aiming at the fact that, the mobile robot based on binocular vision is very easy to be disturbed by the complex environment, such as the influence of noise, illumination change and the occlusion of the robot, which will seriously affect the positioning accuracy of the self localization and the moving objects, the color feature of the HSV model is proposed to accurately segment the artificial landmarks, and the robot position is determined according to the principle of parallax.A method was proposed based on Harris operator which is accurate to the position of a moving object in a complex environment.The dynamic object is detected by the frame difference method.Harris operator was used to extract the feature points on the moving objects, so as to obtain the disparity value, and then to calculate the position of the moving objects.The experimental results show that the self localization and target localization can overcome the external disturbance and have strong adaptability by using this method.The algorithm has good real-time performance.%针对基于双目视觉自定位与动态目标定位极易受复杂环境(如噪声、机器人发生遮挡、光照变化等)的干扰导致移动机器人定位精度低的问题,提出基于HSV颜色模型特征准确分割出人工路标,根据视差原理确定机器人位置.同时提出一种双目机器人基于Harris算子实现在复杂环境下对动态目标精确定位的方法,利用帧间差分法将运动目标检测出来,采用Harris算子在该运动目标上提取特征点,并获得视差值,从而精确的计算出运动目标的位置.实验结果表明,利用该方法进行自定位与目标定位能够克服外界干扰,具有较强的适应性,且算法的实时性好.【期刊名称】《沈阳大学学报》【年(卷),期】2017(029)001【总页数】6页(P37-42)【关键词】双目视觉;目标定位;Harris算子;帧间差分法;HSV模型【作者】卢洪军【作者单位】沈阳工业大学信息科学与工程学院, 辽宁沈阳 110870【正文语种】中文【中图分类】TP391.420世纪末,对目标定位技术主要有基于红外线的定位技术、基于超声波的定位技术和基于频射识别技术等[1].近年来,由于图像处理和计算机视觉的飞速发展,机器视觉的研究越来越受到广大专家和学者的青睐[2].双目立体视觉是机器视觉的一个重要分支,能够直接模仿人类双眼处理外界环境[3],可以代替人类完成危险的工作(如深海探测、火灾救援、核泄漏监测等)[4].而基于双目立体视觉对动态目标检测与定位也是机器视觉领域备受关注的前沿课题之一[5].双目立体视觉定位主要分为六个步骤[6]:①图像获取;②图像预处理;③摄像机标定;④特征点提取;⑤特征点的立体匹配获取视差值;⑥基于视差原理实现机器人定位.特征点提取和立体匹配是实现机器人定位的关键环节.通常的方法是依靠目标的形状、颜色等特征检测目标,并以运动物体的形心或中心作为特征点[7].该方法虽然计算简单,但极易受噪声干扰,只选择一个点作为特征点,一旦该特征点发生遮挡或光照变化等,都会严重影响定位精度.1977年,Moravec提出根据图像的灰度变化来提取图像角点,称为Moravec角点[8].该方法计算相对简单,但对于处于边缘上的点会存在误检,也极易受光照变化的影响.SIFT特征点[9]和CenSurE特征点[10]虽然对尺度、亮度变化不敏感,但在弱纹理等复杂情况下难以提取稳定的特征点,算法复杂度高,计算时间较长.不满足移动机器人对实时性的要求.针对以上缺陷,本文首先利用帧间差分法检测出运动目标,然后在运动目标上基于Harris算法提取多个特征点来实现移动机器人在复杂环境下实时的对运动目标精确定位.机器人整体定位流程如图1所示,移动机器人首先基于HSV颜色模型空间分割出人工路标,实现机器人自定位.然后利用帧间差分法检测出运动目标,根据Harris算法在左右两幅图像上提取特征点,根据区域匹配原理获取视差值,利用视差原理即可求出运动目标的世界坐标,即完成了对运动目标的定位.1.1 人工路标检测(1) HSV颜色模型.RGB色彩空间分别以红色、绿色、蓝色为三原色,通过适当的搭配可以合成成千上万种颜色,是一种常见的颜色表示法.但是RGB色彩空间与人眼的感知差异大,其空间的相似不代表实际颜色的相似.为了能够更准确分割出人工路标,本文采用HSV色彩空间颜色模型,如图2所示.RGB颜色空间转化到HSV色彩空间只是一个简单的非线性变换,计算简单.HSV模型中H代表色调,S代表饱和度,并且独立于亮度信息V.色调H代表颜色信息,取值范围为0～180°,对其设定阈值可以区分不同颜色的路标;饱和度S表示颜色中掺杂白色的程度,取值范围为0～1,S 越大,颜色越深;亮度V表示颜色的明暗程度,取值范围为0～1,V越大,物体亮度越高.(2) 基于颜色特征提取人工路标.由于本文是在室内环境下对移动机器人定位,所以本文设计的人工路标是由红黄蓝三种颜色组成的矩形纸板.如图3a所示为左摄像机拍摄到的带有人工路标的室内环境.根据HSV颜色模型对H、S、V三个分量进行阈值设置即可分割出人工路标,如图3b所示.然后利用图像处理中的形态学操作对分割出的路标进行完善使其效果最佳,如图3c所示.图3d为获取人工路标的中心点,利用视差原理即可得到当前帧机器人的位置.1.2 帧间差分法帧间差分法[11]的思想是对一段连续视频的相邻两帧进行差分运算,从差分运算的结果中得到运动目标的轮廓.该算法的优点是实现简单,对光照变化不敏感,稳定性好.适用于多目标或背景变化较快的场合.图4为在室内环境下用帧间差分法检测到运动物体.结果显示,帧间差分法能够有效的将运动目标检测出来.2.1 双目立体视觉测距原理双目立体视觉的视差原理[12]是利用两台摄像机从两个视点观察同一景物,以获取在不同视角下的感知图像,通过计算空间点在两幅图像中的视差来获取目标物体的三维坐标.2.2 Harris角点检测Harris角点[13]是在Moravec角点的基础进行改进的算法. Harris算子是用高斯函数代替二值窗口函数, 对离中心点越远的像素赋予越小的权重, 以减少噪声的影响. 高斯函数如式(1)所示.Moravec算子只考虑了四个方向的像素值,Harris算子则用Taylor展开式去近似任意方向.图像I(x,y)平移(Δx,Δy)可以一阶近似为在图像I(u,v)中,像点(u,v)平移(Δx,Δy)后的自相关函数为将式(2)代入式(3)可得:其中M如下所示:根据式(5)中矩阵M的特征值可以近似的表示函数C(x,y)的变化特征.矩阵M的特征值需要考虑以下三种情况,如图5所示.(1) 如果矩阵M的两个特征值都比较小,则表征图像灰度变化函数C(x,y)的值也较小,就说明该像素点的邻域内灰度差值不大,图像较平滑,无角点.(2) 如果矩阵M的两个特征值一个较大,一个较小,说明该像素点的曲率也是如此,则该点的窗口区域处于一条边界,无角点.(3) 如果矩阵M的两个特征值都比较大,则图像灰度变化的自相关函数值也较大,该点的窗函数沿任意方向都将引起灰度的剧烈变化,该点即为角点.根据这一准则,只要计算行列式的特征值就可以检测图像中的哪些点是角点.Harris 提出角点的响应函数:det(M)为行列式的值,trace(M)为行列式的迹.而k根据Harris的建议一般取0.04～0.06之间.若Harris角点响应大于阈值,则被认为是角点.Harris角点的生成只涉及到一阶导数,所以该角点对噪声影响、光照变化引起的灰度值变化都不敏感,是一种比较稳定的特征提取算子.3.1 实验环境本文使用的机器人是由北京博创兴盛技术有限公司开发的自主移动机器人旅行家Ⅱ号,如图6所示.该机器人上安装了由加拿大Point Grey Research公司生产的Bumblebee2双目摄像机,其性能参数如表1所示.3.2 传统移动机器人对运动目标定位实验环境为一间办公室,装有双目摄像机Bumblebee2的移动机器人为工作机器人,用于检测运动目标.将另一台机器人视为运动目标,运动速度为0.1 m/s.传统的方法是提取运动目标的中心点,获取视差值,从而给运动目标定位.传统方法仅获取图像中的一个点作为立体匹配的特征点,当该点受到环境的干扰时定位精度会受到极大的影响,图7为传统方法提取运动目标中心点.表2为传统方法对运动目标定位的实验数据,表3为改变光照后传统方法移动机器人对运动目标定位的实验数据.可以得出传统方法机器人定位误差相对较大,一旦光照发生改变,对运动物体定位误差会更加严重.3.3 基于Harris算子机器人对运动目标定位针对传统方法定位精度不足,极易受外界环境的干扰的问题,决定采用基于Harris角点特征提取,即在相机获得的左右两幅图像上基于Harris算子提取多对特征点,如图8所示.表4、表5为基于Harris方法机器人对运动目标定位的实验数据,可以得出基于该方法对运动目标定位误差很小,相对误差降低到1%左右,当光照发生变化时也能实现对运动目标精确定位.最后将每一帧的两幅图像根据区域匹配原理[14]和极限束准则找到正确的匹配点,排出易受噪声干扰的点,从而得到视差值,即可准确的对运动目标定位.(1) 本文研究了机器人基于双目立体视觉实现自定位与对运动目标定位,充分利用双目视差原理,并结合Harris算法和帧间差分法来实现运动目标的精确定位.从仿真结果可以看出,提取多个特征点可以避免只用一个点易受干扰的不足,实现更精确的运动目标定位.(2) 虽然本文在运动目标上提取多个特征点,有效的克服了传统方法的不足.但还存在问题需要改进.首先,需要找到一种更快更准确的特征点立体匹配算法;其次,本文只是将每一帧图像得到的多个视差值做平均值处理,如何有效的将多个视差值融合也是对运动目标精确定位的关键.【相关文献】[1] 李艳. 双视频目标定位技术[J]. 沈阳大学学报(自然科学版), 2016,28(4):302-305. (LI Y. Dual video target location technology[J]. Journal of Shenyang University(Natural Science), 2016,28(4):302-305.)[2] 李天健. 基于机器人视觉的汽车零配件表面缺陷检测算法研究与实现[J]. 沈阳大学学报(自然科学版), 2013,25(6):476-480. (LI T J. Research and implementation of auto parts surface defect detection algorithm bases on robot visio[J]. Journal of Shenyang University (Natural Science), 2013,25(6):476-480.)[3] 杜宇. 三维重建中双目立体视觉关键技术的研究[D]. 哈尔滨:哈尔滨理工大学, 2014:1-5. (DU Y. Research on key technology of binocular stereo vision in three-dimensional reconstruction[D]. Harbin:Harbin University of Science and Technology, 2004:1-5.)[4] 余俊. 基于双目视觉的机器人目标检测与控制研究[D]. 北京:北京交通大学, 2011:1-4. (YU J. Research on target detection and robot control based on binocular vision[D]. Beijing: Beijing Jiaotong University, 2011:1-4.)[5] DESOUZA G N, KAK A C. Vision for mobile robot navigation: A survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002,24(2): 237-267.[6] 高栋栋. 基于双目立体视觉定位和识别技术的研究[D]. 秦皇岛:燕山大学, 2013:9-11. (GAO D D. Research on recognizing and locating binocular stereo vision technology[D]. Qinhuangdao:Yanshan University, 2013:9-11)[7] 崔宝侠,栾婷婷,张弛,等. 基于双目视觉的移动机器人运动目标检测与定位[J]. 沈阳工业大学学报, 2016,38(4):421-427. (CUI B X, LUAN T T, ZHANG C, et al. Moving object detection and positioning of robot based on binocular vision[J]. Journal of Shenyang University of Technology, 2016,38(4):421-427.)[8] 邓国栋. 基于多尺度特征的双目立体视觉目标定位[D]. 哈尔滨:哈尔滨工业大学, 2012: 21-22. (DENG G D. Object location of binocular stereo vision base on multi-scale feature[D]. Harbin: Harbin Institute of Technology, 2012:21-22.)[9] LOWE D G. Distinctive image feature from scale-invariant key point[J]. International Journal of Computer Vision, 2004,60(2):91-110.[10] KONOLIGE K, AGRAWAL M, SOLA J. Large-scale visual odometry for roughterrain[C]∥Robotics Research: The 13thInternational Symposium ISRR, 2011,66:201-212.[11] 熊英. 基于背景和帧间差分法的运动目标提取[J]. 计算机时代, 2014(3):38-41. (XIONG Y. Moving object extraction based on background difference and frame differencemethod[J]. Computer Era, 2014(3):38-41.)[12] 林琳. 机器人双目视觉定位技术研究[D]. 西安:西安电子科技大学, 2009:8-10. (LIN L. The research of visual positioning technology on the binocular robot[D]. Xi’an: Xi'an Electronic and Science University, 2009:8-10.)[13] 张从鹏,魏学光. 基于Harris角点的矩形检测[J]. 光学精密工程, 2014,22(8):2259-2266. (ZHANG C P, WEI X G. Rectangle detection base on Harris corner[J]. Optics and Precision Engineering, 2014,22(8):2259-2266.)[14] 罗桂娥. 双目立体视觉深度感知与三维重建若干问题研究[D]. 长沙:中南大学, 2012:48-53. (LUO G E. Some issues of depth perception and three dimension reconstruction from binocular stereo vision[D]. Changsha: Central South University, 2012:48-53.)。

7th International Conference on Energy and Environmental Protection (ICEEP 2018)Spatial position and posture of component detectionsystem based on binocular stereo visionLiuyuan1,a* Feiyunfeng2,b Wang wei1,c Songyinhao2,d1 Xi’an Research Institute of High Technology, Xi’an 710025, Shaanxi Province, China2 Beijing Special Engineering Design Institute, Beijing 100011, Chinaa********************,b******************,c**********************,d**************** Keywords: binocular stereo vision, position and posture detection, automatic dockingAbstract. With the continuous improvement of the manufacturing level in the aerospace, automobile, shipbuilding and other fields, higher requirements for the assembly precision and speed of the components in the manufacturing process are raised. However, the manual operation method is generally adopted in the work of large mechanical parts assembly in our country. Through the visual feedback of the human eye to the operation interface, the labor intensity is large, the efficiency is low, and the assembly cycle is long. To solve this problem, spatial position and posture of object detection system based on binocular stereo vision is proposed. On the basis of the predecessors, the traditional calibration method is improved, the calibration of the binocular camera is realized and the position and position deviation of the component is measured by the binocular camera. The difference of the position detection system of the binocular vision is less than 0.1mm in the three space positions, and the difference between the three rotation angles is less than 0.15 degrees. It lays the foundation for automatic docking of components.IntroductionIn recent years, the level of aerospace, automotive, shipbuilding and other fields, which is closely related to the manufacturing industry, has developed rapidly in the design and manufacture of large mechanical parts. With the continuous improvement of the factory manufacturing level, the assembly accuracy and speed of components in the manufacturing process are higher. But at present, the work of large mechanical parts assembly in our country mainly depends on manual completion, that is to put each part on the bracket or trailer platform first, then according to the bolts and screw holes at different positions on the combination surface of each component, and rely on a number of workers to push one of the parts slowly close to another part, and pass through continuously. Observe and adjust the relative position of bolt and screw hole to complete the alignment and insertion operation of components. Moreover, the docking effect of this way often depends on the experience and operation level of the personnel. The docking accuracy is low, the installation speed is slow, and the operation stability is poor.With the development of machine vision measurement technology [1-3], laser tracking measurement technology [4,5] and robot technology [6] and so on, automatic docking has become an inevitable trend of development [7], in the docking of large and irregular large mechanical parts. Binocular vision is an important form of machine vision. It is simple, reliable, flexible and widely used. In this paper, binocular vision position and position detection technology is used to propose a binocular vision component space pose detection method, which lays the foundation for automatic docking of components.Construction of binocular vision position detection systemStructure model of binocular stereo vision systemThe structure model of the binocular vision system is shown in Fig. 1. 1o and2o are the optical centers of lenses 1 and 2 respectively. The distance between them is called the baseline andrepresented by B. 1O and 2O are the center of the left and right image planes, respectively. Set two CCD camera horizontally, use CCD1 and CCD2 to represent the image plane of the left and right cameras respectively, and the left and right image coordinate systems are 111Y X O −and 222Y X O −respectively. 11o O is the left camera axis and 22o O is the right camera axis. The spatial points P are respectively imaging on the left and right image plane, and the coordinates corresponding to the image points are ()111,Y X P and ()222,Y X P respectively. It is now used xyz o −1 as the coordinate system of the binocular system. The effective focal length of the camera is 1f and 2f respectively. The angle of the optical axis and the baseline are 1α and 2αrespectively. The horizontal projection angles of the space point P are 1ω and 2ωrespectively. The vertical projection angles are 1φ and 2φ respectively. According to the geometric structure relations, the coordinates of the P points can be obtained as follows:+===+=212211211cot cot sin tan sin tan cot cot cot θθθφθφθθθB z z z y B x （1） 111ωαθ+=，222ωαθ+= （2） 111cos /tan ωφY =，222cos /tan ωφY = （3） ()111/arctan f X =ω，()222/arctan f X =ω （4） The structure parameters of the system constitute the important parameters of the system. By analyzing the error composition of the above formula and the precision requirements of the actual assembly, the configuration of the binocular system can be determined.Fig. 1 Structure Model of binocular stereo vision systemExperimental verification Based on the above theoretical analysis, we build a dual vision component space pose detection system. The main equipment is: industrial camera (Microvision MV-EM200M), industrial lens (Computar M1614- MP2), MITSUBISHI manipulator (RV-3SQ), linear guide rail (single degree of freedom guide, precision =0.01mm). According to the equations(1) to (4), the camera hassymmetrical distribution, the camera's baseline distance is 360mm, and the angle between the camera and the baseline is 67.5°.The realization of calibrationThe traditional camera calibration method can be used for any camera model with high calibration accuracy, but the calibration process is complex and requires expensive high precision calibration equipment. Zhang has proposed a simple, flexible and practical camera calibration method [8]. This paper has made improvements on the basis of previous predecessors, and realized the calibration of the binocular camera. The total size of the calibration board is 100mm*100mm, the checkerboard formation area is 70mm*60mm, and the checkerboard size is 10mm*10mm, the error is 20μm. There are 30 inner corners in the whole checkerboard. Before each shot, it is necessary to change the position and posture of the calibration board until 20 pictures are taken, then the calibration button on the interface is used to get the inside and outside parameters of the camera, and the calibration work is completed. The camera calibration interface is shown in Fig. 2.Fig. 2 The camera calibration interfaceAfter calibration, the internal matrices of camera 1 and camera 2 are (5) and (6) respectively, and the binocular matrix R and T matrix are (7) and (8) respectively.(5) (6)(7)13717.66,0,788.130,3717.35,627.7150,0,1R =23735.48,0,767.9730,3734.93,604.1720,0,1R= -0.03079904083105934,0.007236546355386415,0.99949940044040710.01628958022339765,0.9998446185183378,-0.006737091068492179-0.9993928500144637,0.01607392972381081,-0.03091213553285763R =(8)Measurement component of position and posture error of six degrees of freedom in spaceIn order to verify the measurement accuracy of the position and posture detection system based on binocular vision, the error of the measurement results on the space six degree of freedom is calculated by the experimental method. The error distribution of binocular measuring system in three directions of x, y and z is drawn by comparison of linear guide rail (single degree of freedom guide rail and precision =0.01mm). The error of the binocular measurement system on three rotational degrees of freedom (pitch angle, roll angle, yaw angle) is used in the experiment. In the experiment, the posture of the calibrated plate is used as the detection object, and the result of the posture calculation of the calibrated plate obtained by the high precision manipulator is taken as the reference value, and the binocular vision is compared with the binocular vision measurement results. After 100 time measurements, the difference of the position detection system of the binocular vision is less than 0.1mm in the three space positions, and the difference between the three rotation angles is less than0.15 degrees. Fig.3 is the experimental scene when the error is measured. Fig. 4 is the result of the test.(a)Position error detection (b) Posture error detection Fig. 3 The error measured experimental sceneFig. 4 The error detection result of the position and posture.ConclusionThis paper makes a theoretical study and experiment on the position and posture detection system based on binocular stereo vision. The difference of the position detection system of the binocular vision is less than 0.1mm in the three space positions, and the difference between the three rotation-216.80184487031451.314678751945799233.6980553202565T =angles is less than 0.15 degrees. It is proved that the design method of the system is simple and the structure is easy to be realized. It can detect the position and pose of the components in high precision, which has a good prospect of application.Reference[1] Chen X. N., Guo J., Yao Y. Y. Sichuan Journal of ordnance industry, 34 (2013),P.99 (in Chinese).[2] Gou W. T. Xie W. Q. Journal of Ordnance Equipment Engineering, 2 (2016),P.105 (in Chinese).[3] Li J. Journal of Modern Optics, 39 (2017),P.1.[4] Li J. Q., Zhao R. H., Chen J. L., et al. Iet Science Measurement & Technology, 10(2016),P.704.[5] Cattini S., Rovati L. Instrumentation and Measurement Technology Conference Proceedings. IEEE, 2016, P.1.[6] Mergler H. IEEE Journal on Robotics & Automation, 3(2003),P.87.[7] Huang G. Q., Mak K.L. Computers in Industry ,38(1999),P.17.[8]Lin D.M. Zhang A., Wang P., et al. Journal of Lanzhou University of Technology, 42 (2016),P.78 (in Chinese)。

2021年VR技术应用竞赛理论测试卷及答案一、单项选择题（1-600题，考试时从题库中随机抽60题组卷，每题1分）模块一、多媒体作品制作员试题（第1题~第50题）1.职业道德是指从事一定职业劳动的人们，在长期的（）中形成的行为规范。

A.职业生涯B.社会活动C.职业规范D.职业活动(正确答案)2.在商业活动中，不符合待人热情要求的是（）。

A.严肃待客，不卑不亢(正确答案)B.主动服务，细致周到C.微笑大方，不厌其烦D.亲切友好，宾至如归3.树立对职业道德的认识:培养职业道德情感是（）的内容。

A.职业道德修养(正确答案)B.职业道德教育C.职业道德准则D.职业素质4.国家在职业道德建设上积累了很多经验，我们需要特别重视的是（）。

A.加强职业道德的立法工作B.注重信用档案体系的建立C.重视员工爱国意识的培养(正确答案)D.严格的岗前和岗位培训5.职业纪律是企业的行为规范，职业纪律具有（）的特点。

A.明确的规定性(正确答案)B.高度的强制性C.普遍适用性D.自愿性6.下列四个选项中，（）在我国法律体系中处于最低位置。

A.行政规章(正确答案)B.行政法规C.刑法D.法律7.我国制定的约束计算机在网络上行为的法律法规是（）。

A.计算机软件保护条例B.计算机联网规则C.计算机信息网络国际联网安全保护管理办法(正确答案)D.中华人民共和国计算机安全法8.下列关于勤劳节俭的论述中，不正确的选项是（）。

A.勤劳节俭是促进经济和社会发展的最终动力B.勤劳是现代市场经济需要的，而节俭则不宜提倡(正确答案)C.勤劳和节俭符合可持续发展的要求D.节俭有利于节省资源，但与提高生产力无关9.关于专利权人的权利，表述不正确的是（）B.许可他人实施专利的权利A.实施其专利的权利C.接受推广和应用的权利(正确答案)D.禁止他人实施其专利技术的权利10.《中华人民共和国劳动法》主要调整的对象是( C. ).A.劳动合同关系(正确答案)B.与劳动关系密切的所有关系C.劳动关系D.劳动收入问题11.合同员工违犯职业纪律，在给其处分时应把握的原则是（）。

智能车辆基于双目视觉系统障碍物识别的研究张继晨（武汉理工大学汽车工程学院；汽研1202班；学号：1049721202240）摘要：随着汽车的普及，交通运输问题日益严重，近年来人们对汽车的智能性的要求越来越迫切。

与此同时，以实现无人、全自动和安全驾驶为最终目标的智能车辆今年来发展较为迅速，其关键技术之一是目标障碍物自动识别。

本文主要探讨利用双目视觉目标识别技术，来进行车辆前方障碍物的自动识别。

此外指出任何一种有效的障碍物检测系统不能只依靠单一传感器进行环境感知,因此利用多种传感器信息融合技术检测智能车辆前方障碍物,是未来该领域的研究重点与难点。

关键词：智能汽车；双目视觉；障碍物识别；信息融合The Study of Obstacle Detection Technology Based on BinocularStereovision for Intelligent VehicleZhang Jichen(School of Automobile Engineering, Wuhan University of Technology, Class: 1202, Number: 1049721202240)Abstract: With the popularization of the vehicle, traffic problem has increasingly become a hot problem. In recent years, people are expecting a type of cars which can be intelligent. At the same time, the realization of completely automatic and safe driving intelligent vehicles is a new rapidly developed research area in recent years, and the key technology of it is automatically detect the obstacles. This article mainly aims at automatically detect the obstacles in front of the vehicle by use of the binocular stereovision obstacle detection technology. In addition, the obstacle can not be detected based on only one kind of sensor, many kinds of information from different sensors must be employed. So the research focus in obstacle detection for intelligent vehicles is located at multi-sensors data fusion.Key words: intelligent vehicle; binocular stereovision; obstacle detection; data fusion引言目标识别技术作为图像理解领域研究的一个重要分支，是利用计算机自动分析和提取原始图像中的前景信息，也就是模糊识别、数据处理等理论，模仿人的视觉系统感知外部世界各种形状的一项技术。

双目视觉成像原理1.引言双目立体视觉(Binocular Stereo Vision)是机器视觉的一种重要形式，它是基于视差原理并利用成像设备从不同的位置获取被测物体的两幅图像，通过计算图像对应点间的位置偏差，来获取物体三维几何信息的方法。

融合两只眼睛获得的图像并观察它们之间的差别，使我们可以获得明显的深度感，建立特征间的对应关系，将同一空间物理点在不同图像中的映像点对应起来，这个差别，我们称作视差(Disparity)图。

双目立体视觉测量方法具有效率高、精度合适、系统结构简单、成本低等优点，非常适合于制造现场的在线、非接触产品检测和质量控制。

对运动物体（包括动物和人体形体）测量中，由于图像获取是在瞬间完成的，因此立体视觉方法是一种更有效的测量方法。

双目立体视觉系统是计算机视觉的关键技术之一，获取空间三维场景的距离信息也是计算机视觉研究中最基础的内容。

2.双目立体视觉系统立体视觉系统由左右两部摄像机组成。

如图一所示，图中分别以下标L和r标注左、右摄像机的相应参数。

世界空间中一点A(X，Y，Z)在左右摄像机的成像面C L和C R上的像点分别为al(ul，vl)和ar(ur，vr)。

这两个像点是世界空间中同一个对象点A的像，称为“共轭点”。

知道了这两个共轭像点，分别作它们与各自相机的光心Ol和Or的连线，即投影线alOl和arOr，它们的交点即为世界空间中的对象点A(X，Y，Z)。

这就是立体视觉的基本原理。

图1：立体视觉系统3.双目立体视觉相关基本理论说明3．1 双目立体视觉原理双目立体视觉三维测量是基于视差原理，图2所示为简单的平视双目立体成像原理图，两摄像机的投影中心的连线的距离，即基线距为b。

摄像机坐标系的原点在摄像机镜头的光心处，坐标系如图2所示。

事实上摄像机的成像平面在镜头的光心后，图2中将左右成像平面绘制在镜头的光心前f处，这个虚拟的图像平面坐标系O1uv的u轴和v 轴与和摄像机坐标系的x 轴和y 轴方向一致，这样可以简化计算过程。

视差Disparity与深度图
双目立体视觉，在百度百科里的解释是这样解释的：双目立体视觉（Binocular Stereo Vision）是机器视觉的一种重要形式，它是基于视差原理并利用成像设备从不同的位置获取被测物体的两幅图像，通过计算图像对应点间的位置偏差，来获取物体三维几何信息的方法。

一、视差Disparity与深度图
提到双目视觉就不得不提视差图：双目立体视觉融合两只眼睛获得的图像并观察它们之间的差别，使我们可以获得明显的深度感，建立特征间的对应关系，将同一空间物理点在不同图像中的映像点对应起来，这个差别，我们称作视差(Disparity)图像。

对于视差的理解自己可以体验一下：将手指头放在离眼睛不同距离的位置，并轮换睁、闭左右眼，可以发现手指在不同距离的位置，视觉差也不同，且距离越近，视差越大。

那么提到视差图，就有深度图，深度图像也叫距离影像，是指将从图像采集器到场景中各点的距离（深度）值作为像素值的图像。

获取方法有：激光雷达深度成像法、计算机立体视觉成像、坐标测量机法、莫尔条纹法、结构光法。

那么这里引申一下深度图与点云的区别，点云：当一束激光照射到物体表面时，所反射的激光会携带方位、距离等信息。

若将激光束按照某种轨迹进行扫描，便会边扫描边记录到反射的激光点信息，由于扫描极为精细，则能够得到大量的激光点，因而就可形成激光点云。

深度图像经过坐标转换可以计算为点云数据；有规则及必要信息的点云数据可以反算为深度图像。

两者在一定条件下是可以相互转化的，之前的博客里，有使用PCL库实现过点云提取深度图，当然给出相机参数也是可以由深度图转为点云的。

截图一个深度图：
所以深度与视差的关系如下
比如绝对差值法绝对差值图的计算方法如下：。

鹰眼系统简介基于计算机视觉技术的⽻⽑球“鹰眼”系统的设计与研究摘要计算机视觉技术被⼴泛应⽤于国防，⼯业，交通和体育产业，并发挥着越来越重要的作⽤。

国际⽹球公开赛中的鹰眼系统作为计算机视觉在体育中应⽤的典型代表，为⽹球运动带来了前所未有的发展和新的活⼒。

与此同时，呼唤⽻⽑球鹰眼系统的呼声越来越⾼，但是应⽤在⽻⽑球运动上的鹰眼技术还有⼀定的局限性，如何将相应的计算机视觉技术和⽻⽑球运动结合起来，为⽻⽑球赛事、⽻⽑球教学中带来新的活⼒和兴奋点以及新的⽅法和⼿段，成为了⽬前⼀个新的研究课题。

⽻⽑球“鹰眼”系统以计算机视觉技术为背景，使计算机技术与⽻⽑球运动相结合，核⼼是研究摄像机标定、运动物体检测、⽬标定位等计算机视觉技术。

本⽂在阅读⼤量国内外⽂献的基础上，根据⽻⽑球运动的特点，设计并通过实验实现了基于计算机视觉的⽻⽑球“鹰眼”系统。

针对系统的要求和实际试验情况，采⽤了SONYFDR-AX1E摄像机采集视频图像；使⽤了Tsai两步法对摄像机进⾏了标定，并获得了左右两个摄像机的内参数和外参数、两个摄像机之间的位姿变换关系以及摄像机坐标系与世界坐标系之间的关系，为三维信息获取奠基了基础；检测运动物体⽬标时，背景差分法的使⽤，以及结合了实时更新背景图像的⽅法，提⾼了⽻⽑球检测的准确性；在对⽻⽑球进⾏定位时，联合背景差分、数学形态学中的开运算和特征点统计的⽅法，可以稳定、精确的检测⽻⽑球⽬标的质⼼，对光照变化不敏感，鲁棒性较好；基于⽻⽑球“鹰眼”系统⽬标唯⼀的特点以及实时性的要求，在获取⽻⽑球⽬标的⼆维质⼼之后，使⽤⽴体视觉算法求得了⽻⽑球⽬标的三维信息；针对三维数据中的误差，⽤最⼩⼆乘法对曲线进⾏拟合，平滑处理了数据。

系统运⾏表明使⽤的⽅法快速、有效，具有很好的鲁棒性，精度误差约为2cm，准确的判断⽻⽑球的第⼀落点，可以满⾜系统的要求。

关键词：⽻⽑球“鹰眼”；计算机视觉；运动⽬标检测；Design and Research of the Badminton Hawkeye System Based onComputer VisionAbstractComputer Vision Technology is widely used in the national defense, industry, transportation and sports, and plays an increasing important role in these field. As the typical example of the application of the computer vision in the sports field, Tennis Hawkeye System brings the unprecedented development and new energy. At the same time, the voice of calling Badminton Hawkeye System is get ting higher in recent years. However, there’s still some limitations when applying Badminton Hawkeye System. So it becomes a new research topic on how to combine the computer vision technology and badminton, and to bring the new energy and excitement to the badminton matches and teaching.Based on the computer vision, the badminton Hawkeye system offers an effective platform of the combination of the computer technology and badminton sports. In the process of the system design and implementation, the camera calibration, moving object detection, object location and binocular stereo vision are focused researched.On the basis of reading a large number of literature at home and abroad, and according to the characteristic of badminton sports, the Badminton Hawkeye System based on computer vision is studied and designed. Two SONY FDR-AX1E cameras are used to capture video images according to the system requirements and reality situation of the experiment. Tsai two-step method is used to calibrate the cameras, and the internal and external parameters of the two cameras, the pose transformation relations of the two cameras and the relationships of the camera coordinate and the world coordinate are got, which lay the foundation for 3D information acquisition. When detecting the moving object, combined with the method of updating background, background differencing is used for the accuracy of the test. When locating the object, the method of background differencing, opening operation of the mathematical morphology and static of feature points are combined to detect the object steadily and accurately, and which are not sensitive to illumination variation, and the system has the better robustness. After getting the two-dimensional centroid of the object, the three-dimension spatial information are obtained using stereo vision algorithm based on the uniqueness and the real-time of the badminton Hawkeye system. Curve fitting is used for reducing the three-dimension data error, and smoothing process the data.The operation shows that the methods used in the system are fast and effective, and have better robustness, the accuracy error is about 2cm，which satisfied the requirements and could judge the first placement of badminton.Key word: Badminton Hawkeye system; computer vision; moving object detection⽬录摘要 (2)Abstract (4)第⼀章绪论 (9)1.1引⾔ (9)1.2研究背景及意义 (9)1.2.1鹰眼系统简介 (9)1.2.2计算机视觉技术及其发展 (10)1.2.3摄像机标定技术 (12)1.2.4运动⽬标检测技术的发展和应⽤ (14) 1.2.5三维重建技术的应⽤及其发展 (14)第⼆章⽻⽑球“鹰眼”系统的分析与设计 (17) 2.1⽻⽑球“鹰眼”系统的提出 (17)2.2⽻⽑球“鹰眼”系统的特点 (17)2.3⽻⽑球“鹰眼”系统的设计 (18)2.3.1系统描述 (18)2.3.2系统的硬件⽅案设计 (18)2.3.3系统软件⽅案设计 (19)2.4系统⼯作流程 (21)2.5系统关键技术 (22)2.5.1摄像机标定 (22)2.5.2⽻⽑球的⼆维检测及定位 (23)2.5.3⽻⽑球的三维信息获取 (23)2.6本章⼩结 (23)第三章摄像机标定算法的研究 (25)3.1摄像机成像模型 (25)3.1.1图像坐标系 (25)3.1.2摄像机坐标系 (27)3.1.3世界坐标系 (27)3.1.4针孔成像摄像机模型 (28)3.2传统摄像机标定算法 (29)3.2.1⾮线性优化⽅法 (29)3.2.2直接线性变化法（DLT） (30)3.2.3Tsai两步法 (30)3.3系统使⽤⽅法及实验结果 (33)3.3.1双⽬视觉系统的标定 (33)3.3.2世界坐标系的获取 (35)3.4本章⼩结 (36)第四章⽻⽑球的⼆维检测及定位研究 (37)4.1⽻⽑球⽬标的检测 (37)4.1.1现有的运动检测⽅法 (37)4.1.2基于背景差分的⼆维运动⽬标检测 (38)4.1.3实验结果 (40)4.2⽻⽑球⽬标的定位 (41)4.2.1噪声处理 (41)4.2.2特征提取 (42)4.2.3⽻⽑球⽬标的定位 (43)4.2.4实验结果 (44)4.3本章⼩结 (46)第五章⽻⽑球的三维信息获取与重建 (47)5.1双⽬⽴体视觉原理........................................................... 错误！未定义书签。