32.2 ABSTRACT Exploiting K-Distance Signature for Boolean Matching and G-Symmetry Detection

第 38 卷第 11 期2023 年 11 月Vol.38 No.11Nov. 2023液晶与显示Chinese Journal of Liquid Crystals and Displays基于自适应聚焦CRIoU损失的目标检测算法肖振久1，赵昊泽2，张莉莉2，夏羽3，郭杰龙4*，俞辉4，李成龙2，王俐文2（1.辽宁工程技术大学软件学院，辽宁葫芦岛 125000；2.中国兵器工业集团航空弹药研究院有限公司，黑龙江哈尔滨， 150000；3.上海宇航系统工程研究所，上海 201100；4.中国科学院海西研究院泉州装备制造研究中心，福建泉州 362000）摘要：在目标检测任务中，传统的边界框回归损失函数所回归的内容与评价标准IoU（Intersection over Union）之间存在不相关性，并且对于边界框的回归属性存在一定不合理性，使得回归属性不完整，降低了检测精度和收敛速度，甚至还会造成回归阻碍的情况。

并且在回归任务中也存在样本不均衡的情况，大量的低质量样本影响了损失收敛。

为了提高检测精度和回归收敛速度提出了一种新的边界框回归损失函数。

首先确定设计思想并设计IoU系列损失函数的范式；其次在IoU损失的基础上引入两中心点形成矩形的周长和两框形成的最小闭包矩形周长的比值作为边界框中心点距离惩罚项，并且将改进的IoU损失应用到非极大值抑制（Non-Maximum Suppression，NMS）处理中。

接着引入两框的宽高误差和最小外包框的宽高平方作为宽高惩罚项，确定CRIoU（Complete Relativity IoU，CRIoU）损失函数。

最后在CRIoU 的基础上加入自适应加权因子，对高质量样本的回归损失加权，定义了自适应聚焦CRIoU（Adaptive focal CRIoU，AF-CRIoU）。

实验结果表明，使用AF-CRIoU损失函数对比传统非IoU系列损失的检测精度最高相对提升了8.52%，对比CIoU系列损失的检测精度最高相对提升了2.69%，使用A-CRIoU-NMS（Around CRIoU NMS）方法对比原NMS 方法的检测精度提升0.14%。

摘要摘要本文介绍的是一种从图像中获得关键点并用关键点去提供可靠的图像匹配的方法——SIFT算法。

SIFT算法是一种提取局部特征的算法，它能在尺度空间提取关键点。

关键点的信息包括位置，尺度，方向。

相对于全局特征，SIFT特征描述子独特性好，信息量丰富，对旋转、尺度缩放、视角变化、噪声等大多数变换或干扰具有很强的不变性，是一种非常优秀的局部特征描述。

本文对SIFT算法中的关键点检测技术进行了详细的阐述，并用程序将关键点检测过程实现。

实现过程包括高斯金字塔的建立，DOG金字塔的建立，尺度空间生成，关键点提取以及描述子的生成等。

最后进行了实验，对在检测中起到关键作用的参数进行了验证性实验，得到了满意的结果。

关键词：关键点尺度空间描述子AbstractAbstractThis paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene，which called ——SIFT. SIFT is algorithm that can detect local descriptors, extrema in scale-space. This method can extract location, scale, and rotation. Sift feature is highly distinctive and show extraordinary robustness against most disturbances such as scaling,rotation, occlusion, perspective and illumination changes，and is a kind of excellent local descriptors.The key point detection technology in SIFT algorithm is described in details in this paper. The detection process is realized. The process includes building the Gaussian pyramid and DOG pyramid, setting up scale-space,the key descriptor extraction and descriptors’ generation and so on. Finally，we make experiment testing the parameters which play an important role in key points detection. We get satisfied results. Keyword: keypoints scale-space descriptors目录 i目录第一章绪论 ......................................................................................... 错误！未定义书签。

Full-Dimension MIMO (FD-MIMO) for Next Generation Cellular Technology AbstractThis article considers a practical implementation of massive MIMO system.Although the best performance can be achieved when a large number of active antennas are placed only in horizontal domain, BS form factor(形状系数) limitation often makes horizontal array placement infeasible. To cope with this limitation, this article introduces full-dimension MIMO (FD-MIMO) cellular wireless communication system, where active antennas are placed in a 2D grid at BSs.动机：大量有源天线水平放置可以达到最好性能，但基站的形状系统限制使水平阵列放置不灵活。

为此引入full-dimension MIMO 蜂窝无线通信系统，其中基站端的有源天线以2维网络状分布。

For analysis of the FD-MIMO systems, a 3D spatial channel model is introduced, on which system-level simulations are conducted.内容：为了分析FD-MIMO系统，介绍了一个3维空间信道模型，并在该信道模型的基础上进行了系统级的仿真。

The simulation results show that the proposed FD-MIMO system with 32 antenna ports achieves 2–3.6 times cell average throughput gain and 1.5–5 times cell edge throughput gain compared to the 4G LTE system of two antenna ports at the BS.结论：仿真结果显示，相比于具有2天线端口的4G LTE系统，所提FD-MIMO系统在拥有32天线端口天线时，小区平均吞吐量增加2-3.6倍，小区边缘吞吐量增加1.5-5倍。

聚类算法在网络入侵检测技术中的作用作者：张培帅来源：《电脑知识与技术·学术交流》2008年第32期摘要：该文探讨了聚类算法作为一种无监督的异常检测技术，在网络入侵检测技术中的作用，并通过具体分析K-means算法和迭代最优化算法的优劣，把两种算法结合起来，提出一种新的分类算法。

关键词：模式识别；无监督的异常检测技术；聚类算法K-means算法；迭代最优化算法中图分类号：TP393文献标识码：A文章编号：1009-3044(2008)32-1194-03The Role of the Clustering Algorithm in the Network Intrusion Detection TechnologyZHANG Pei-shuai(The PLA 71977 Unit, Rizhao 276800, China)Abstract: The paper studies the role of the clustering algorithm in the network intrusion detection technology, and create a new algorithm by analysising the Pros and Cons of K-means algorithm and Iterative optimization algorithms.Key words: pattern recognition; anomaly detection technology without supervision; clustering algorithm; K-means algorithm; iterative optimization algorithms1 引言我们知道，在一个多因素问题中，结果即目标与各因素即指标之间难以找出直接的联系，很难用理论的途径去解决。

Draft:Deep Learning in Neural Networks:An OverviewTechnical Report IDSIA-03-14/arXiv:1404.7828(v1.5)[cs.NE]J¨u rgen SchmidhuberThe Swiss AI Lab IDSIAIstituto Dalle Molle di Studi sull’Intelligenza ArtificialeUniversity of Lugano&SUPSIGalleria2,6928Manno-LuganoSwitzerland15May2014AbstractIn recent years,deep artificial neural networks(including recurrent ones)have won numerous con-tests in pattern recognition and machine learning.This historical survey compactly summarises relevantwork,much of it from the previous millennium.Shallow and deep learners are distinguished by thedepth of their credit assignment paths,which are chains of possibly learnable,causal links between ac-tions and effects.I review deep supervised learning(also recapitulating the history of backpropagation),unsupervised learning,reinforcement learning&evolutionary computation,and indirect search for shortprograms encoding deep and large networks.PDF of earlier draft(v1):http://www.idsia.ch/∼juergen/DeepLearning30April2014.pdfLATEX source:http://www.idsia.ch/∼juergen/DeepLearning30April2014.texComplete BIBTEXfile:http://www.idsia.ch/∼juergen/bib.bibPrefaceThis is the draft of an invited Deep Learning(DL)overview.One of its goals is to assign credit to those who contributed to the present state of the art.I acknowledge the limitations of attempting to achieve this goal.The DL research community itself may be viewed as a continually evolving,deep network of scientists who have influenced each other in complex ways.Starting from recent DL results,I tried to trace back the origins of relevant ideas through the past half century and beyond,sometimes using“local search”to follow citations of citations backwards in time.Since not all DL publications properly acknowledge earlier relevant work,additional global search strategies were employed,aided by consulting numerous neural network experts.As a result,the present draft mostly consists of references(about800entries so far).Nevertheless,through an expert selection bias I may have missed important work.A related bias was surely introduced by my special familiarity with the work of my own DL research group in the past quarter-century.For these reasons,the present draft should be viewed as merely a snapshot of an ongoing credit assignment process.To help improve it,please do not hesitate to send corrections and suggestions to juergen@idsia.ch.Contents1Introduction to Deep Learning(DL)in Neural Networks(NNs)3 2Event-Oriented Notation for Activation Spreading in FNNs/RNNs3 3Depth of Credit Assignment Paths(CAPs)and of Problems4 4Recurring Themes of Deep Learning54.1Dynamic Programming(DP)for DL (5)4.2Unsupervised Learning(UL)Facilitating Supervised Learning(SL)and RL (6)4.3Occam’s Razor:Compression and Minimum Description Length(MDL) (6)4.4Learning Hierarchical Representations Through Deep SL,UL,RL (6)4.5Fast Graphics Processing Units(GPUs)for DL in NNs (6)5Supervised NNs,Some Helped by Unsupervised NNs75.11940s and Earlier (7)5.2Around1960:More Neurobiological Inspiration for DL (7)5.31965:Deep Networks Based on the Group Method of Data Handling(GMDH) (8)5.41979:Convolution+Weight Replication+Winner-Take-All(WTA) (8)5.51960-1981and Beyond:Development of Backpropagation(BP)for NNs (8)5.5.1BP for Weight-Sharing Feedforward NNs(FNNs)and Recurrent NNs(RNNs)..95.6Late1980s-2000:Numerous Improvements of NNs (9)5.6.1Ideas for Dealing with Long Time Lags and Deep CAPs (10)5.6.2Better BP Through Advanced Gradient Descent (10)5.6.3Discovering Low-Complexity,Problem-Solving NNs (11)5.6.4Potential Benefits of UL for SL (11)5.71987:UL Through Autoencoder(AE)Hierarchies (12)5.81989:BP for Convolutional NNs(CNNs) (13)5.91991:Fundamental Deep Learning Problem of Gradient Descent (13)5.101991:UL-Based History Compression Through a Deep Hierarchy of RNNs (14)5.111992:Max-Pooling(MP):Towards MPCNNs (14)5.121994:Contest-Winning Not So Deep NNs (15)5.131995:Supervised Recurrent Very Deep Learner(LSTM RNN) (15)5.142003:More Contest-Winning/Record-Setting,Often Not So Deep NNs (16)5.152006/7:Deep Belief Networks(DBNs)&AE Stacks Fine-Tuned by BP (17)5.162006/7:Improved CNNs/GPU-CNNs/BP-Trained MPCNNs (17)5.172009:First Official Competitions Won by RNNs,and with MPCNNs (18)5.182010:Plain Backprop(+Distortions)on GPU Yields Excellent Results (18)5.192011:MPCNNs on GPU Achieve Superhuman Vision Performance (18)5.202011:Hessian-Free Optimization for RNNs (19)5.212012:First Contests Won on ImageNet&Object Detection&Segmentation (19)5.222013-:More Contests and Benchmark Records (20)5.22.1Currently Successful Supervised Techniques:LSTM RNNs/GPU-MPCNNs (21)5.23Recent Tricks for Improving SL Deep NNs(Compare Sec.5.6.2,5.6.3) (21)5.24Consequences for Neuroscience (22)5.25DL with Spiking Neurons? (22)6DL in FNNs and RNNs for Reinforcement Learning(RL)236.1RL Through NN World Models Yields RNNs With Deep CAPs (23)6.2Deep FNNs for Traditional RL and Markov Decision Processes(MDPs) (24)6.3Deep RL RNNs for Partially Observable MDPs(POMDPs) (24)6.4RL Facilitated by Deep UL in FNNs and RNNs (25)6.5Deep Hierarchical RL(HRL)and Subgoal Learning with FNNs and RNNs (25)6.6Deep RL by Direct NN Search/Policy Gradients/Evolution (25)6.7Deep RL by Indirect Policy Search/Compressed NN Search (26)6.8Universal RL (27)7Conclusion271Introduction to Deep Learning(DL)in Neural Networks(NNs) Which modifiable components of a learning system are responsible for its success or failure?What changes to them improve performance?This has been called the fundamental credit assignment problem(Minsky, 1963).There are general credit assignment methods for universal problem solvers that are time-optimal in various theoretical senses(Sec.6.8).The present survey,however,will focus on the narrower,but now commercially important,subfield of Deep Learning(DL)in Artificial Neural Networks(NNs).We are interested in accurate credit assignment across possibly many,often nonlinear,computational stages of NNs.Shallow NN-like models have been around for many decades if not centuries(Sec.5.1).Models with several successive nonlinear layers of neurons date back at least to the1960s(Sec.5.3)and1970s(Sec.5.5). An efficient gradient descent method for teacher-based Supervised Learning(SL)in discrete,differentiable networks of arbitrary depth called backpropagation(BP)was developed in the1960s and1970s,and ap-plied to NNs in1981(Sec.5.5).BP-based training of deep NNs with many layers,however,had been found to be difficult in practice by the late1980s(Sec.5.6),and had become an explicit research subject by the early1990s(Sec.5.9).DL became practically feasible to some extent through the help of Unsupervised Learning(UL)(e.g.,Sec.5.10,5.15).The1990s and2000s also saw many improvements of purely super-vised DL(Sec.5).In the new millennium,deep NNs havefinally attracted wide-spread attention,mainly by outperforming alternative machine learning methods such as kernel machines(Vapnik,1995;Sch¨o lkopf et al.,1998)in numerous important applications.In fact,supervised deep NNs have won numerous of-ficial international pattern recognition competitions(e.g.,Sec.5.17,5.19,5.21,5.22),achieving thefirst superhuman visual pattern recognition results in limited domains(Sec.5.19).Deep NNs also have become relevant for the more generalfield of Reinforcement Learning(RL)where there is no supervising teacher (Sec.6).Both feedforward(acyclic)NNs(FNNs)and recurrent(cyclic)NNs(RNNs)have won contests(Sec.5.12,5.14,5.17,5.19,5.21,5.22).In a sense,RNNs are the deepest of all NNs(Sec.3)—they are general computers more powerful than FNNs,and can in principle create and process memories of ar-bitrary sequences of input patterns(e.g.,Siegelmann and Sontag,1991;Schmidhuber,1990a).Unlike traditional methods for automatic sequential program synthesis(e.g.,Waldinger and Lee,1969;Balzer, 1985;Soloway,1986;Deville and Lau,1994),RNNs can learn programs that mix sequential and parallel information processing in a natural and efficient way,exploiting the massive parallelism viewed as crucial for sustaining the rapid decline of computation cost observed over the past75years.The rest of this paper is structured as follows.Sec.2introduces a compact,event-oriented notation that is simple yet general enough to accommodate both FNNs and RNNs.Sec.3introduces the concept of Credit Assignment Paths(CAPs)to measure whether learning in a given NN application is of the deep or shallow type.Sec.4lists recurring themes of DL in SL,UL,and RL.Sec.5focuses on SL and UL,and on how UL can facilitate SL,although pure SL has become dominant in recent competitions(Sec.5.17-5.22). Sec.5is arranged in a historical timeline format with subsections on important inspirations and technical contributions.Sec.6on deep RL discusses traditional Dynamic Programming(DP)-based RL combined with gradient-based search techniques for SL or UL in deep NNs,as well as general methods for direct and indirect search in the weight space of deep FNNs and RNNs,including successful policy gradient and evolutionary methods.2Event-Oriented Notation for Activation Spreading in FNNs/RNNs Throughout this paper,let i,j,k,t,p,q,r denote positive integer variables assuming ranges implicit in the given contexts.Let n,m,T denote positive integer constants.An NN’s topology may change over time(e.g.,Fahlman,1991;Ring,1991;Weng et al.,1992;Fritzke, 1994).At any given moment,it can be described as afinite subset of units(or nodes or neurons)N= {u1,u2,...,}and afinite set H⊆N×N of directed edges or connections between nodes.FNNs are acyclic graphs,RNNs cyclic.Thefirst(input)layer is the set of input units,a subset of N.In FNNs,the k-th layer(k>1)is the set of all nodes u∈N such that there is an edge path of length k−1(but no longer path)between some input unit and u.There may be shortcut connections between distant layers.The NN’s behavior or program is determined by a set of real-valued,possibly modifiable,parameters or weights w i(i=1,...,n).We now focus on a singlefinite episode or epoch of information processing and activation spreading,without learning through weight changes.The following slightly unconventional notation is designed to compactly describe what is happening during the runtime of the system.During an episode,there is a partially causal sequence x t(t=1,...,T)of real values that I call events.Each x t is either an input set by the environment,or the activation of a unit that may directly depend on other x k(k<t)through a current NN topology-dependent set in t of indices k representing incoming causal connections or links.Let the function v encode topology information and map such event index pairs(k,t)to weight indices.For example,in the non-input case we may have x t=f t(net t)with real-valued net t= k∈in t x k w v(k,t)(additive case)or net t= k∈in t x k w v(k,t)(multiplicative case), where f t is a typically nonlinear real-valued activation function such as tanh.In many recent competition-winning NNs(Sec.5.19,5.21,5.22)there also are events of the type x t=max k∈int (x k);some networktypes may also use complex polynomial activation functions(Sec.5.3).x t may directly affect certain x k(k>t)through outgoing connections or links represented through a current set out t of indices k with t∈in k.Some non-input events are called output events.Note that many of the x t may refer to different,time-varying activations of the same unit in sequence-processing RNNs(e.g.,Williams,1989,“unfolding in time”),or also in FNNs sequentially exposed to time-varying input patterns of a large training set encoded as input events.During an episode,the same weight may get reused over and over again in topology-dependent ways,e.g.,in RNNs,or in convolutional NNs(Sec.5.4,5.8).I call this weight sharing across space and/or time.Weight sharing may greatly reduce the NN’s descriptive complexity,which is the number of bits of information required to describe the NN (Sec.4.3).In Supervised Learning(SL),certain NN output events x t may be associated with teacher-given,real-valued labels or targets d t yielding errors e t,e.g.,e t=1/2(x t−d t)2.A typical goal of supervised NN training is tofind weights that yield episodes with small total error E,the sum of all such e t.The hope is that the NN will generalize well in later episodes,causing only small errors on previously unseen sequences of input events.Many alternative error functions for SL and UL are possible.SL assumes that input events are independent of earlier output events(which may affect the environ-ment through actions causing subsequent perceptions).This assumption does not hold in the broaderfields of Sequential Decision Making and Reinforcement Learning(RL)(Kaelbling et al.,1996;Sutton and Barto, 1998;Hutter,2005)(Sec.6).In RL,some of the input events may encode real-valued reward signals given by the environment,and a typical goal is tofind weights that yield episodes with a high sum of reward signals,through sequences of appropriate output actions.Sec.5.5will use the notation above to compactly describe a central algorithm of DL,namely,back-propagation(BP)for supervised weight-sharing FNNs and RNNs.(FNNs may be viewed as RNNs with certainfixed zero weights.)Sec.6will address the more general RL case.3Depth of Credit Assignment Paths(CAPs)and of ProblemsTo measure whether credit assignment in a given NN application is of the deep or shallow type,I introduce the concept of Credit Assignment Paths or CAPs,which are chains of possibly causal links between events.Let usfirst focus on SL.Consider two events x p and x q(1≤p<q≤T).Depending on the appli-cation,they may have a Potential Direct Causal Connection(PDCC)expressed by the Boolean predicate pdcc(p,q),which is true if and only if p∈in q.Then the2-element list(p,q)is defined to be a CAP from p to q(a minimal one).A learning algorithm may be allowed to change w v(p,q)to improve performance in future episodes.More general,possibly indirect,Potential Causal Connections(PCC)are expressed by the recursively defined Boolean predicate pcc(p,q),which in the SL case is true only if pdcc(p,q),or if pcc(p,k)for some k and pdcc(k,q).In the latter case,appending q to any CAP from p to k yields a CAP from p to q(this is a recursive definition,too).The set of such CAPs may be large but isfinite.Note that the same weight may affect many different PDCCs between successive events listed by a given CAP,e.g.,in the case of RNNs, or weight-sharing FNNs.Suppose a CAP has the form(...,k,t,...,q),where k and t(possibly t=q)are thefirst successive elements with modifiable w v(k,t).Then the length of the suffix list(t,...,q)is called the CAP’s depth (which is0if there are no modifiable links at all).This depth limits how far backwards credit assignment can move down the causal chain tofind a modifiable weight.1Suppose an episode and its event sequence x1,...,x T satisfy a computable criterion used to decide whether a given problem has been solved(e.g.,total error E below some threshold).Then the set of used weights is called a solution to the problem,and the depth of the deepest CAP within the sequence is called the solution’s depth.There may be other solutions(yielding different event sequences)with different depths.Given somefixed NN topology,the smallest depth of any solution is called the problem’s depth.Sometimes we also speak of the depth of an architecture:SL FNNs withfixed topology imply a problem-independent maximal problem depth bounded by the number of non-input layers.Certain SL RNNs withfixed weights for all connections except those to output units(Jaeger,2001;Maass et al.,2002; Jaeger,2004;Schrauwen et al.,2007)have a maximal problem depth of1,because only thefinal links in the corresponding CAPs are modifiable.In general,however,RNNs may learn to solve problems of potentially unlimited depth.Note that the definitions above are solely based on the depths of causal chains,and agnostic of the temporal distance between events.For example,shallow FNNs perceiving large“time windows”of in-put events may correctly classify long input sequences through appropriate output events,and thus solve shallow problems involving long time lags between relevant events.At which problem depth does Shallow Learning end,and Deep Learning begin?Discussions with DL experts have not yet yielded a conclusive response to this question.Instead of committing myself to a precise answer,let me just define for the purposes of this overview:problems of depth>10require Very Deep Learning.The difficulty of a problem may have little to do with its depth.Some NNs can quickly learn to solve certain deep problems,e.g.,through random weight guessing(Sec.5.9)or other types of direct search (Sec.6.6)or indirect search(Sec.6.7)in weight space,or through training an NNfirst on shallow problems whose solutions may then generalize to deep problems,or through collapsing sequences of(non)linear operations into a single(non)linear operation—but see an analysis of non-trivial aspects of deep linear networks(Baldi and Hornik,1994,Section B).In general,however,finding an NN that precisely models a given training set is an NP-complete problem(Judd,1990;Blum and Rivest,1992),also in the case of deep NNs(S´ıma,1994;de Souto et al.,1999;Windisch,2005);compare a survey of negative results(S´ıma, 2002,Section1).Above we have focused on SL.In the more general case of RL in unknown environments,pcc(p,q) is also true if x p is an output event and x q any later input event—any action may affect the environment and thus any later perception.(In the real world,the environment may even influence non-input events computed on a physical hardware entangled with the entire universe,but this is ignored here.)It is possible to model and replace such unmodifiable environmental PCCs through a part of the NN that has already learned to predict(through some of its units)input events(including reward signals)from former input events and actions(Sec.6.1).Its weights are frozen,but can help to assign credit to other,still modifiable weights used to compute actions(Sec.6.1).This approach may lead to very deep CAPs though.Some DL research is about automatically rephrasing problems such that their depth is reduced(Sec.4). In particular,sometimes UL is used to make SL problems less deep,e.g.,Sec.5.10.Often Dynamic Programming(Sec.4.1)is used to facilitate certain traditional RL problems,e.g.,Sec.6.2.Sec.5focuses on CAPs for SL,Sec.6on the more complex case of RL.4Recurring Themes of Deep Learning4.1Dynamic Programming(DP)for DLOne recurring theme of DL is Dynamic Programming(DP)(Bellman,1957),which can help to facili-tate credit assignment under certain assumptions.For example,in SL NNs,backpropagation itself can 1An alternative would be to count only modifiable links when measuring depth.In many typical NN applications this would not make a difference,but in some it would,e.g.,Sec.6.1.be viewed as a DP-derived method(Sec.5.5).In traditional RL based on strong Markovian assumptions, DP-derived methods can help to greatly reduce problem depth(Sec.6.2).DP algorithms are also essen-tial for systems that combine concepts of NNs and graphical models,such as Hidden Markov Models (HMMs)(Stratonovich,1960;Baum and Petrie,1966)and Expectation Maximization(EM)(Dempster et al.,1977),e.g.,(Bottou,1991;Bengio,1991;Bourlard and Morgan,1994;Baldi and Chauvin,1996; Jordan and Sejnowski,2001;Bishop,2006;Poon and Domingos,2011;Dahl et al.,2012;Hinton et al., 2012a).4.2Unsupervised Learning(UL)Facilitating Supervised Learning(SL)and RL Another recurring theme is how UL can facilitate both SL(Sec.5)and RL(Sec.6).UL(Sec.5.6.4) is normally used to encode raw incoming data such as video or speech streams in a form that is more convenient for subsequent goal-directed learning.In particular,codes that describe the original data in a less redundant or more compact way can be fed into SL(Sec.5.10,5.15)or RL machines(Sec.6.4),whose search spaces may thus become smaller(and whose CAPs shallower)than those necessary for dealing with the raw data.UL is closely connected to the topics of regularization and compression(Sec.4.3,5.6.3). 4.3Occam’s Razor:Compression and Minimum Description Length(MDL) Occam’s razor favors simple solutions over complex ones.Given some programming language,the prin-ciple of Minimum Description Length(MDL)can be used to measure the complexity of a solution candi-date by the length of the shortest program that computes it(e.g.,Solomonoff,1964;Kolmogorov,1965b; Chaitin,1966;Wallace and Boulton,1968;Levin,1973a;Rissanen,1986;Blumer et al.,1987;Li and Vit´a nyi,1997;Gr¨u nwald et al.,2005).Some methods explicitly take into account program runtime(Al-lender,1992;Watanabe,1992;Schmidhuber,2002,1995);many consider only programs with constant runtime,written in non-universal programming languages(e.g.,Rissanen,1986;Hinton and van Camp, 1993).In the NN case,the MDL principle suggests that low NN weight complexity corresponds to high NN probability in the Bayesian view(e.g.,MacKay,1992;Buntine and Weigend,1991;De Freitas,2003), and to high generalization performance(e.g.,Baum and Haussler,1989),without overfitting the training data.Many methods have been proposed for regularizing NNs,that is,searching for solution-computing, low-complexity SL NNs(Sec.5.6.3)and RL NNs(Sec.6.7).This is closely related to certain UL methods (Sec.4.2,5.6.4).4.4Learning Hierarchical Representations Through Deep SL,UL,RLMany methods of Good Old-Fashioned Artificial Intelligence(GOFAI)(Nilsson,1980)as well as more recent approaches to AI(Russell et al.,1995)and Machine Learning(Mitchell,1997)learn hierarchies of more and more abstract data representations.For example,certain methods of syntactic pattern recog-nition(Fu,1977)such as grammar induction discover hierarchies of formal rules to model observations. The partially(un)supervised Automated Mathematician/EURISKO(Lenat,1983;Lenat and Brown,1984) continually learns concepts by combining previously learnt concepts.Such hierarchical representation learning(Ring,1994;Bengio et al.,2013;Deng and Yu,2014)is also a recurring theme of DL NNs for SL (Sec.5),UL-aided SL(Sec.5.7,5.10,5.15),and hierarchical RL(Sec.6.5).Often,abstract hierarchical representations are natural by-products of data compression(Sec.4.3),e.g.,Sec.5.10.4.5Fast Graphics Processing Units(GPUs)for DL in NNsWhile the previous millennium saw several attempts at creating fast NN-specific hardware(e.g.,Jackel et al.,1990;Faggin,1992;Ramacher et al.,1993;Widrow et al.,1994;Heemskerk,1995;Korkin et al., 1997;Urlbe,1999),and at exploiting standard hardware(e.g.,Anguita et al.,1994;Muller et al.,1995; Anguita and Gomes,1996),the new millennium brought a DL breakthrough in form of cheap,multi-processor graphics cards or GPUs.GPUs are widely used for video games,a huge and competitive market that has driven down hardware prices.GPUs excel at fast matrix and vector multiplications required not only for convincing virtual realities but also for NN training,where they can speed up learning by a factorof50and more.Some of the GPU-based FNN implementations(Sec.5.16-5.19)have greatly contributed to recent successes in contests for pattern recognition(Sec.5.19-5.22),image segmentation(Sec.5.21), and object detection(Sec.5.21-5.22).5Supervised NNs,Some Helped by Unsupervised NNsThe main focus of current practical applications is on Supervised Learning(SL),which has dominated re-cent pattern recognition contests(Sec.5.17-5.22).Several methods,however,use additional Unsupervised Learning(UL)to facilitate SL(Sec.5.7,5.10,5.15).It does make sense to treat SL and UL in the same section:often gradient-based methods,such as BP(Sec.5.5.1),are used to optimize objective functions of both UL and SL,and the boundary between SL and UL may blur,for example,when it comes to time series prediction and sequence classification,e.g.,Sec.5.10,5.12.A historical timeline format will help to arrange subsections on important inspirations and techni-cal contributions(although such a subsection may span a time interval of many years).Sec.5.1briefly mentions early,shallow NN models since the1940s,Sec.5.2additional early neurobiological inspiration relevant for modern Deep Learning(DL).Sec.5.3is about GMDH networks(since1965),perhaps thefirst (feedforward)DL systems.Sec.5.4is about the relatively deep Neocognitron NN(1979)which is similar to certain modern deep FNN architectures,as it combines convolutional NNs(CNNs),weight pattern repli-cation,and winner-take-all(WTA)mechanisms.Sec.5.5uses the notation of Sec.2to compactly describe a central algorithm of DL,namely,backpropagation(BP)for supervised weight-sharing FNNs and RNNs. It also summarizes the history of BP1960-1981and beyond.Sec.5.6describes problems encountered in the late1980s with BP for deep NNs,and mentions several ideas from the previous millennium to overcome them.Sec.5.7discusses afirst hierarchical stack of coupled UL-based Autoencoders(AEs)—this concept resurfaced in the new millennium(Sec.5.15).Sec.5.8is about applying BP to CNNs,which is important for today’s DL applications.Sec.5.9explains BP’s Fundamental DL Problem(of vanishing/exploding gradients)discovered in1991.Sec.5.10explains how a deep RNN stack of1991(the History Compressor) pre-trained by UL helped to solve previously unlearnable DL benchmarks requiring Credit Assignment Paths(CAPs,Sec.3)of depth1000and more.Sec.5.11discusses a particular WTA method called Max-Pooling(MP)important in today’s DL FNNs.Sec.5.12mentions afirst important contest won by SL NNs in1994.Sec.5.13describes a purely supervised DL RNN(Long Short-Term Memory,LSTM)for problems of depth1000and more.Sec.5.14mentions an early contest of2003won by an ensemble of shallow NNs, as well as good pattern recognition results with CNNs and LSTM RNNs(2003).Sec.5.15is mostly about Deep Belief Networks(DBNs,2006)and related stacks of Autoencoders(AEs,Sec.5.7)pre-trained by UL to facilitate BP-based SL.Sec.5.16mentions thefirst BP-trained MPCNNs(2007)and GPU-CNNs(2006). Sec.5.17-5.22focus on official competitions with secret test sets won by(mostly purely supervised)DL NNs since2009,in sequence recognition,image classification,image segmentation,and object detection. Many RNN results depended on LSTM(Sec.5.13);many FNN results depended on GPU-based FNN code developed since2004(Sec.5.16,5.17,5.18,5.19),in particular,GPU-MPCNNs(Sec.5.19).5.11940s and EarlierNN research started in the1940s(e.g.,McCulloch and Pitts,1943;Hebb,1949);compare also later work on learning NNs(Rosenblatt,1958,1962;Widrow and Hoff,1962;Grossberg,1969;Kohonen,1972; von der Malsburg,1973;Narendra and Thathatchar,1974;Willshaw and von der Malsburg,1976;Palm, 1980;Hopfield,1982).In a sense NNs have been around even longer,since early supervised NNs were essentially variants of linear regression methods going back at least to the early1800s(e.g.,Legendre, 1805;Gauss,1809,1821).Early NNs had a maximal CAP depth of1(Sec.3).5.2Around1960:More Neurobiological Inspiration for DLSimple cells and complex cells were found in the cat’s visual cortex(e.g.,Hubel and Wiesel,1962;Wiesel and Hubel,1959).These cellsfire in response to certain properties of visual sensory inputs,such as theorientation of plex cells exhibit more spatial invariance than simple cells.This inspired later deep NN architectures(Sec.5.4)used in certain modern award-winning Deep Learners(Sec.5.19-5.22).5.31965:Deep Networks Based on the Group Method of Data Handling(GMDH) Networks trained by the Group Method of Data Handling(GMDH)(Ivakhnenko and Lapa,1965; Ivakhnenko et al.,1967;Ivakhnenko,1968,1971)were perhaps thefirst DL systems of the Feedforward Multilayer Perceptron type.The units of GMDH nets may have polynomial activation functions imple-menting Kolmogorov-Gabor polynomials(more general than traditional NN activation functions).Given a training set,layers are incrementally grown and trained by regression analysis,then pruned with the help of a separate validation set(using today’s terminology),where Decision Regularisation is used to weed out superfluous units.The numbers of layers and units per layer can be learned in problem-dependent fashion. This is a good example of hierarchical representation learning(Sec.4.4).There have been numerous ap-plications of GMDH-style networks,e.g.(Ikeda et al.,1976;Farlow,1984;Madala and Ivakhnenko,1994; Ivakhnenko,1995;Kondo,1998;Kord´ık et al.,2003;Witczak et al.,2006;Kondo and Ueno,2008).5.41979:Convolution+Weight Replication+Winner-Take-All(WTA)Apart from deep GMDH networks(Sec.5.3),the Neocognitron(Fukushima,1979,1980,2013a)was per-haps thefirst artificial NN that deserved the attribute deep,and thefirst to incorporate the neurophysiolog-ical insights of Sec.5.2.It introduced convolutional NNs(today often called CNNs or convnets),where the(typically rectangular)receptivefield of a convolutional unit with given weight vector is shifted step by step across a2-dimensional array of input values,such as the pixels of an image.The resulting2D array of subsequent activation events of this unit can then provide inputs to higher-level units,and so on.Due to massive weight replication(Sec.2),relatively few parameters may be necessary to describe the behavior of such a convolutional layer.Competition layers have WTA subsets whose maximally active units are the only ones to adopt non-zero activation values.They essentially“down-sample”the competition layer’s input.This helps to create units whose responses are insensitive to small image shifts(compare Sec.5.2).The Neocognitron is very similar to the architecture of modern,contest-winning,purely super-vised,feedforward,gradient-based Deep Learners with alternating convolutional and competition lay-ers(e.g.,Sec.5.19-5.22).Fukushima,however,did not set the weights by supervised backpropagation (Sec.5.5,5.8),but by local un supervised learning rules(e.g.,Fukushima,2013b),or by pre-wiring.In that sense he did not care for the DL problem(Sec.5.9),although his architecture was comparatively deep indeed.He also used Spatial Averaging(Fukushima,1980,2011)instead of Max-Pooling(MP,Sec.5.11), currently a particularly convenient and popular WTA mechanism.Today’s CNN-based DL machines profita lot from later CNN work(e.g.,LeCun et al.,1989;Ranzato et al.,2007)(Sec.5.8,5.16,5.19).5.51960-1981and Beyond:Development of Backpropagation(BP)for NNsThe minimisation of errors through gradient descent(Hadamard,1908)in the parameter space of com-plex,nonlinear,differentiable,multi-stage,NN-related systems has been discussed at least since the early 1960s(e.g.,Kelley,1960;Bryson,1961;Bryson and Denham,1961;Pontryagin et al.,1961;Dreyfus,1962; Wilkinson,1965;Amari,1967;Bryson and Ho,1969;Director and Rohrer,1969;Griewank,2012),ini-tially within the framework of Euler-LaGrange equations in the Calculus of Variations(e.g.,Euler,1744). Steepest descent in such systems can be performed(Bryson,1961;Kelley,1960;Bryson and Ho,1969)by iterating the ancient chain rule(Leibniz,1676;L’Hˆo pital,1696)in Dynamic Programming(DP)style(Bell-man,1957).A simplified derivation of the method uses the chain rule only(Dreyfus,1962).The methods of the1960s were already efficient in the DP sense.However,they backpropagated derivative information through standard Jacobian matrix calculations from one“layer”to the previous one, explicitly addressing neither direct links across several layers nor potential additional efficiency gains due to network sparsity(but perhaps such enhancements seemed obvious to the authors).。

人工智能- K近邻分类算法的Java实现_天天向上-CSDN博客// 从文件中读取数据public static void Input(String path,int num,Flower[] arr) throws FileNotFoundException { File file new File(path); Scanner scanner new Scanner(file); for(int i 0;i num;i ) { DoublecalyxLength,calyxWidth,petalLength,petalWidth; Integer type; calyxLength scanner.nextDouble(); calyxWidthscanner.nextDouble(); petalLength scanner.nextDouble(); petalWidth scanner.nextDouble(); type scanner.nextInt(); arr[i] newFlower(calyxLength,calyxWidth,petalLength,petalWidth,type); // 计算K private static int calculateK(Flower[] dataTesting,int numberTesting,Flower[] dataTraining,int numberTraining) { int bestK 1; //最佳的k double bestValue 0; //识别率for(int k 1;k numberTraining;k ) { double temKNN(dataTesting,numberTesting,dataTraining,numberTraining,k ); System.out.println( k k 识别率tem); if(tem bestValue-0.000001) { bestValue tem; bestK k; return bestK; // KNN private static Double KNN(Flower[] dataTesting,int numberTesting,Flower[] dataTraining,int numberTraining, int k){ double cnt 0; for(int i 0;i numberTesting;i ) { // 清空计数器int count[] new int[5]; for(int j 1;j 4;j ) { count[j] 0; Queue DistanceAndType priorityQueue new PriorityQueue DistanceAndType (); //开始找最近的k个//距离类型排序比较器优先队列for(int j 0;j numberTraining;j ){ priorityQueue.add( newDistanceAndType( dataTesting[i].distance( dataTraining[j] ), dataTraining[j].getType() for(int j 0;j k;j ) { DistanceAndType dat priorityQueue.poll(); count[ dat.getType() ] ;//System.out.println( dat ); priorityQueue.clear();//System.out.println(); int bestCount 0,bestType -1; for(int j 1;j 4;j ) { if(count[j] bestCount) { bestCount count[j]; bestType j;if(bestType dataTesting[i].getType()) { cnt cnt 1; returncnt/numberTesting; public static void main(String[] args) throws FileNotFoundException { // 定义变量Scanner scanner new Scanner(System.in); Flower[] dataTesting new Flower[35]; Flower[] dataTraining new Flower[125]; int numberTesting 31,numberTraining 120,k; char option; // 输入数据Input( C:\\Users\\Administrator\\Desktop\\人工智能\\1033180327_孙创昱_AI_project6\\AI-实验6-KNN\\iris-data-testing.txt ,numberTesting,dataTesting); Input( C:\\Users\\Administrator\\Desktop\\人工智能\\1033180327_孙创昱_AI_project6\\AI-实验6-KNN\\iris-data-training.txt ,numberTraining,dataTraining); //输出检查for(int i 0;i 31;i ) System.out.println(dataTesting[i]); System.out.println( -------------------------------------------------------- ); for(int i 0;i 120;i ) System.out.println(dataTraining[i]); // 获得K System.out.println( 计算K吗(Y/N) ); optionscanner.next().charAt(0); if(option Y || option y ) { k calculateK(dataTesting,numberTesting,dataTraining,numberTrai ning); }else { System.out.println( 请输入K: ); kscanner.nextInt(); System.out.println( K k); //计算准确率Double resKNN(dataTesting,numberTesting,dataTraining,numberTraining,k ); System.out.println( 识别率为res); 运行结果可以看到k比较好的值为45 这个时候识别率为0.9677。

华中科技大学硕士学位论文摘要随着人工智能的兴起，计算机视觉领域近年来得到快速的发展。

作为计算机视觉领域研究的核心之一，视觉目标跟踪已经广泛应用于许多领域。

由于其应用领域十分广泛，应用场景也变得更加复杂。

尺度变化、遮挡、旋转、复杂背景、低分辨率等因素的存在给视觉目标跟踪带来了的挑战。

基于核相关滤波的KCF(Kernelized Correlation Filters)跟踪算法自问世以来受到了广泛的关注。

本文基于KCF跟踪算法，针对KCF跟踪算法的不足提出了一些改进方法。

针对KCF跟踪算法不能自适应目标尺度的变化，本文提出了SMIKCF(Scale-Adaptive Multiple-Feature Improved-Template-Update KCF)跟踪算法。

SMIKCF在KCF算法的基础上，加入一个尺度估计滤波器；提取样本的HOG((Histogram of Oriented Gradient))特征和CN(Color Name)特征，并把两个特征相融合；利用APCE判据改进位置估计滤波器模型的更新方式。

针对KCF算法在目标发生遮挡时容易出现目标丢失的情况，本文提出了AOKCF(Anti-Occlusion KCF)跟踪算法。

AOKCF在KCF算法的基础上，利用APCE 进行跟踪可靠性检测；加入检测模块，在检测到跟踪结果不可靠时，启动检测模块对目标进行检测，并使用位置滤波器进行目标识别，当识别出目标时，更新目标位置以及位置滤波器，否则直接进入下一帧，继续采用检测模块进行检测。

本文实验数据来源于Visual Tracker Benchmark。

采用OPE(One-Pass Evaluation)的评估方式，在Benchmark的50个测试视频序列上进行实验，来评估算法的性能。

实验表明，本文提出的方法能够有效地解决相关问题，提高跟踪算法的性能。

关键词：目标跟踪核相关滤波尺度自适应遮挡华中科技大学硕士学位论文ABSTRACTWith growing of artificial intelligence, computer vision has been rapid development in recent years. Visual target tracking is one of the most important part of computer vision research, with widely used in many fields. The application situation is becoming more and more complicated due to the wide application fields. The existence of scale changes, occlusion, rotation, complex background, low resolution and other factors has brought higher challenges to visual target tracking. KCF (Kernelized Correlation Filters) tracking algorithm has been widely concerned since it was proposed. Based on the KCF tracking algorithm, this paper proposed improved methods for the shortcoming of KCF tracking algorithm.Firstly, the SMIKCF(Scale-Adaptive Multiple-Feature Improved-Template-Update KCF) tracking algorithm is proposed in this paper to solve the problem that the KCF tracking algorithm can not adapt to the target scale. On the basis of KCF algorithm, SMIKCF adds a scale estimation filter, and combines HOG characteristics and CN characteristics, using the APCE criterion to improve the updating method of the position estimation filter model.Secondly, the AOKCF(Anti-Occlusion KCF) tracking algorithm is proposed to solve the problem of occlusion. AOKCF tracking algorithm is based on KCF tracking algorithm. APCE criterion is used to check the reliability of tracking results. When the tracking result is unreliable, add detection module to detect the target, and then use the position filter to the target recognition. If the target is recognized, then update the target position and the position filter module. Otherwise, go directly to the next frame.The experimental datas are from Visual Tracker Benchmark. Experiments were performed on Benchmark's 50 test video sequences and use the OPE (One-Pass Evaluation) evaluation method to evaluate the performance of the algorithm.华中科技大学硕士学位论文Experimental results show that the proposed method can effectively solve the related problems and improve the performance of the tracking algorithm.Keywords: Object tracking Kernelized correlation filters Scale adaptive Occlusion华中科技大学硕士学位论文目录摘要 (I)Abstract (II)目录 (IV)1 绪论 (1)1.1研究背景与意义 (1)1.2国内外研究概况 (1)1.3论文的主要研究内容 (4)1.4主要组织结构 (4)2 KCF跟踪算法介绍 (6)2.1引言 (6)2.2相关滤波器 (6)2.3核相关滤波器 (9)2.4本章小结 (17)3 改进的尺度自适应KCF跟踪算法研究 (18)3.1引言 (18)3.2尺度估计 (19)3.3多特征融合 (22)3.4模型更新策略改进 (22)3.5算法整体框架 (24)华中科技大学硕士学位论文3.6实验结果与分析 (25)3.7本章小结 (40)4 抗遮挡KCF跟踪算法研究 (41)4.1引言 (41)4.2跟踪可靠性检测 (43)4.3重检测算法研究 (43)4.4算法整体框架 (47)4.5实验结果与分析 (48)4.6本章小结 (60)5 总结与展望 (61)5.1全文总结 (61)5.2课题展望 (62)致谢 (63)参考文献 (64)附录1 攻读学位期间发表论文与参与课题 (70)华中科技大学硕士学位论文1 绪论1.1 研究背景与意义人工智能的飞速发展使得计算机视觉备受关注，作为计算机视觉领域研究的核心之一，视觉目标跟踪已经广泛应用于许多领域[1]。

I.J.Mathematical Sciences and Computing,2016, 2, 24-33Published Online April 2016 in MECS ()DOI: 10.5815/ijmsc.2016.02.03Available online at /ijmscRMSD Protein Tertiary Structure Prediction with Soft ComputingMohammad Saber Iraji a, Hakimeh Ameri ba Faculty Member of Department of Computer Engineering and Information Technology, Payame NoorUniversity, I.R. of Iranb Teacher of Department of Computer Engineering and Information Technology, Payame Noor University, I.R.of IranAbstractRoot-mean-square-deviation (RMSD) is an indicator in protein-structure-prediction-algorithms (PSPAs). Goal of PSP algorithms is to obtain 0 Å RMSD from native protein structures. Protein structure and RMSD prediction is very essential. In 2013, the estimated RMSD proteins based on nine features were obtained best results using D2N (Distance to the native). We presented in This paper proposed approach to reduce predicted RMSD Error Than the actual amount for RMSD and calculate mean absolute error (MAE), through feed forward neural network, adaptive neuro fuzzy method. ANFIS is achieved better and more accurate results.Index Terms: Root-mean-square-deviation (RMSD), protein, native structure, neural network, fuzzy.© 2016 Published by MECS Publisher. Selection and/or peer review under responsibility of the Research Association of Modern Education and Computer Science1.IntroductionFinding an accurate algorithm to predict the protein structure is proved as an extremely difficult problem. There are two fundamentally different approaches: one is ab initio prediction that did not take any help of previously known protein structures, which employ computer-based applications to minimize very large functions corresponding to the free energy in the molecules. The other one is known as a knowledge-based approach which used amino acid sequences as a source of knowledge. This knowledge then extracts and stores to be used to predict any knew amino acid sequence (proteins) with known sequence but unknown structure. We can categorize knowledge-based methods into two groups: statistics based, and neural network based [1].During the past four decades, several proteins secondary structural class prediction algorithms based on the sequence of protein amino acids have been studied and developed. Techniques such as neural networks, Fuzzy neural network, Support Vector Machines (SVM), component-coupled algorithm, and nearest neighbor classifier with a complexity-based distance measure (NN-CDM) are some of them. Among all these works the* Corresponding author.E-mail address: iraji.ms@, ha.amery@neural network based methods showed the most efficiency. Neural networks are an artificial intelligence technique and a family of statistical learning algorithms that usually involves a large number of processors operating in parallel, each with its own small knowledge to create a secondary structure model with optimal parameters. Neurons in a neural network can be processed to encode the amino acid sequences into a usable numerical form. The classification performance of a neural network prediction algorithm can be considerably differing depending on the derivation of a sustainable parametric model from the training data set [2]. A fuzzy neural network or neuro-fuzzy system is a learning machine that finds the parameters of a fuzzy system by exploiting approximation techniques from neural networks. This technique used to calculate the degree of confidence for each prediction result and overcome the uncertainties prediction problem [3]. Another technique that can be used to predict the secondary structural protein based on the extracted feathers of predicted burial information from amino acid residues is SVM.2.Related WorkSepideh Babaei and et al. (2010) studied a supervised learning of recurrent neural networks, which is well-suited for prediction of protein secondary structures from the underlying amino acid's sequence. They devised a modular prediction system based on the interactive integration of the MRR-NN and the MBR-NN structures. To model the strong correlations between adjacent secondary structure elements a Modular mutual recurrent neural network (MRR-NN) has been proposed. And, MBR-NN is a multilayer bidirectional recurrent neural network which can capture the long-range intra-molecular interactions between amino acids in formation of the secondary structure. The proposed model used to arbitrarily engage the neigh boring effects of the secondary structure types concurrent with memorizing the sequential dependencies of amino acids along the protein chain. They used their model on PSIPRED data set into three-fold cross-validation. The reported accuracy was 79.36% and boosts the segment overlap (SOV) was up to 70.09% [4]. Zhun Zhou and et al. (2010) proposed a model named SAC method based on association rule classifier, and rules are the rules are obtained by the KDD* model mining secondary structure for information of mutual impact. The KDD* Model focused on high confidence and low support rules, which is called …knowledge in shortage‟. The proposed model was based on CMAR (Classification based on Multiple Association rules) algorithm. The accuracies of the proposed model tested on RS126 and CB513 data sets are reported as 83.6% and 80.49%, respectively [5].Wu Qu and et al. (2011) proposed a multi-modal back propagation neural network (MMBP) method to predict the protein secondary structures. The created model is a compound pyramid model (CPM), which is composed of three layers of the intelligent interface that integrate multi-modal back propagation neural network (MMBP), mixed-modal SVM (MMS), modified Knowledge Discovery in the Databases (KDD) process and so on. The claimed accuracy of the proposed model on a non-redundant test data set of 256 proteins from RCASP256 was 86.13% [6]. Rohayan tihasan and et al. (2011) recommended a derivative feature vector, DPS that utilizes the optimal length of the local protein structure to predict the secondary structures of proteins. The DPS denotes the unification of amino acid propensity score and dihedral angle score. Secondary structure assignment method (SSAM) and secondary structure prediction method (SSPM) generate class labels for DPS feature vectors and the Support Vector Machine (SVM) used for prediction. The accuracy of the proposed model on the RS126 sequences was 84.4% [7]. Sepideh Babaei and et al. (2012) used a multilayer perceptron of recurrent neural network (RNN) pruned for optimizing size of hidden layers to enhance the secondary structure prediction. A type of reciprocal recurrent neural networks (MRR-NN) and the long-range interactions between amino acids in formation of the secondary structure (bidirectional RNN) and a multilayer bidirectional recurrent neural network (MBR-NN) consecutively applied to capture the predominant long-term dependencies. Finally, a modular prediction system (the MRR-NN and MBR-NN) used on the trusted PSIPRED data set and report the percentage accuracy (Q3) up to 76.91% and augment the segment overlap (SOV) up to 68.13% [8].Mohammad Hossein Zangooei and et al. (2012) proposed a method based on Support Vector Regression (SVR) classification. They used non-dominated Sorting Genetic Algorithm II (NSGAII) to find mapping points (MPs) for mapping a real-value to an integer one. We applied non-dominated Sorting Genetic Algorithm II (NSGAII)to find mapping points (MPs) to round a real-value to an integer one. In order to enhance the performance of the proposed model, they used the NSGAII to find and tune the SVR kernel parameters optimally. To improve the prediction result, the Dynamic Weighted Kernel Fusion (DWKF) method for fusing of three SVR kernels was used. The obtained classification accuracy of the proposed model on RS126 and CB513 data sets reported as 85.79% and 84.94% respectively [9].Maqsood Hayat and et al. (2014) proposed a model employing hybrid descriptor space along with the optimized evidence-theoretic K-nearest neighbor algorithms. The hybrid space is a composition of two descriptor spaces, including Multi-profile Bays and bi-gram probability. The high discriminative descriptors from the hybrid space have been selected by use of a well-known evolutionary feature selection technique named particle swarm optimization. These features are extracted to enhance the generalization power of the classifier. They used the jack knife test on three low similarity benchmark databases, including PDB, 1189, and 640 to evaluate the performance of their proposed model. The success rates of their proposed model are 87.0%, 86.6%, and 88.4%, respectively on the three benchmark data sets [10]. Maulika S. Patel and et al. (2014)provided a hybrid novel algorithm, KB-PROSSP-NN, which is a combination of knowledge base method (KB-PROSSP) and modelling of the exceptions in the knowledge base using neural networks (NN) for protein secondary structure prediction (PSSP). These two methods are used in cascade to optimize the results. They used two popular data sets RS126 and CB396 to evaluate the accuracy of the proposed model. The Q3 accuracy of 90.16% and 82.28% achieved on the RS126 and CB396 test sets respectively [11]. Yong Tat Tan and et al. (2015) claimed that nearest neighbor –complexity distance measure (NN-CDM) algorithm using Lempel–Ziv (LZ) complexity-based distance measure had a problem. NN-CDM algorithm is slow and ineffective in handling uncertainties. To solve this problem, they proposed fuzzy NN-CDM (FKNN-CDM) algorithm that combines the confidence level of prediction results and enhances the prediction process by designing hardware architecture. The high average prediction accuracies for Z277 and PDB data sets using the hybrid proposed algorithm are 84.12% and 47.81% respectively [3].Avinash Mishra and et al. (2014) used random forest machine learning methods to predict the protein structure. The two types of data sets have been chosen are: (i) Public decoys from Decoys …R‟us (/) and, (ii) Server predictions of CASP experiments CASP5 to CASP9 (http:// /download area/). This contains 409 systems with their decoys covering a Root-mean-square-deviation (RMSD) range of 0–30 Å There are 278 systems belonging to CASP decoys while 131 systems are from public decoy dataset. Three training models based on RMSDs are designed. The first model named “Model-I” trained on the complete training set consist of 64,827 structures that covered the whole range of RMSDs (0–30 Å) The second one named “Model-II”trained on 13,793 structures which covered RMSD range of 0–10 Ǻ and the last model named “Model-III” trained on 13,793 structures which covered an RMSD range of 0–5 Ǻ They combined these tree models together to set their final prediction structure. These models are combined as three different layers and used to predict the most accurate distance of any given structure. A 10-fold cross validation performed and Correlation, R2 and accuracy used for evaluation. The best reported results are as follow: that native can be predicted as less than 1.5 Å RMSD structure in~89.3% of the cases. High quality structures (0–3 Å) can be predicted to within ±1.5 Å error in ~98.5% of the cases. It means a 2 Å structure may be reported as either 1 Å or 3 Å structure or anywhere in between with ~ 98% accuracy [4]. Yuehui Chen and et al. (2012) proposed a novel method based on an integrated RBF neural network to predict the protein interaction sites. Features named sequence profiles, entropy, relative entropy, conservation weight, accessible surface area and sequence variability are extracted in the first step. They made six sliding windows on these features that contained 1, 3, 5, 7, 9 and 11 amino acid residues respectively, and used them as input layer of six radial basis functional (RBF) neural networks. These RBF neural networks were trained to predict the protein–protein interaction sites. Decision fusion (DF) and Genetic Algorithm based Selective Ensembles (GASEN) are used to obtain the final results. The recall and accuracy of their proposed methods are and 0.8032 respectively [12].We proposed a model for prediction the distance of an estimate structure from native based on Physicochemical Properties of Protein Tertiary Structure In this paper. The paper is organized in five sections.After the introduction Section 1 which introduces the related works of protein structure estimate, Section 2 continues with Mathematical Model in section 3. Section 4 and 5 presents the results, conclusions of the research. The paper ends with a list of references.3.Mathematical ModelSoft computing are included, different types of neural networks, fuzzy systems, genetic algorithms, etc. that used in information retrieval applications. Fuzzy theory was developed by Zadeh, is a new intelligent method, stated to solve unlike problems more efficient than the old calculations.3.1.Neural NetworkDeveloping a neural net solution means teaching the net a desired behavior. This is called the learning phase. Either sample data sets or a “teacher” can be used in this step. A teacher is either a numerical function or a person that rates the quality of the neural net performance. Since neural nets are mostly used for complex applications where no adequate mathematical models exist and rating the performance of a neural net is difficult in most applications, most are trained with sample data (figure 1).Fig.1. Training and Working Phase for Supervised Learning3.1.1.Neuron ModelAn elementary neuron with R inputs is shown in figure 2. Each input is weighted with an appropriate w. The sum of the weighted inputs and the bias, forms the input to the transfer function f. Neurons may use any differentiable transfer function f to generate their output.Fig.2. Structure a NeuronFeed forward networks often have one or more hidden layers of sigmoid neurons followed by an output layer of linear neurons. Multiple layers of neurons with nonlinear transfer functions allow the network to learn nonlinear and linear relationships between input and output vectors(figure 3). The linear output layer lets the network produce values outside the range –1 to +1. On the other hand, if you want to constrain the outputs of a network (such as between 0 and 1), then the output layer should use a sigmoid transfer function (such as logsig). This network can be used as a general function approaching. It can approximate any function with a finite number of discontinuities, arbitrarily well, given sufficient neurons in the hidden layer [14].3.1.2.Neural Network Step for Estimating RsmdBack-propagation feed forward neural network approach is used to predict the RMSD from native structure.A neural network is constructed with six inputs and one output RMSD.Fig.3. Feed Forward Neural Network with Two LayersThe input nodes represent the distinguishing parameters of proteins approach, and the output nodes represent the RMSD. The network is constructed by using MATLAB. In our experiment, the training function we have considered is a train GDA, the adaptation learning function considered learns GDM, and the performance function used was Mean Square Error (MSE). The projects considered for these researches are taken from the UCI. This is a data set of Physicochemical Properties of Protein Tertiary Structure. The data set is taken from CASP 5-9. There are 45730 decoys and size varying from 0 to 21 Armstrong [13]. After determining affect following features was extracted between the data set. The input nodes represent the physic-chemical properties of proteins.F1-Total surface area.F2- Non polar exposed area.F3- Fractional area of exposed non polar residue.F4- Fractional area of exposed non polar part of residue.F5- Average deviation from standard exposed area of residue.F6- Special Distribution constraints (N, K Value).3.2.ANFIS (Adaptive Neuro Fuzzy Inference System)ANFIS (Adaptive Neuro Fuzzy Inference System) is based sugeno (Jang, Sun & Mizutani, 1997; Jang & Sun, 1995) A generic rule in a Sugeno fuzzy pattern has the form If Input 1 = x and Input 2 = y, then output is z = ax + by + c. Figure 4 explains the ANFIS neural network [14]. In Figure 4 first layer are the degree of membershipof linguistic variables. The second layer is 'rules layer'. After the linear composition of rules at third layer then specify the degree of belonging to a special class by Sigmund's function in layer 4. ANFIS is a type of fuzzy neural network with a learning algorithm based on a set of training data for tuning an available rule base that permits the rule base to reconcile the training data [14].Fig.4. Adaptive Neuro fuzzy Network (anfis) for rmsdWe have applied features f1...f6 to ANFIS the given training data, the related rules is set, and obtain more accurate output RMSD (Figure 4).4.Experimental ResultsWe implement our proposed system in MATLAB version 7.12 on Laptop, 1.7 GHZ CPU. In Anfis proposed system was considered a database of CASP 5-9 with 45730 records, In order to train and test the fuzzy neural network. After calculate six Features: Total surface area, Non polar exposed area, Fractional area of exposed non polar residue, Fractional area of exposed non polar part of residue, Average deviation from standard exposed area of residue, Special Distribution constraint (N, K Value) described above for 38110 records was considered for train, and 7620 records were allocated to Test system. Neural network set six input, sevenFig.5. Adaptive Neuro fuzzy Network (ANFIS) for RMSDhidden neuron, one output RMSD(figure 5) and number of epochs 5 were considered. After train neural network, Rmse(Root mean squared errors ) was obtained for train data 5. 1044 and for test data 5.1008 (figure 6, 7).also Rmse of test data for six hidden neurons and 12 hidden neuron 5.1347 and 5.1226 was obtained. However, in Adaptive Fuzzy Neural Network After setting network parameters to generate fis =grid partition, optim. method= hybrid, linear, train fis epochs=5 Rmse for training data Obtained 4.7386 (Figure 8).Fig.6. Sqrt mse=5.1044 for Train DataFig.7. Error for Test Data Sqrt(mse)= 5.1008After completing the process the training adaptive fuzzy neural network, fuzzy input variables were calibrated (Figure 9) and the number 7620 record was to test.Fig.8. CalculateRMSE for Train Data with Proposed ANFIS SystemFig.9. Input Variable Fuzzy Membership For Proposed ANFIS System Before and After CalibrateTable 1. Comparison Actual RMSD and Predicted RMSD for Test DataSeveral methods exist to compare cost estimation models. Each method has its advantages and disadvantages. In this work, mean absolute error (MAE) of RMSD will be used. AE for each observation i can be obtained as:(1) MAE can be achieved through the summation of MAE over N observations:∑ (2) After training adaptive fuzzy neural network by uci data, we have applied our proposed neuro fuzzy system on 7620 records. In the proposed system predicted for row, 2 RMSD obtain 3.250838 When F1=7511.48, F2=1975.7, F3=0.26302, F4=85.9864, F5=114.437, F6=37.2338.5.ConclusionsOur purpose in this research is to predict the exact quantity of the Root-mean-square-deviation (RMSD) protein-structure from native protein structure's Protein and creating an adaptive neuro fuzzy model for this purpose, as yet. The Total surface area, Non polar exposed area, Fractional area of exposed non polar residue, Fractional area of exposed non polar part of residue, Average deviation from standard exposed area of residue, Special Distribution constraints (N, K Value) features were considered In order to The RMSD protein-structure estimation. Using adaptive neuro fuzzy combinatorial proposed system offered a2lgorithm, the amount of mean absolute error (MAE) indicator for ANFIS is3.845204 and neural network is 4.202547. ANFIS model is better, performed calculations in experimental results in table 1, prove this claim. In order to perform future works, the proposed model for distance from native protein structure can be developed with Raising The data relating to projects, and also other's neural methods can be used in order to determine the exact amount of RMSD in industrial environments and other data sets. Other features of protein can also be considered for prediction protein structure. Probably better results can be achieved by changing the number of linguistic variables or the type of membership function.References[1]Saha, S. (2008). Protein Secondary Structure Prediction by Fuzzy Min Max Neural Network withCompensatory Neurons (Doctoral dissertation, Indian Institute of Technology, Kharagpur).[2] E. Sakk, A. Alexander, On the variability of neural network classfication measures in the proteinsecondary structure prediction problem, Appl. Comput. Intell. Soft Comput. (2013) 1–9. Available online at 4/16/2015.(/journals/acisc/2013/794350/).[3]Tan, Y. T., &Rosdi, B. A. (2015). FPGA-based hardware accelerator for the prediction of proteinsecondary class via fuzzy K-nearest neighbors with Lempel–Ziv complexity based distance measure.Neurocomputing, 148, 409-419.[4]Babaei, S., Geranmayeh, A., &Seyyedsalehi, S. A. (2010). Protein secondary structure prediction usingmodular reciprocal bidirectional recurrent neural networks. Computer methods and programs in biomedicine, 100(3), 237-247.[5]Zhou, Z., Yang, B., &Hou, W. (2010). Association classification algorithm based on structure sequence inprotein secondary structure prediction. Expert Systems with Applications, 37(9), 6381-6389.[6]Qu, W., Sui, H., Yang, B., &Qian, W. (2011). Improving protein secondary structure prediction using amulti-modal BP method. Computers in biology and Medicine, 41(10), 946-959.[7]Hassan, R., Othman, R. M., Saad, P., &Kasim, S. (2011). A compact hybrid feature vector for an accuratesecondary structure prediction. Information Sciences, 181(23), 5267-5277.[8]Babaei, S., Geranmayeh, A., &Seyyedsalehi, S. A. (2012). Towards designing modular recurrent neuralnetworks in learning protein secondary structures.Expert Systems with Applications, 39(6), 6263-6274. [9]Zangooei, M. H., &Jalili, S. (2012). Protein secondary structure prediction using DWKF based on SVR-NSGAII. Neurocomputing, 94, 87-101.[10]Hayat, M., Tahir, M., & Khan, S. A. (2014). Prediction of protein structure classes using hybrid space ofmulti-profile Bayes and bi-gram probability feature spaces. Journal of theoretical biology, 346, 8-15. [11]Patel, M. S., &Mazumdar, H. S. (2014). Knowledge base and neural network approach for proteinsecondary structure prediction. Journal of theoretical biology, 361, 182-189.[12]Chen, Y., Xu, J., Yang, B., Zhao, Y., & He, W. (2012). A novel method for prediction of proteininteraction sites based on integrated RBF neural puters in biology and medicine, 42(4), 402-407.[13]Mishra, A., Rana, P. S., Mittal, A., &Jayaram, B. (2014). D2N: Distance to the native. Biochimica etBiophysicaActa (BBA)-Proteins and Proteomics, 1844(10), 1798-1807.[14]Iraji, M. S., &Motameni, H. (2012). Object Oriented Software Effort Estimate with Adaptive NeuroFuzzy use Case Size Point (ANFUSP). International Journal of Intelligent Systems and Applications (IJISA), 4(6), 14.Authors’ ProfilesMohammad Saber Iraji received B.Sc in Computer Software engineering from Shomaluniversity, Iran, Amol; M.Sc1in industrial engineering (system management and productivity)from Iran, Tehran and M.Sc2 in Computer Science. Currently, he is engaged in research andteaching on Computer Graphics, Image Processing, Fuzzy and Artificial Intelligent, DataMining, Software engineering and he is Faculty Member of Department of ComputerEngineering and Information Technology, Payame Noor University, I.R. of Iran.Hakimeh Ameri is graduated in M.S at K.N.Toosi Univerity of science and technology ininformation technology. Her main research interest is on bioinformatics, Data analysis andbig data. She has 7 published papers in this filed. She now teaching Artificial Intelligence,data mining, Information technology, programming languages and data structure inUniversity.How to cite this paper: Mohammad Saber Iraji, Hakimeh Ameri,"RMSD Protein Tertiary Structure Prediction with Soft Computing", International Journal of Mathematical Sciences and Computing(IJMSC), Vol.2, No.2, pp.24-33, 2016.DOI: 10.5815/ijmsc.2016.02.03。