NLM---Neuropeptide S interacts with the basolateral amygdala noradrenergic system in

T细胞怎样致力于阿尔茨海默病时海马神经元的再生阿尔茨海默病的发生与脑内神经元新生机制障碍有关，免疫学机制对神经元再生的影响成为阿尔茨海默病研究领域的关注点，T细胞是否有助于海马区神经元的再生目前还没有直接证据。

中国南方医科大学刘靖所在课题组进行的一项实验研究结果证实，T细胞有助于阿尔茨海默病时海马神经元再生，而T细胞缺陷时海马神经元再生受限，其机制可能与外周T细胞和中枢小胶质细胞产生的细胞因子表达增加有关。

实验为揭示T细胞在阿尔茨海默病病理中对神经元的作用提供了实验依据。

文章发表在《中国神经再生研究（英文版）》杂志2014年8月第16期。

免疫组织化学染色结果表明，注射Aβ1-42肽的BALB/c-nude小鼠海马神经元数量较少。

Article: "T cells promote the regeneration of neural precursor cells in the hippocampus of Alzheimer’s disease mice," by Jing Liu1, 2, Yuxin Ma2, Sumin Tian2, Li Zhang2, Mengmeng Zhao2, Yaqiong Zhang2, Dachuan Xu1 (1 Department of Human Anatomy, School of Basic Medical Sciences, Southern Medical University, Guangzhou, Guangdong Province, China；2 Department of Human Anatomy, School of Basic Courses, Guangdong Pharmaceutical University, Guangzhou, Guangdong Province, China)Liu J, Ma YX, Tian SM, Zhang L, Zhao MM, Zhang YQ, Xu DC. T cells promote the regenera tion of neural precursor cells in the hippocampus of Alzheimer’s disease mice. Neural Regen Res. 2014;9(16):1541-1547.欲获更多资讯：Neural Regen ResT cells promote the neural regeneration in the hippocampus of Alzheimer’s disease miceAlzheimer’s disease is closely associated with disorders of neurogen esis in the brain, and growing evidence supports the involvement of immunological mechanisms in the development of the disease. However, at present, the role of T cells in neuronal regeneration in the brain is unknown. Jing Liu and co-workers from Southern Medical University in China discovered that T cells promote hippocampal neurogenesis in Alzheimer’s disease and T-cell immunodeficiency restricts neuronal regeneration in the hippocampus. The mechanism underlying the promotion of neuronal regeneration by T cells is mediated by an increased expression of peripheral T cells and central microglial cytokines in Alzheimer’s disease mice. Their findings which have been reported in the Neural Regeneration Research (Vol. 9, No. 16, 2014) provide an experimental basis for understanding the role of T cells in Alzheimer’s disease.The number of neurons in the hippocampus decreased after hippocampal injection of Aβ1–42 (immunohistochemistry staining).Article: " T cells promote the regeneration of neural precursor cells in the hippocampus o f Alzheimer’s disease mice," by Jing Liu1, 2, Yuxin Ma2, Sumin Tian2, Li Zhang2, Mengmeng Zhao2, Yaqiong Zhang2, Dachuan Xu1 (1 Department of Human Anatomy, School of Basic Medical Sciences, Southern Medical University, Guangzhou, Guangdong Province, China；2 Department of Human Anatomy, School of Basic Courses, Guangdong Pharmaceutical University, Guangzhou, Guangdong Province, China)Liu J, Ma YX, Tian SM, Zhang L, Zhao MM, Zhang YQ, Xu DC. T cells promote the regeneration of neural precursor cells in the hippocampus of Alzheimer’s disease mice. Neural Regen Res. 2014;9(16):1541-1547.。

2 ＤＯＩ：１０．３９６９／ｊ．ｉｓｓｎ．１００１－５２５６．２０２３．０１．０２８细胞器之间相互作用在非酒精性脂肪性肝病发生发展中的作用刘天会首都医科大学附属北京友谊医院肝病中心，北京１０００５０通信作者：刘天会，ｌｉｕ＿ｔｉａｎｈｕｉ＠１６３．ｃｏｍ（ＯＲＣＩＤ：００００－０００１－６７８９－３０１６）摘要：细胞器除了具有各自特定的功能外，还可与其他细胞器相互作用完成重要的生理功能。

细胞器之间相互作用的异常与疾病的发生发展密切相关。

近年来，细胞器之间相互作用在非酒精性脂肪性肝病（ＮＡＦＬＤ）发生发展中的作用受到关注，特别是线粒体、脂滴与其他细胞器之间的相互作用。

关键词：非酒精性脂肪性肝病；细胞器；线粒体；脂肪滴基金项目：国家自然科学基金面上项目（８２０７０６１８）ＲｏｌｅｏｆｏｒｇａｎｅｌｌｅｉｎｔｅｒａｃｔｉｏｎｉｎｔｈｅｄｅｖｅｌｏｐｍｅｎｔａｎｄｐｒｏｇｒｅｓｓｉｏｎｏｆｎｏｎａｌｃｏｈｏｌｉｃｆａｔｔｙｌｉｖｅｒｄｉｓｅａｓｅＬＩＵＴｉａｎｈｕｉ．（ＬｉｖｅｒＲｅｓｅａｒｃｈＣｅｎｔｅｒ，ＢｅｉｊｉｎｇＦｒｉｅｎｄｓｈｉｐＨｏｓｐｉｔａｌ，ＣａｐｉｔａｌＭｅｄｉｃａｌＵｎｉｖｅｒｓｉｔｙ，Ｂｅｉｊｉｎｇ１０００５０，Ｃｈｉｎａ）Ｃｏｒｒｅｓｐｏｎｄｉｎｇａｕｔｈｏｒ：ＬＩＵＴｉａｎｈｕｉ，ｌｉｕ＿ｔｉａｎｈｕｉ＠１６３．ｃｏｍ（ＯＲＣＩＤ：００００－０００１－６７８９－３０１６）Ａｂｓｔｒａｃｔ：Ｉｎａｄｄｉｔｉｏｎｔｏｉｔｓｏｗｎｓｐｅｃｉｆｉｃｆｕｎｃｔｉｏｎｓ，ａｎｏｒｇａｎｅｌｌｅｃａｎａｌｓｏｉｎｔｅｒａｃｔｗｉｔｈｏｔｈｅｒｏｒｇａｎｅｌｌｅｓｔｏｃｏｍｐｌｅｔｅｉｍｐｏｒｔａｎｔｐｈｙｓｉｏｌｏｇｉｃａｌｆｕｎｃｔｉｏｎｓ．Ｔｈｅｄｉｓｏｒｄｅｒｓｏｆｏｒｇａｎｅｌｌｅｉｎｔｅｒａｃｔｉｏｎｓａｒｅｃｌｏｓｅｌｙａｓｓｏｃｉａｔｅｄｔｈｅｄｅｖｅｌｏｐｍｅｎｔａｎｄｐｒｏｇｒｅｓｓｉｏｎｏｆｖａｒｉｏｕｓｄｉｓｅａｓｅｓ．Ｉｎｒｅｃｅｎｔｙｅａｒｓ，ｔｈｅｒｏｌｅｏｆｏｒｇａｎｅｌｌｅｉｎｔｅｒａｃｔｉｏｎｓｈａｓａｔｔｒａｃｔｅｄｍｏｒｅａｔｔｅｎｔｉｏｎｉｎｔｈｅｐｒｏｇｒｅｓｓｉｏｎｏｆｎｏｎａｌｃｏｈｏｌｉｃｆａｔｔｙｌｉｖｅｒｄｉｓｅａｓｅ，ｅｓｐｅｃｉａｌｌｙｔｈｅｉｎｔｅｒａｃｔｉｏｎｓｂｅｔｗｅｅｎｍｉｔｏｃｈｏｎｄｒｉａ，ｌｉｐｉｄｄｒｏｐｌｅｔｓ，ａｎｄｏｔｈｅｒｏｒｇａｎｅｌｌｅｓ．Ｋｅｙｗｏｒｄｓ：Ｎｏｎ－ａｌｃｏｈｏｌｉｃＦａｔｔｙＬｉｖｅｒＤｉｓｅａｓｅ；Ｏｒｇａｎｅｌｌｅｓ；Ｍｉｔｏｃｈｏｎｄｒｉａ；ＬｉｐｉｄＤｒｏｐｌｅｔｓＲｅｓｅａｒｃｈｆｕｎｄｉｎｇ：ＮａｔｉｏｎａｌＮａｔｕｒａｌＳｃｉｅｎｃｅＦｏｕｎｄａｔｉｏｎｏｆＣｈｉｎａ（８２０７０６１８） 细胞器可以通过膜接触位点与其他细胞器相互作用，完成物质与信息的交换，形成互作网络［１］。

Draft:Deep Learning in Neural Networks:An OverviewTechnical Report IDSIA-03-14/arXiv:1404.7828(v1.5)[cs.NE]J¨u rgen SchmidhuberThe Swiss AI Lab IDSIAIstituto Dalle Molle di Studi sull’Intelligenza ArtificialeUniversity of Lugano&SUPSIGalleria2,6928Manno-LuganoSwitzerland15May2014AbstractIn recent years,deep artificial neural networks(including recurrent ones)have won numerous con-tests in pattern recognition and machine learning.This historical survey compactly summarises relevantwork,much of it from the previous millennium.Shallow and deep learners are distinguished by thedepth of their credit assignment paths,which are chains of possibly learnable,causal links between ac-tions and effects.I review deep supervised learning(also recapitulating the history of backpropagation),unsupervised learning,reinforcement learning&evolutionary computation,and indirect search for shortprograms encoding deep and large networks.PDF of earlier draft(v1):http://www.idsia.ch/∼juergen/DeepLearning30April2014.pdfLATEX source:http://www.idsia.ch/∼juergen/DeepLearning30April2014.texComplete BIBTEXfile:http://www.idsia.ch/∼juergen/bib.bibPrefaceThis is the draft of an invited Deep Learning(DL)overview.One of its goals is to assign credit to those who contributed to the present state of the art.I acknowledge the limitations of attempting to achieve this goal.The DL research community itself may be viewed as a continually evolving,deep network of scientists who have influenced each other in complex ways.Starting from recent DL results,I tried to trace back the origins of relevant ideas through the past half century and beyond,sometimes using“local search”to follow citations of citations backwards in time.Since not all DL publications properly acknowledge earlier relevant work,additional global search strategies were employed,aided by consulting numerous neural network experts.As a result,the present draft mostly consists of references(about800entries so far).Nevertheless,through an expert selection bias I may have missed important work.A related bias was surely introduced by my special familiarity with the work of my own DL research group in the past quarter-century.For these reasons,the present draft should be viewed as merely a snapshot of an ongoing credit assignment process.To help improve it,please do not hesitate to send corrections and suggestions to juergen@idsia.ch.Contents1Introduction to Deep Learning(DL)in Neural Networks(NNs)3 2Event-Oriented Notation for Activation Spreading in FNNs/RNNs3 3Depth of Credit Assignment Paths(CAPs)and of Problems4 4Recurring Themes of Deep Learning54.1Dynamic Programming(DP)for DL (5)4.2Unsupervised Learning(UL)Facilitating Supervised Learning(SL)and RL (6)4.3Occam’s Razor:Compression and Minimum Description Length(MDL) (6)4.4Learning Hierarchical Representations Through Deep SL,UL,RL (6)4.5Fast Graphics Processing Units(GPUs)for DL in NNs (6)5Supervised NNs,Some Helped by Unsupervised NNs75.11940s and Earlier (7)5.2Around1960:More Neurobiological Inspiration for DL (7)5.31965:Deep Networks Based on the Group Method of Data Handling(GMDH) (8)5.41979:Convolution+Weight Replication+Winner-Take-All(WTA) (8)5.51960-1981and Beyond:Development of Backpropagation(BP)for NNs (8)5.5.1BP for Weight-Sharing Feedforward NNs(FNNs)and Recurrent NNs(RNNs)..95.6Late1980s-2000:Numerous Improvements of NNs (9)5.6.1Ideas for Dealing with Long Time Lags and Deep CAPs (10)5.6.2Better BP Through Advanced Gradient Descent (10)5.6.3Discovering Low-Complexity,Problem-Solving NNs (11)5.6.4Potential Benefits of UL for SL (11)5.71987:UL Through Autoencoder(AE)Hierarchies (12)5.81989:BP for Convolutional NNs(CNNs) (13)5.91991:Fundamental Deep Learning Problem of Gradient Descent (13)5.101991:UL-Based History Compression Through a Deep Hierarchy of RNNs (14)5.111992:Max-Pooling(MP):Towards MPCNNs (14)5.121994:Contest-Winning Not So Deep NNs (15)5.131995:Supervised Recurrent Very Deep Learner(LSTM RNN) (15)5.142003:More Contest-Winning/Record-Setting,Often Not So Deep NNs (16)5.152006/7:Deep Belief Networks(DBNs)&AE Stacks Fine-Tuned by BP (17)5.162006/7:Improved CNNs/GPU-CNNs/BP-Trained MPCNNs (17)5.172009:First Official Competitions Won by RNNs,and with MPCNNs (18)5.182010:Plain Backprop(+Distortions)on GPU Yields Excellent Results (18)5.192011:MPCNNs on GPU Achieve Superhuman Vision Performance (18)5.202011:Hessian-Free Optimization for RNNs (19)5.212012:First Contests Won on ImageNet&Object Detection&Segmentation (19)5.222013-:More Contests and Benchmark Records (20)5.22.1Currently Successful Supervised Techniques:LSTM RNNs/GPU-MPCNNs (21)5.23Recent Tricks for Improving SL Deep NNs(Compare Sec.5.6.2,5.6.3) (21)5.24Consequences for Neuroscience (22)5.25DL with Spiking Neurons? (22)6DL in FNNs and RNNs for Reinforcement Learning(RL)236.1RL Through NN World Models Yields RNNs With Deep CAPs (23)6.2Deep FNNs for Traditional RL and Markov Decision Processes(MDPs) (24)6.3Deep RL RNNs for Partially Observable MDPs(POMDPs) (24)6.4RL Facilitated by Deep UL in FNNs and RNNs (25)6.5Deep Hierarchical RL(HRL)and Subgoal Learning with FNNs and RNNs (25)6.6Deep RL by Direct NN Search/Policy Gradients/Evolution (25)6.7Deep RL by Indirect Policy Search/Compressed NN Search (26)6.8Universal RL (27)7Conclusion271Introduction to Deep Learning(DL)in Neural Networks(NNs) Which modifiable components of a learning system are responsible for its success or failure?What changes to them improve performance?This has been called the fundamental credit assignment problem(Minsky, 1963).There are general credit assignment methods for universal problem solvers that are time-optimal in various theoretical senses(Sec.6.8).The present survey,however,will focus on the narrower,but now commercially important,subfield of Deep Learning(DL)in Artificial Neural Networks(NNs).We are interested in accurate credit assignment across possibly many,often nonlinear,computational stages of NNs.Shallow NN-like models have been around for many decades if not centuries(Sec.5.1).Models with several successive nonlinear layers of neurons date back at least to the1960s(Sec.5.3)and1970s(Sec.5.5). An efficient gradient descent method for teacher-based Supervised Learning(SL)in discrete,differentiable networks of arbitrary depth called backpropagation(BP)was developed in the1960s and1970s,and ap-plied to NNs in1981(Sec.5.5).BP-based training of deep NNs with many layers,however,had been found to be difficult in practice by the late1980s(Sec.5.6),and had become an explicit research subject by the early1990s(Sec.5.9).DL became practically feasible to some extent through the help of Unsupervised Learning(UL)(e.g.,Sec.5.10,5.15).The1990s and2000s also saw many improvements of purely super-vised DL(Sec.5).In the new millennium,deep NNs havefinally attracted wide-spread attention,mainly by outperforming alternative machine learning methods such as kernel machines(Vapnik,1995;Sch¨o lkopf et al.,1998)in numerous important applications.In fact,supervised deep NNs have won numerous of-ficial international pattern recognition competitions(e.g.,Sec.5.17,5.19,5.21,5.22),achieving thefirst superhuman visual pattern recognition results in limited domains(Sec.5.19).Deep NNs also have become relevant for the more generalfield of Reinforcement Learning(RL)where there is no supervising teacher (Sec.6).Both feedforward(acyclic)NNs(FNNs)and recurrent(cyclic)NNs(RNNs)have won contests(Sec.5.12,5.14,5.17,5.19,5.21,5.22).In a sense,RNNs are the deepest of all NNs(Sec.3)—they are general computers more powerful than FNNs,and can in principle create and process memories of ar-bitrary sequences of input patterns(e.g.,Siegelmann and Sontag,1991;Schmidhuber,1990a).Unlike traditional methods for automatic sequential program synthesis(e.g.,Waldinger and Lee,1969;Balzer, 1985;Soloway,1986;Deville and Lau,1994),RNNs can learn programs that mix sequential and parallel information processing in a natural and efficient way,exploiting the massive parallelism viewed as crucial for sustaining the rapid decline of computation cost observed over the past75years.The rest of this paper is structured as follows.Sec.2introduces a compact,event-oriented notation that is simple yet general enough to accommodate both FNNs and RNNs.Sec.3introduces the concept of Credit Assignment Paths(CAPs)to measure whether learning in a given NN application is of the deep or shallow type.Sec.4lists recurring themes of DL in SL,UL,and RL.Sec.5focuses on SL and UL,and on how UL can facilitate SL,although pure SL has become dominant in recent competitions(Sec.5.17-5.22). Sec.5is arranged in a historical timeline format with subsections on important inspirations and technical contributions.Sec.6on deep RL discusses traditional Dynamic Programming(DP)-based RL combined with gradient-based search techniques for SL or UL in deep NNs,as well as general methods for direct and indirect search in the weight space of deep FNNs and RNNs,including successful policy gradient and evolutionary methods.2Event-Oriented Notation for Activation Spreading in FNNs/RNNs Throughout this paper,let i,j,k,t,p,q,r denote positive integer variables assuming ranges implicit in the given contexts.Let n,m,T denote positive integer constants.An NN’s topology may change over time(e.g.,Fahlman,1991;Ring,1991;Weng et al.,1992;Fritzke, 1994).At any given moment,it can be described as afinite subset of units(or nodes or neurons)N= {u1,u2,...,}and afinite set H⊆N×N of directed edges or connections between nodes.FNNs are acyclic graphs,RNNs cyclic.Thefirst(input)layer is the set of input units,a subset of N.In FNNs,the k-th layer(k>1)is the set of all nodes u∈N such that there is an edge path of length k−1(but no longer path)between some input unit and u.There may be shortcut connections between distant layers.The NN’s behavior or program is determined by a set of real-valued,possibly modifiable,parameters or weights w i(i=1,...,n).We now focus on a singlefinite episode or epoch of information processing and activation spreading,without learning through weight changes.The following slightly unconventional notation is designed to compactly describe what is happening during the runtime of the system.During an episode,there is a partially causal sequence x t(t=1,...,T)of real values that I call events.Each x t is either an input set by the environment,or the activation of a unit that may directly depend on other x k(k<t)through a current NN topology-dependent set in t of indices k representing incoming causal connections or links.Let the function v encode topology information and map such event index pairs(k,t)to weight indices.For example,in the non-input case we may have x t=f t(net t)with real-valued net t= k∈in t x k w v(k,t)(additive case)or net t= k∈in t x k w v(k,t)(multiplicative case), where f t is a typically nonlinear real-valued activation function such as tanh.In many recent competition-winning NNs(Sec.5.19,5.21,5.22)there also are events of the type x t=max k∈int (x k);some networktypes may also use complex polynomial activation functions(Sec.5.3).x t may directly affect certain x k(k>t)through outgoing connections or links represented through a current set out t of indices k with t∈in k.Some non-input events are called output events.Note that many of the x t may refer to different,time-varying activations of the same unit in sequence-processing RNNs(e.g.,Williams,1989,“unfolding in time”),or also in FNNs sequentially exposed to time-varying input patterns of a large training set encoded as input events.During an episode,the same weight may get reused over and over again in topology-dependent ways,e.g.,in RNNs,or in convolutional NNs(Sec.5.4,5.8).I call this weight sharing across space and/or time.Weight sharing may greatly reduce the NN’s descriptive complexity,which is the number of bits of information required to describe the NN (Sec.4.3).In Supervised Learning(SL),certain NN output events x t may be associated with teacher-given,real-valued labels or targets d t yielding errors e t,e.g.,e t=1/2(x t−d t)2.A typical goal of supervised NN training is tofind weights that yield episodes with small total error E,the sum of all such e t.The hope is that the NN will generalize well in later episodes,causing only small errors on previously unseen sequences of input events.Many alternative error functions for SL and UL are possible.SL assumes that input events are independent of earlier output events(which may affect the environ-ment through actions causing subsequent perceptions).This assumption does not hold in the broaderfields of Sequential Decision Making and Reinforcement Learning(RL)(Kaelbling et al.,1996;Sutton and Barto, 1998;Hutter,2005)(Sec.6).In RL,some of the input events may encode real-valued reward signals given by the environment,and a typical goal is tofind weights that yield episodes with a high sum of reward signals,through sequences of appropriate output actions.Sec.5.5will use the notation above to compactly describe a central algorithm of DL,namely,back-propagation(BP)for supervised weight-sharing FNNs and RNNs.(FNNs may be viewed as RNNs with certainfixed zero weights.)Sec.6will address the more general RL case.3Depth of Credit Assignment Paths(CAPs)and of ProblemsTo measure whether credit assignment in a given NN application is of the deep or shallow type,I introduce the concept of Credit Assignment Paths or CAPs,which are chains of possibly causal links between events.Let usfirst focus on SL.Consider two events x p and x q(1≤p<q≤T).Depending on the appli-cation,they may have a Potential Direct Causal Connection(PDCC)expressed by the Boolean predicate pdcc(p,q),which is true if and only if p∈in q.Then the2-element list(p,q)is defined to be a CAP from p to q(a minimal one).A learning algorithm may be allowed to change w v(p,q)to improve performance in future episodes.More general,possibly indirect,Potential Causal Connections(PCC)are expressed by the recursively defined Boolean predicate pcc(p,q),which in the SL case is true only if pdcc(p,q),or if pcc(p,k)for some k and pdcc(k,q).In the latter case,appending q to any CAP from p to k yields a CAP from p to q(this is a recursive definition,too).The set of such CAPs may be large but isfinite.Note that the same weight may affect many different PDCCs between successive events listed by a given CAP,e.g.,in the case of RNNs, or weight-sharing FNNs.Suppose a CAP has the form(...,k,t,...,q),where k and t(possibly t=q)are thefirst successive elements with modifiable w v(k,t).Then the length of the suffix list(t,...,q)is called the CAP’s depth (which is0if there are no modifiable links at all).This depth limits how far backwards credit assignment can move down the causal chain tofind a modifiable weight.1Suppose an episode and its event sequence x1,...,x T satisfy a computable criterion used to decide whether a given problem has been solved(e.g.,total error E below some threshold).Then the set of used weights is called a solution to the problem,and the depth of the deepest CAP within the sequence is called the solution’s depth.There may be other solutions(yielding different event sequences)with different depths.Given somefixed NN topology,the smallest depth of any solution is called the problem’s depth.Sometimes we also speak of the depth of an architecture:SL FNNs withfixed topology imply a problem-independent maximal problem depth bounded by the number of non-input layers.Certain SL RNNs withfixed weights for all connections except those to output units(Jaeger,2001;Maass et al.,2002; Jaeger,2004;Schrauwen et al.,2007)have a maximal problem depth of1,because only thefinal links in the corresponding CAPs are modifiable.In general,however,RNNs may learn to solve problems of potentially unlimited depth.Note that the definitions above are solely based on the depths of causal chains,and agnostic of the temporal distance between events.For example,shallow FNNs perceiving large“time windows”of in-put events may correctly classify long input sequences through appropriate output events,and thus solve shallow problems involving long time lags between relevant events.At which problem depth does Shallow Learning end,and Deep Learning begin?Discussions with DL experts have not yet yielded a conclusive response to this question.Instead of committing myself to a precise answer,let me just define for the purposes of this overview:problems of depth>10require Very Deep Learning.The difficulty of a problem may have little to do with its depth.Some NNs can quickly learn to solve certain deep problems,e.g.,through random weight guessing(Sec.5.9)or other types of direct search (Sec.6.6)or indirect search(Sec.6.7)in weight space,or through training an NNfirst on shallow problems whose solutions may then generalize to deep problems,or through collapsing sequences of(non)linear operations into a single(non)linear operation—but see an analysis of non-trivial aspects of deep linear networks(Baldi and Hornik,1994,Section B).In general,however,finding an NN that precisely models a given training set is an NP-complete problem(Judd,1990;Blum and Rivest,1992),also in the case of deep NNs(S´ıma,1994;de Souto et al.,1999;Windisch,2005);compare a survey of negative results(S´ıma, 2002,Section1).Above we have focused on SL.In the more general case of RL in unknown environments,pcc(p,q) is also true if x p is an output event and x q any later input event—any action may affect the environment and thus any later perception.(In the real world,the environment may even influence non-input events computed on a physical hardware entangled with the entire universe,but this is ignored here.)It is possible to model and replace such unmodifiable environmental PCCs through a part of the NN that has already learned to predict(through some of its units)input events(including reward signals)from former input events and actions(Sec.6.1).Its weights are frozen,but can help to assign credit to other,still modifiable weights used to compute actions(Sec.6.1).This approach may lead to very deep CAPs though.Some DL research is about automatically rephrasing problems such that their depth is reduced(Sec.4). In particular,sometimes UL is used to make SL problems less deep,e.g.,Sec.5.10.Often Dynamic Programming(Sec.4.1)is used to facilitate certain traditional RL problems,e.g.,Sec.6.2.Sec.5focuses on CAPs for SL,Sec.6on the more complex case of RL.4Recurring Themes of Deep Learning4.1Dynamic Programming(DP)for DLOne recurring theme of DL is Dynamic Programming(DP)(Bellman,1957),which can help to facili-tate credit assignment under certain assumptions.For example,in SL NNs,backpropagation itself can 1An alternative would be to count only modifiable links when measuring depth.In many typical NN applications this would not make a difference,but in some it would,e.g.,Sec.6.1.be viewed as a DP-derived method(Sec.5.5).In traditional RL based on strong Markovian assumptions, DP-derived methods can help to greatly reduce problem depth(Sec.6.2).DP algorithms are also essen-tial for systems that combine concepts of NNs and graphical models,such as Hidden Markov Models (HMMs)(Stratonovich,1960;Baum and Petrie,1966)and Expectation Maximization(EM)(Dempster et al.,1977),e.g.,(Bottou,1991;Bengio,1991;Bourlard and Morgan,1994;Baldi and Chauvin,1996; Jordan and Sejnowski,2001;Bishop,2006;Poon and Domingos,2011;Dahl et al.,2012;Hinton et al., 2012a).4.2Unsupervised Learning(UL)Facilitating Supervised Learning(SL)and RL Another recurring theme is how UL can facilitate both SL(Sec.5)and RL(Sec.6).UL(Sec.5.6.4) is normally used to encode raw incoming data such as video or speech streams in a form that is more convenient for subsequent goal-directed learning.In particular,codes that describe the original data in a less redundant or more compact way can be fed into SL(Sec.5.10,5.15)or RL machines(Sec.6.4),whose search spaces may thus become smaller(and whose CAPs shallower)than those necessary for dealing with the raw data.UL is closely connected to the topics of regularization and compression(Sec.4.3,5.6.3). 4.3Occam’s Razor:Compression and Minimum Description Length(MDL) Occam’s razor favors simple solutions over complex ones.Given some programming language,the prin-ciple of Minimum Description Length(MDL)can be used to measure the complexity of a solution candi-date by the length of the shortest program that computes it(e.g.,Solomonoff,1964;Kolmogorov,1965b; Chaitin,1966;Wallace and Boulton,1968;Levin,1973a;Rissanen,1986;Blumer et al.,1987;Li and Vit´a nyi,1997;Gr¨u nwald et al.,2005).Some methods explicitly take into account program runtime(Al-lender,1992;Watanabe,1992;Schmidhuber,2002,1995);many consider only programs with constant runtime,written in non-universal programming languages(e.g.,Rissanen,1986;Hinton and van Camp, 1993).In the NN case,the MDL principle suggests that low NN weight complexity corresponds to high NN probability in the Bayesian view(e.g.,MacKay,1992;Buntine and Weigend,1991;De Freitas,2003), and to high generalization performance(e.g.,Baum and Haussler,1989),without overfitting the training data.Many methods have been proposed for regularizing NNs,that is,searching for solution-computing, low-complexity SL NNs(Sec.5.6.3)and RL NNs(Sec.6.7).This is closely related to certain UL methods (Sec.4.2,5.6.4).4.4Learning Hierarchical Representations Through Deep SL,UL,RLMany methods of Good Old-Fashioned Artificial Intelligence(GOFAI)(Nilsson,1980)as well as more recent approaches to AI(Russell et al.,1995)and Machine Learning(Mitchell,1997)learn hierarchies of more and more abstract data representations.For example,certain methods of syntactic pattern recog-nition(Fu,1977)such as grammar induction discover hierarchies of formal rules to model observations. The partially(un)supervised Automated Mathematician/EURISKO(Lenat,1983;Lenat and Brown,1984) continually learns concepts by combining previously learnt concepts.Such hierarchical representation learning(Ring,1994;Bengio et al.,2013;Deng and Yu,2014)is also a recurring theme of DL NNs for SL (Sec.5),UL-aided SL(Sec.5.7,5.10,5.15),and hierarchical RL(Sec.6.5).Often,abstract hierarchical representations are natural by-products of data compression(Sec.4.3),e.g.,Sec.5.10.4.5Fast Graphics Processing Units(GPUs)for DL in NNsWhile the previous millennium saw several attempts at creating fast NN-specific hardware(e.g.,Jackel et al.,1990;Faggin,1992;Ramacher et al.,1993;Widrow et al.,1994;Heemskerk,1995;Korkin et al., 1997;Urlbe,1999),and at exploiting standard hardware(e.g.,Anguita et al.,1994;Muller et al.,1995; Anguita and Gomes,1996),the new millennium brought a DL breakthrough in form of cheap,multi-processor graphics cards or GPUs.GPUs are widely used for video games,a huge and competitive market that has driven down hardware prices.GPUs excel at fast matrix and vector multiplications required not only for convincing virtual realities but also for NN training,where they can speed up learning by a factorof50and more.Some of the GPU-based FNN implementations(Sec.5.16-5.19)have greatly contributed to recent successes in contests for pattern recognition(Sec.5.19-5.22),image segmentation(Sec.5.21), and object detection(Sec.5.21-5.22).5Supervised NNs,Some Helped by Unsupervised NNsThe main focus of current practical applications is on Supervised Learning(SL),which has dominated re-cent pattern recognition contests(Sec.5.17-5.22).Several methods,however,use additional Unsupervised Learning(UL)to facilitate SL(Sec.5.7,5.10,5.15).It does make sense to treat SL and UL in the same section:often gradient-based methods,such as BP(Sec.5.5.1),are used to optimize objective functions of both UL and SL,and the boundary between SL and UL may blur,for example,when it comes to time series prediction and sequence classification,e.g.,Sec.5.10,5.12.A historical timeline format will help to arrange subsections on important inspirations and techni-cal contributions(although such a subsection may span a time interval of many years).Sec.5.1briefly mentions early,shallow NN models since the1940s,Sec.5.2additional early neurobiological inspiration relevant for modern Deep Learning(DL).Sec.5.3is about GMDH networks(since1965),perhaps thefirst (feedforward)DL systems.Sec.5.4is about the relatively deep Neocognitron NN(1979)which is similar to certain modern deep FNN architectures,as it combines convolutional NNs(CNNs),weight pattern repli-cation,and winner-take-all(WTA)mechanisms.Sec.5.5uses the notation of Sec.2to compactly describe a central algorithm of DL,namely,backpropagation(BP)for supervised weight-sharing FNNs and RNNs. It also summarizes the history of BP1960-1981and beyond.Sec.5.6describes problems encountered in the late1980s with BP for deep NNs,and mentions several ideas from the previous millennium to overcome them.Sec.5.7discusses afirst hierarchical stack of coupled UL-based Autoencoders(AEs)—this concept resurfaced in the new millennium(Sec.5.15).Sec.5.8is about applying BP to CNNs,which is important for today’s DL applications.Sec.5.9explains BP’s Fundamental DL Problem(of vanishing/exploding gradients)discovered in1991.Sec.5.10explains how a deep RNN stack of1991(the History Compressor) pre-trained by UL helped to solve previously unlearnable DL benchmarks requiring Credit Assignment Paths(CAPs,Sec.3)of depth1000and more.Sec.5.11discusses a particular WTA method called Max-Pooling(MP)important in today’s DL FNNs.Sec.5.12mentions afirst important contest won by SL NNs in1994.Sec.5.13describes a purely supervised DL RNN(Long Short-Term Memory,LSTM)for problems of depth1000and more.Sec.5.14mentions an early contest of2003won by an ensemble of shallow NNs, as well as good pattern recognition results with CNNs and LSTM RNNs(2003).Sec.5.15is mostly about Deep Belief Networks(DBNs,2006)and related stacks of Autoencoders(AEs,Sec.5.7)pre-trained by UL to facilitate BP-based SL.Sec.5.16mentions thefirst BP-trained MPCNNs(2007)and GPU-CNNs(2006). Sec.5.17-5.22focus on official competitions with secret test sets won by(mostly purely supervised)DL NNs since2009,in sequence recognition,image classification,image segmentation,and object detection. Many RNN results depended on LSTM(Sec.5.13);many FNN results depended on GPU-based FNN code developed since2004(Sec.5.16,5.17,5.18,5.19),in particular,GPU-MPCNNs(Sec.5.19).5.11940s and EarlierNN research started in the1940s(e.g.,McCulloch and Pitts,1943;Hebb,1949);compare also later work on learning NNs(Rosenblatt,1958,1962;Widrow and Hoff,1962;Grossberg,1969;Kohonen,1972; von der Malsburg,1973;Narendra and Thathatchar,1974;Willshaw and von der Malsburg,1976;Palm, 1980;Hopfield,1982).In a sense NNs have been around even longer,since early supervised NNs were essentially variants of linear regression methods going back at least to the early1800s(e.g.,Legendre, 1805;Gauss,1809,1821).Early NNs had a maximal CAP depth of1(Sec.3).5.2Around1960:More Neurobiological Inspiration for DLSimple cells and complex cells were found in the cat’s visual cortex(e.g.,Hubel and Wiesel,1962;Wiesel and Hubel,1959).These cellsfire in response to certain properties of visual sensory inputs,such as theorientation of plex cells exhibit more spatial invariance than simple cells.This inspired later deep NN architectures(Sec.5.4)used in certain modern award-winning Deep Learners(Sec.5.19-5.22).5.31965:Deep Networks Based on the Group Method of Data Handling(GMDH) Networks trained by the Group Method of Data Handling(GMDH)(Ivakhnenko and Lapa,1965; Ivakhnenko et al.,1967;Ivakhnenko,1968,1971)were perhaps thefirst DL systems of the Feedforward Multilayer Perceptron type.The units of GMDH nets may have polynomial activation functions imple-menting Kolmogorov-Gabor polynomials(more general than traditional NN activation functions).Given a training set,layers are incrementally grown and trained by regression analysis,then pruned with the help of a separate validation set(using today’s terminology),where Decision Regularisation is used to weed out superfluous units.The numbers of layers and units per layer can be learned in problem-dependent fashion. This is a good example of hierarchical representation learning(Sec.4.4).There have been numerous ap-plications of GMDH-style networks,e.g.(Ikeda et al.,1976;Farlow,1984;Madala and Ivakhnenko,1994; Ivakhnenko,1995;Kondo,1998;Kord´ık et al.,2003;Witczak et al.,2006;Kondo and Ueno,2008).5.41979:Convolution+Weight Replication+Winner-Take-All(WTA)Apart from deep GMDH networks(Sec.5.3),the Neocognitron(Fukushima,1979,1980,2013a)was per-haps thefirst artificial NN that deserved the attribute deep,and thefirst to incorporate the neurophysiolog-ical insights of Sec.5.2.It introduced convolutional NNs(today often called CNNs or convnets),where the(typically rectangular)receptivefield of a convolutional unit with given weight vector is shifted step by step across a2-dimensional array of input values,such as the pixels of an image.The resulting2D array of subsequent activation events of this unit can then provide inputs to higher-level units,and so on.Due to massive weight replication(Sec.2),relatively few parameters may be necessary to describe the behavior of such a convolutional layer.Competition layers have WTA subsets whose maximally active units are the only ones to adopt non-zero activation values.They essentially“down-sample”the competition layer’s input.This helps to create units whose responses are insensitive to small image shifts(compare Sec.5.2).The Neocognitron is very similar to the architecture of modern,contest-winning,purely super-vised,feedforward,gradient-based Deep Learners with alternating convolutional and competition lay-ers(e.g.,Sec.5.19-5.22).Fukushima,however,did not set the weights by supervised backpropagation (Sec.5.5,5.8),but by local un supervised learning rules(e.g.,Fukushima,2013b),or by pre-wiring.In that sense he did not care for the DL problem(Sec.5.9),although his architecture was comparatively deep indeed.He also used Spatial Averaging(Fukushima,1980,2011)instead of Max-Pooling(MP,Sec.5.11), currently a particularly convenient and popular WTA mechanism.Today’s CNN-based DL machines profita lot from later CNN work(e.g.,LeCun et al.,1989;Ranzato et al.,2007)(Sec.5.8,5.16,5.19).5.51960-1981and Beyond:Development of Backpropagation(BP)for NNsThe minimisation of errors through gradient descent(Hadamard,1908)in the parameter space of com-plex,nonlinear,differentiable,multi-stage,NN-related systems has been discussed at least since the early 1960s(e.g.,Kelley,1960;Bryson,1961;Bryson and Denham,1961;Pontryagin et al.,1961;Dreyfus,1962; Wilkinson,1965;Amari,1967;Bryson and Ho,1969;Director and Rohrer,1969;Griewank,2012),ini-tially within the framework of Euler-LaGrange equations in the Calculus of Variations(e.g.,Euler,1744). Steepest descent in such systems can be performed(Bryson,1961;Kelley,1960;Bryson and Ho,1969)by iterating the ancient chain rule(Leibniz,1676;L’Hˆo pital,1696)in Dynamic Programming(DP)style(Bell-man,1957).A simplified derivation of the method uses the chain rule only(Dreyfus,1962).The methods of the1960s were already efficient in the DP sense.However,they backpropagated derivative information through standard Jacobian matrix calculations from one“layer”to the previous one, explicitly addressing neither direct links across several layers nor potential additional efficiency gains due to network sparsity(but perhaps such enhancements seemed obvious to the authors).。

J Apoplexy and Nervous Diseases, July 2024, Vol 41，No. 7偏头痛发病机制及生物标志物研究进展毛西京， 朱博驰综述， 于挺敏审校摘要： 偏头痛是一种具有多种亚表型的异质性疾病，其诊断主要基于临床标准，缺乏特异性的生物标志物进行客观评估，影响了偏头痛的精确诊断、治疗选择以及预后评估。

近年来偏头痛在遗传、生化、影像等方面研究取得重大进展，为临床诊断及治疗偏头痛提供了客观的检测指标。

如能在临床工作中选择特异性、敏感性、易检测、可行性高的标志物将推动偏头痛早期诊断、精准化治疗的步伐。

关键词： 偏头痛； 生物标志物； 神经元； 胶质细胞中图分类号：R747.2 文献标识码：A Research advances in the pathogenesis and biomarkers of migraine MAO Xijing ，ZHU Bochi ，YU Tingmin. （The Sec⁃ond Hospital of Jilin University ， Changchun 130000， China ）Abstract ： Migraine is a heterogeneous disease with various subtypes ， and the diagnosis of migraine mainly relies on clinical criteria. The lack of specific biomarkers for objective assessment impacts the precise diagnosis ， treatment selec⁃tion ， and prognostic assessment of migraine. In recent years ， great progress has been made in migraine in terms of genet⁃ics ， biochemistry ，and imaging ， which provides objective indicators for the clinical diagnosis and treatment of migraine. Identifying specific ，sensitive ，easily detectable ，and highly feasible markers in clinical practice will accelerate the early di⁃agnosis and precise treatment of migraine.Key words ： Migraine ； Biomarkers ； Neurons ； Glial cells偏头痛的发病机制尚不完全明确，越来越多的研究发现神经元-神经胶质细胞-血管交互作用的炎性病理生理过程参与其中，并且从血液、脑脊液、唾液、影像检查中均发现了有意义的标志物，这些标志物成为偏头痛诊疗的潜在靶点。

- 151 -[28] MUBDER M，AZAB M，JAYARAJ M，et al. Autoimmunehepatitis in patients with human immunodeficiency virus infection: a systematic review of the published literature[J/OL]. Medicine (Baltimore)，2019，98（37）：e17094.https:///31517833/.[29] STEFANOU M I，KRUMBHOLZ M，ZIEMANN U，et al.Human immunodeficiency virus and multiple sclerosis: a review of the literature[J]. Neurol Res Pract，2019，1：24.[30] KARAMPOOR S，ZAHEDNASAB H，BOKHARAEI-SALIM F，et al. HIV-1 Tat protein attenuates the clinical course of experimental autoimmune encephalomyelitis (EAE)[J]. Int Immunopharmacol，2020，78：105943.[31] FERNANDEZ-GUTIERREZ B. COVID-19 with pulmonaryinvolvement. an autoimmune disease of known cause[J]. Reumatol Clin (Engl Ed)，2020，16（4）：253-254.[32] ZHOU Y，HAN T，CHEN J，et al. Clinical and autoimmunecharacteristics of severe and critical cases of COVID-19[J]. Clin Transl Sci，2020，13（6）：1077-1086.[33] CASO F，COSTA L，RUSCITTI P，et al. Could Sars-coronavirus-2 trigger autoimmune and/or autoinflammatory mechanisms in genetically predisposed subjects?[J]. Autoimmun Rev，2020，19（5）：102524.[34] GAGIANNIS D，STEINESTEL J，HACKENBROCH C，et al.Clinical，serological，and histopathological similarities between severe COVID-19 and acute exacerbation of connective tissue disease-associated interstitial lung disease (CTD-ILD)[J]. Front Immunol，2020，11：587517.[35] TSAO H S，CHASON H M，FEARON D M. Immunethrombocytopenia (ITP) in a pediatric patient positive for SARS-CoV-2[J/OL]. Pediatrics，2020，146（2）：e20201419.https:///32439817/.[36] SZEKANECZ Z，BALOG A，CONSTANTIN T，et al.COVID-19: autoimmunity, multisystemic inflammation and autoimmune rheumatic patients[J/OL]. Expert Rev Mol Med，2022，24：e13.https:///35311631/.（收稿日期：2023-08-24）*基金项目：国家自然科学基金项目（81774243）；江苏省中医药管理局科技项目（JD2022SZ03）①南京中医药大学第一临床医学院　江苏　南京　210023②南京中医药大学附属医院通信作者：赵崧中药治疗胃食管反流病的研究进展*李牛牛①　赵崧② 【摘要】　胃食管反流病是消化内科的常见病，目前西医治疗主要以抑酸为主，但对于气体反流、非酸反流、内脏高敏感患者的疗效欠佳，抑酸不能调节反流本身，并可能因其强力抑酸作用造成不良反应。

外周免疫系统与中枢神经系统疾病交互影响的机制研究①李葛韩根成（军事医学研究院，北京100850）中图分类号R392文献标志码A文章编号1000-484X（2021）22-2689-05［摘要］脑科学作为21世纪的新兴学科，是探索人类认知功能的基础，也是解决神经系统疾病的关键学科。

近年来研究证实脑内免疫稳态对于正常神经认知功能的维持有重要作用，且脑内与外周免疫系统间存在相互影响，但在中枢神经系统疾病中，脑内免疫反应的机制，尤其是脑内与外周免疫应答相互影响的机制尚不明确。

本文拟对几种中枢神经系统疾病中外周与脑内免疫系统交互影响的机制研究进展进行综述。

［关键词］中枢神经系统；血脑屏障；细胞因子；中枢神经系统疾病Research on mechanism of interaction between peripheral immune system and central nervous system diseasesLI Ge，HAN Gen-Cheng.Academic of Military Medical Science，Beijing100850，China［Abstract］Brain science as a new discipline during the21st century，has established the key basement to explore the basis of human cognitive functions and to solve neurological diseases.In recent years，studies have confirmed that the immune homeostasis in the brain plays an important role in the maintenance of normal neurocognitive functions，and there is an interaction between the brain and the peripheral immune system.However，in central nervous system diseases，the mechanism of the brain immune response，espe‐cially the crosstalk of brain and peripheral immune responses is remaining fully known.This article intends to review the research prog‐ress on mechanism of the interaction between the peripheral and the brain immune system in several central nervous system diseases.［Key words］CNS；BBB；Cytokine；Central nervous system disease近年来，神经系统疾病已成为全球致死的主要原因，而由于社会环境和工作压力等因素的影响，中枢神经系统疾病发病率持续升高，且其还受遗传、感染及机体免疫失衡等复杂因素影响，目前对于中枢神经系统疾病的诊断及治疗仍然是被关注的焦点。

Deep Sparse Rectifier Neural NetworksXavier Glorot Antoine Bordes Yoshua BengioDIRO,Universit´e de Montr´e al Montr´e al,QC,Canada glorotxa@iro.umontreal.ca Heudiasyc,UMR CNRS6599UTC,Compi`e gne,FranceandDIRO,Universit´e de Montr´e alMontr´e al,QC,Canadaantoine.bordes@hds.utc.frDIRO,Universit´e de Montr´e alMontr´e al,QC,Canadabengioy@iro.umontreal.caAbstractWhile logistic sigmoid neurons are more bi-ologically plausible than hyperbolic tangentneurons,the latter work better for train-ing multi-layer neural networks.This pa-per shows that rectifying neurons are aneven better model of biological neurons andyield equal or better performance than hy-perbolic tangent networks in spite of thehard non-linearity and non-differentiabilityat zero,creating sparse representations withtrue zeros,which seem remarkably suitablefor naturally sparse data.Even though theycan take advantage of semi-supervised setupswith extra-unlabeled data,deep rectifier net-works can reach their best performance with-out requiring any unsupervised pre-trainingon purely supervised tasks with large labeleddatasets.Hence,these results can be seen asa new milestone in the attempts at under-standing the difficulty in training deep butpurely supervised neural networks,and clos-ing the performance gap between neural net-works learnt with and without unsupervisedpre-training.1IntroductionMany differences exist between the neural network models used by machine learning researchers and those used by computational neuroscientists.This is in part Appearing in Proceedings of the14th International Con-ference on Artificial Intelligence and Statistics(AISTATS) 2011,Fort Lauderdale,FL,USA.Volume15of JMLR: W&CP15.Copyright2011by the authors.because the objective of the former is to obtain com-putationally efficient learners,that generalize well to new examples,whereas the objective of the latter is to abstract out neuroscientific data while obtaining ex-planations of the principles involved,providing predic-tions and guidance for future biological experiments. Areas where both objectives coincide are therefore particularly worthy of investigation,pointing towards computationally motivated principles of operation in the brain that can also enhance research in artificial intelligence.In this paper we show that two com-mon gaps between computational neuroscience models and machine learning neural network models can be bridged by using the following linear by part activa-tion:max(0,x),called the rectifier(or hinge)activa-tion function.Experimental results will show engaging training behavior of this activation function,especially for deep architectures(see Bengio(2009)for a review), i.e.,where the number of hidden layers in the neural network is3or more.Recent theoretical and empirical work in statistical machine learning has demonstrated the importance of learning algorithms for deep architectures.This is in part inspired by observations of the mammalian vi-sual cortex,which consists of a chain of processing elements,each of which is associated with a different representation of the raw visual input.This is partic-ularly clear in the primate visual system(Serre et al., 2007),with its sequence of processing stages:detection of edges,primitive shapes,and moving up to gradu-ally more complex visual shapes.Interestingly,it was found that the features learned in deep architectures resemble those observed in thefirst two of these stages (in areas V1and V2of visual cortex)(Lee et al.,2008), and that they become increasingly invariant to factors of variation(such as camera movement)in higher lay-ers(Goodfellow et al.,2009).Deep Sparse Rectifier Neural NetworksRegarding the training of deep networks,something that can be considered a breakthrough happened in2006,with the introduction of Deep Belief Net-works(Hinton et al.,2006),and more generally the idea of initializing each layer by unsupervised learn-ing(Bengio et al.,2007;Ranzato et al.,2007).Some authors have tried to understand why this unsuper-vised procedure helps(Erhan et al.,2010)while oth-ers investigated why the original training procedure for deep neural networks failed(Bengio and Glorot,2010). From the machine learning point of view,this paper brings additional results in these lines of investigation. We propose to explore the use of rectifying non-linearities as alternatives to the hyperbolic tangent or sigmoid in deep artificial neural networks,in ad-dition to using an L1regularizer on the activation val-ues to promote sparsity and prevent potential numer-ical problems with unbounded activation.Nair and Hinton(2010)present promising results of the influ-ence of such units in the context of Restricted Boltz-mann Machines compared to logistic sigmoid activa-tions on image classification tasks.Our work extends this for the case of pre-training using denoising auto-encoders(Vincent et al.,2008)and provides an exten-sive empirical comparison of the rectifying activation function against the hyperbolic tangent on image clas-sification benchmarks as well as an original derivation for the text application of sentiment analysis.Our experiments on image and text data indicate that training proceeds better when the artificial neurons are either offor operating mostly in a linear regime.Sur-prisingly,rectifying activation allows deep networks to achieve their best performance without unsupervised pre-training.Hence,our work proposes a new contri-bution to the trend of understanding and merging the performance gap between deep networks learnt with and without unsupervised pre-training(Erhan et al., 2010;Bengio and Glorot,2010).Still,rectifier net-works can benefit from unsupervised pre-training in the context of semi-supervised learning where large amounts of unlabeled data are provided.Furthermore, as rectifier units naturally lead to sparse networks and are closer to biological neurons’responses in their main operating regime,this work also bridges(in part)a machine learning/neuroscience gap in terms of acti-vation function and sparsity.This paper is organized as follows.Section2presents some neuroscience and machine learning background which inspired this work.Section3introduces recti-fier neurons and explains their potential benefits and drawbacks in deep networks.Then we propose an experimental study with empirical results on image recognition in Section4.1and sentiment analysis in Section4.2.Section5presents our conclusions.2Background2.1Neuroscience ObservationsFor models of biological neurons,the activation func-tion is the expectedfiring rate as a function of the total input currently arising out of incoming signals at synapses(Dayan and Abott,2001).An activation function is termed,respectively antisymmetric or sym-metric when its response to the opposite of a strongly excitatory input pattern is respectively a strongly in-hibitory or excitatory one,and one-sided when this response is zero.The main gaps that we wish to con-sider between computational neuroscience models and machine learning models are the following:•Studies on brain energy expense suggest that neurons encode information in a sparse and dis-tributed way(Attwell and Laughlin,2001),esti-mating the percentage of neurons active at the same time to be between1and4%(Lennie,2003).This corresponds to a trade-offbetween richness of representation and small action potential en-ergy expenditure.Without additional regulariza-tion,such as an L1penalty,ordinary feedforward neural nets do not have this property.For ex-ample,the sigmoid activation has a steady state regime around12,therefore,after initializing with small weights,all neuronsfire at half their satura-tion regime.This is biologically implausible and hurts gradient-based optimization(LeCun et al., 1998;Bengio and Glorot,2010).•Important divergences between biological and machine learning models concern non-linear activation functions.A common biological model of neuron,the leaky integrate-and-fire(or LIF)(Dayan and Abott,2001),gives the follow-ing relation between thefiring rate and the input current,illustrated in Figure1(left):f(I)=τlogE+RI−V rE+RI−V th+t ref−1,if E+RI>V th0,if E+RI≤V thwhere t ref is the refractory period(minimal time between two action potentials),I the input cur-rent,V r the resting potential and V th the thresh-old potential(with V th>V r),and R,E,τthe membrane resistance,potential and time con-stant.The most commonly used activation func-tions in the deep learning and neural networks lit-erature are the standard logistic sigmoid and theXavier Glorot,Antoine Bordes,YoshuaBengioFigure1:Left:Common neural activation function motivated by biological data.Right:Commonly used activation functions in neural networks literature:logistic sigmoid and hyperbolic tangent(tanh).hyperbolic tangent(see Figure1,right),which areequivalent up to a linear transformation.The hy-perbolic tangent has a steady state at0,and istherefore preferred from the optimization stand-point(LeCun et al.,1998;Bengio and Glorot,2010),but it forces an antisymmetry around0which is absent in biological neurons.2.2Advantages of SparsitySparsity has become a concept of interest,not only incomputational neuroscience and machine learning butalso in statistics and signal processing(Candes andTao,2005).It wasfirst introduced in computationalneuroscience in the context of sparse coding in the vi-sual system(Olshausen and Field,1997).It has beena key element of deep convolutional networks exploit-ing a variant of auto-encoders(Ranzato et al.,2007,2008;Mairal et al.,2009)with a sparse distributedrepresentation,and has also become a key ingredientin Deep Belief Networks(Lee et al.,2008).A sparsitypenalty has been used in several computational neuro-science(Olshausen and Field,1997;Doi et al.,2006)and machine learning models(Lee et al.,2007;Mairalet al.,2009),in particular for deep architectures(Leeet al.,2008;Ranzato et al.,2007,2008).However,inthe latter,the neurons end up taking small but non-zero activation orfiring probability.We show here thatusing a rectifying non-linearity gives rise to real zerosof activations and thus truly sparse representations.From a computational point of view,such representa-tions are appealing for the following reasons:•Information disentangling.One of theclaimed objectives of deep learning algo-rithms(Bengio,2009)is to disentangle thefactors explaining the variations in the data.Adense representation is highly entangled becausealmost any change in the input modifies most ofthe entries in the representation vector.Instead,if a representation is both sparse and robust tosmall input changes,the set of non-zero featuresis almost always roughly conserved by smallchanges of the input.•Efficient variable-size representation.Dif-ferent inputs may contain different amounts of in-formation and would be more conveniently repre-sented using a variable-size data-structure,whichis common in computer representations of infor-mation.Varying the number of active neuronsallows a model to control the effective dimension-ality of the representation for a given input andthe required precision.•Linear separability.Sparse representations arealso more likely to be linearly separable,or moreeasily separable with less non-linear machinery,simply because the information is represented ina high-dimensional space.Besides,this can reflectthe original data format.In text-related applica-tions for instance,the original raw data is alreadyvery sparse(see Section4.2).•Distributed but sparse.Dense distributed rep-resentations are the richest representations,be-ing potentially exponentially more efficient thanpurely local ones(Bengio,2009).Sparse repre-sentations’efficiency is still exponentially greater,with the power of the exponent being the numberof non-zero features.They may represent a goodtrade-offwith respect to the above criteria.Nevertheless,forcing too much sparsity may hurt pre-dictive performance for an equal number of neurons,because it reduces the effective capacity of the model.Deep Sparse Rectifier NeuralNetworksFigure 2:Left:Sparse propagation of activations and gradients in a network of rectifier units.The input selects a subset of active neurons and computation is linear in this subset.Right:Rectifier and softplus activation functions.The second one is a smooth version of the first.3Deep Rectifier Networks3.1Rectifier NeuronsThe neuroscience literature (Bush and Sejnowski,1995;Douglas and al.,2003)indicates that corti-cal neurons are rarely in their maximum saturation regime ,and suggests that their activation function can be approximated by a rectifier.Most previous stud-ies of neural networks involving a rectifying activation function concern recurrent networks (Salinas and Ab-bott,1996;Hahnloser,1998).The rectifier function rectifier(x )=max(0,x )is one-sided and therefore does not enforce a sign symmetry 1or antisymmetry 1:instead,the response to the oppo-site of an excitatory input pattern is 0(no response).However,we can obtain symmetry or antisymmetry by combining two rectifier units sharing parameters.Advantages The rectifier activation function allows a network to easily obtain sparse representations.For example,after uniform initialization of the weights,around 50%of hidden units continuous output val-ues are real zeros,and this fraction can easily increase with sparsity-inducing regularization.Apart from be-ing more biologically plausible,sparsity also leads to mathematical advantages (see previous section).As illustrated in Figure 2(left),the only non-linearity in the network comes from the path selection associ-ated with individual neurons being active or not.For a given input only a subset of neurons are active .Com-putation is linear on this subset:once this subset of neurons is selected,the output is a linear function of1The hyperbolic tangent absolute value non-linearity |tanh(x )|used by Jarrett et al.(2009)enforces sign symme-try.A tanh(x )non-linearity enforces sign antisymmetry.the input (although a large enough change can trigger a discrete change of the active set of neurons).The function computed by each neuron or by the network output in terms of the network input is thus linear by parts.We can see the model as an exponential num-ber of linear models that share parameters (Nair and Hinton,2010).Because of this linearity,gradients flow well on the active paths of neurons (there is no gra-dient vanishing effect due to activation non-linearities of sigmoid or tanh units),and mathematical investi-gation is putations are also cheaper:there is no need for computing the exponential function in activations,and sparsity can be exploited.Potential Problems One may hypothesize that the hard saturation at 0may hurt optimization by block-ing gradient back-propagation.To evaluate the poten-tial impact of this effect we also investigate the soft-plus activation:softplus (x )=log (1+e x )(Dugas et al.,2001),a smooth version of the rectifying non-linearity.We lose the exact sparsity,but may hope to gain eas-ier training.However,experimental results (see Sec-tion 4.1)tend to contradict that hypothesis,suggesting that hard zeros can actually help supervised training.We hypothesize that the hard non-linearities do not hurt so long as the gradient can propagate along some paths ,i.e.,that some of the hidden units in each layer are non-zero.With the credit and blame assigned to these ON units rather than distributed more evenly,we hypothesize that optimization is easier.Another prob-lem could arise due to the unbounded behavior of the activations;one may thus want to use a regularizer to prevent potential numerical problems.Therefore,we use the L 1penalty on the activation values,which also promotes additional sparsity.Also recall that,in or-der to efficiently represent symmetric/antisymmetric behavior in the data,a rectifier network would needXavier Glorot,Antoine Bordes,Yoshua Bengiotwice as many hidden units as a network of symmet-ric/antisymmetric activation functions.Finally,rectifier networks are subject to ill-conditioning of the parametrization.Biases and weights can be scaled in different (and consistent)ways while preserving the same overall network function.More precisely,consider for each layer of depth i of the network a scalar αi ,and scaling the parameters asW i =W iαi and b i =b i ij =1αj.The output units values then change as follow:s =sn j =1αj .Therefore,aslong as nj =1αj is 1,the network function is identical.3.2Unsupervised Pre-trainingThis paper is particularly inspired by the sparse repre-sentations learned in the context of auto-encoder vari-ants,as they have been found to be very useful intraining deep architectures (Bengio,2009),especially for unsupervised pre-training of neural networks (Er-han et al.,2010).Nonetheless,certain difficulties arise when one wants to introduce rectifier activations into stacked denois-ing auto-encoders (Vincent et al.,2008).First,the hard saturation below the threshold of the rectifier function is not suited for the reconstruction units.In-deed,whenever the network happens to reconstruct a zero in place of a non-zero target,the reconstruc-tion unit can not backpropagate any gradient.2Sec-ond,the unbounded behavior of the rectifier activation also needs to be taken into account.In the follow-ing,we denote ˜x the corrupted version of the input x ,σ()the logistic sigmoid function and θthe model pa-rameters (W enc ,b enc ,W dec ,b dec ),and define the linear recontruction function as:f (x,θ)=W dec max(W enc x +b enc ,0)+b dec .Here are the several strategies we have experimented:e a softplus activation function for the recon-struction layer,along with a quadratic cost:L (x,θ)=||x −log(1+exp(f (˜x ,θ)))||2.2.Scale the rectifier activation values coming from the previous encoding layer to bound them be-tween 0and 1,then use a sigmoid activation func-tion for the reconstruction layer,along with a cross-entropy reconstruction cost.L (x,θ)=−x log(σ(f (˜x ,θ)))−(1−x )log(1−σ(f (˜x ,θ))).2Why is this not a problem for hidden layers too?we hy-pothesize that it is because gradients can still flow throughthe active (non-zero),possibly helping rather than hurting the assignment of credit.e a linear activation function for the reconstruc-tion layer,along with a quadratic cost.We triedto use input unit values either before or after the rectifier non-linearity as reconstruction targets.(For the first layer,raw inputs are directly used.)e a rectifier activation function for the recon-struction layer,along with a quadratic cost.The first strategy has proven to yield better gener-alization on image data and the second one on text data.Consequently,the following experimental study presents results using those two.4Experimental StudyThis section discusses our empirical evaluation of recti-fier units for deep networks.We first compare them to hyperbolic tangent and softplus activations on image benchmarks with and without pre-training,and then apply them to the text task of sentiment analysis.4.1Image RecognitionExperimental setup We considered the image datasets detailed below.Each of them has a train-ing set (for tuning parameters),a validation set (for tuning hyper-parameters)and a test set (for report-ing generalization performance).They are presented according to their number of training/validation/test examples,their respective image sizes,as well as their number of classes:•MNIST (LeCun et al.,1998):50k/10k/10k,28×28digit images,10classes.•CIFAR10(Krizhevsky and Hinton,2009):50k/5k/5k,32×32×3RGB images,10classes.•NISTP:81,920k/80k/20k,32×32character im-ages from the NIST database 19,with randomized distortions (Bengio and al,2010),62classes.This dataset is much larger and more difficult than the original NIST (Grother,1995).•NORB:233,172/58,428/58,320,taken from Jittered-Cluttered NORB (LeCun et al.,2004).Stereo-pair images of toys on a cluttered background,6classes.The data has been prepro-cessed similarly to (Nair and Hinton,2010):we subsampled the original 2×108×108stereo-pair images to 2×32×32and scaled linearly the image in the range [−1,1].We followed the procedure used by Nair and Hinton (2010)to create the validation set.Deep Sparse Rectifier Neural NetworksTable1:Test error on networks of depth3.Bold results represent statistical equivalence between similar ex-periments,with and without pre-training,under the null hypothesis of the pairwise test with p=0.05.Neuron MNIST CIF AR10NISTP NORB With unsupervised pre-trainingRectifier 1.20%49.96%32.86%16.46% Tanh 1.16%50.79%35.89%17.66% Softplus 1.17%49.52%33.27%19.19% Without unsupervised pre-trainingRectifier 1.43%50.86%32.64%16.40% Tanh 1.57%52.62%36.46%19.29% Softplus 1.77%53.20%35.48%17.68% For all experiments except on the NORB data(Le-Cun et al.,2004),the models we used are stacked denoising auto-encoders(Vincent et al.,2008)with three hidden layers and1000units per layer.The ar-chitecture of Nair and Hinton(2010)has been used on NORB:two hidden layers with respectively4000 and2000units.We used a cross-entropy reconstruc-tion cost for tanh networks and a quadratic cost over a softplus reconstruction layer for the rectifier and softplus networks.We chose masking noise as the corruption process:each pixel has a probability of0.25of being artificially set to0.The unsuper-vised learning rate is constant,and the following val-ues have been explored:{.1,.01,.001,.0001}.We se-lect the model with the lowest reconstruction error. For the supervisedfine-tuning we chose a constant learning rate in the same range as the unsupervised learning rate with respect to the supervised valida-tion error.The training cost is the negative log likeli-hood−log P(correct class|input)where the probabil-ities are obtained from the output layer(which imple-ments a softmax logistic regression).We used stochas-tic gradient descent with mini-batches of size10for both unsupervised and supervised training phases.To take into account the potential problem of rectifier units not being symmetric around0,we use a vari-ant of the activation function for whichhalf of the units output values are multiplied by-1.This serves to cancel out the mean activation value for each layer and can be interpreted either as inhibitory neurons or simply as a way to equalize activations numerically. Additionally,an L1penalty on the activations with a coefficient of0.001was added to the cost function dur-ing pre-training andfine-tuning in order to increase the amount of sparsity in the learned representations. Main results Table1summarizes the results on networks of3hidden layers of1000hidden units each,Figure3:Influence offinal sparsity on accu-racy.200randomly initialized deep rectifier networks were trained on MNIST with various L1penalties(from 0to0.01)to obtain different sparsity levels.Results show that enforcing sparsity of the activation does not hurtfinal performance until around85%of true zeros.comparing all the neuron types3on all the datasets, with or without unsupervised pre-training.In the lat-ter case,the supervised training phase has been carried out using the same experimental setup as the one de-scribed above forfine-tuning.The main observations we make are the following:•Despite the hard threshold at0,networks trained with the rectifier activation function canfind lo-cal minima of greater or equal quality than those obtained with its smooth counterpart,the soft-plus.On NORB,we tested a rescaled version of the softplus defined by1αsoftplus(αx),which allows to interpolate in a smooth manner be-tween the softplus(α=1)and the rectifier(α=∞).We obtained the followingα/test error cou-ples:1/17.68%,1.3/17.53%,2/16.9%,3/16.66%, 6/16.54%,∞/16.40%.There is no trade-offbe-tween those activation functions.Rectifiers are not only biologically plausible,they are also com-putationally efficient.•There is almost no improvement when using un-supervised pre-training with rectifier activations, contrary to what is experienced using tanh or soft-plus.Purely supervised rectifier networks remain competitive on all4datasets,even against the pretrained tanh or softplus models.3We also tested a rescaled version of the LIF and max(tanh(x),0)as activation functions.We obtained worse generalization performance than those of Table1, and chose not to report them.Xavier Glorot,Antoine Bordes,Yoshua Bengio•Rectifier networks are truly deep sparse networks.There is an average exact sparsity(fraction of ze-ros)of the hidden layers of83.4%on MNIST,72.0%on CIFAR10,68.0%on NISTP and73.8%on NORB.Figure3provides a better understand-ing of the influence of sparsity.It displays the MNIST test error of deep rectifier networks(with-out pre-training)according to different average sparsity obtained by varying the L1penalty on the works appear to be quite ro-bust to it as models with70%to almost85%of true zeros can achieve similar performances. With labeled data,deep rectifier networks appear to be attractive models.They are biologically credible, and,compared to their standard counterparts,do not seem to depend as much on unsupervised pre-training, while ultimately yielding sparse representations.This last conclusion is slightly different from those re-ported in(Nair and Hinton,2010)in which is demon-strated that unsupervised pre-training with Restricted Boltzmann Machines and using rectifier units is ben-eficial.In particular,the paper reports that pre-trained rectified Deep Belief Networks can achieve a test error on NORB below16%.However,we be-lieve that our results are compatible with those:we extend the experimental framework to a different kind of models(stacked denoising auto-encoders)and dif-ferent datasets(on which conclusions seem to be differ-ent).Furthermore,note that our rectified model with-out pre-training on NORB is very competitive(16.4% error)and outperforms the17.6%error of the non-pretrained model from Nair and Hinton(2010),which is basically what wefind with the non-pretrained soft-plus units(17.68%error).Semi-supervised setting Figure4presents re-sults of semi-supervised experiments conducted on the NORB dataset.We vary the percentage of the orig-inal labeled training set which is used for the super-vised training phase of the rectifier and hyperbolic tan-gent networks and evaluate the effect of the unsuper-vised pre-training(using the whole training set,unla-beled).Confirming conclusions of Erhan et al.(2010), the network with hyperbolic tangent activations im-proves with unsupervised pre-training for any labeled set size(even when all the training set is labeled). However,the picture changes with rectifying activa-tions.In semi-supervised setups(with few labeled data),the pre-training is highly beneficial.But the more the labeled set grows,the closer the models with and without pre-training.Eventually,when all avail-able data is labeled,the two models achieve identical performance.Rectifier networks can maximally ex-ploit labeled and unlabeledinformation.Figure4:Effect of unsupervised pre-training.On NORB,we compare hyperbolic tangent and rectifier net-works,with or without unsupervised pre-training,andfine-tune only on subsets of increasing size of the training set.4.2Sentiment AnalysisNair and Hinton(2010)also demonstrated that recti-fier units were efficient for image-related tasks.They mentioned the intensity equivariance property(i.e. without bias parameters the network function is lin-early variant to intensity changes in the input)as ar-gument to explain this observation.This would sug-gest that rectifying activation is mostly useful to im-age data.In this section,we investigate on a different modality to cast a fresh light on rectifier units.A recent study(Zhou et al.,2010)shows that Deep Be-lief Networks with binary units are competitive with the state-of-the-art methods for sentiment analysis. This indicates that deep learning is appropriate to this text task which seems therefore ideal to observe the behavior of rectifier units on a different modality,and provide a data point towards the hypothesis that rec-tifier nets are particarly appropriate for sparse input vectors,such as found in NLP.Sentiment analysis is a text mining area which aims to determine the judg-ment of a writer with respect to a given topic(see (Pang and Lee,2008)for a review).The basic task consists in classifying the polarity of reviews either by predicting whether the expressed opinions are positive or negative,or by assigning them star ratings on either 3,4or5star scales.Following a task originally proposed by Snyder and Barzilay(2007),our data consists of restaurant reviews which have been extracted from the restaurant review site .We have access to10,000 labeled and300,000unlabeled training reviews,while the test set contains10,000examples.The goal is to predict the rating on a5star scale and performance is evaluated using Root Mean Squared Error(RMSE).4 4Even though our tasks are identical,our database is。

PCR 之引物设计--------基因序列查询篇•确定目标基因的形态：DNA or RNA•查找目标基因•比对目标基因同源性和特点•选取目标基因序列•选取目标基因的目标区段第一步：如何查基因：1、mRNA 序列的查询；以查大鼠cort为例；/search 选择11、在NCBI上搜索到目的基因，找到该基因的mRNA，Copy该ｍＲＮＡ序列作为软件查询序列的候选对象。

该mRNA文件的命名：Rattusnorvegicuscortistatin (Cort), mRNA；NM_012835：是唯一的编号；把以下序列存成ｔｘｔ文件：Rattusnorvegicuscortistatin (Cort), mRNA* Comment* Features* SequenceLOCUS NM_012835 438 bp mRNA linear ROD 11-FEB-2008DEFINITION Rattusnorvegicuscortistatin (Cort), mRNA.ACCESSION NM_012835VERSION NM_012835.1 GI:6978682KEYWORDS .SOURCE Rattusnorvegicus (Norway rat)ORGANISM RattusnorvegicusEukaryota; Metazoa; Chordata; Craniata; Vertebrata; Euteleostomi;Mammalia; Eutheria; Euarchontoglires; Glires; Rodentia;Sciurognathi; Muroidea; Muridae; Murinae; Rattus.REFERENCE 1 (bases 1 to 438)AUTHORS Xidakis,C., Mastrodimou,N., Notas,G., Renieri,E., Kolios,G.,Kouroumalis,E. and Thermos,K.TITLE RT-PCR and immunocytochemistry studies support the presence ofsomatostatin, cortistatin and somatostatin receptor subtypes in ratKupffer cellsJOURNAL Regul.Pept. 143 (1-3), 76-82 (2007)PUBMED 17481746REMARK GeneRIF: RT-PCR and immunocytochemistry studies support the presence of cortistatin in rat Kupffer cells.REFERENCE 2 (bases 1 to 438)AUTHORS Bourgin,P., Fabre,V., Huitron-Resendiz,S., Henriksen,S.J., Prospero-Garcia,O., Criado,J.R. and de Lecea,L.TITLE Cortistatin promotes and negatively correlates with slow-wave sleepJOURNAL Eur. J. Neurosci. 26 (3), 729-738 (2007)PUBMED 17686045REMARK GeneRIF: The capacity of CST-14 to increase SWA, together with preprocortistatin's inverse correlation with time spent in SWS, suggests a potential role in sleep homeostatic processes.REFERENCE 3 (bases 1 to 438)AUTHORS Deghenghi,R., Avallone,R., Torsello,A., Muccioli,G., Ghigo,E. and Locatelli,V.TITLE Growth hormone-inhibiting activity of cortistatin in the ratJOURNAL J. Endocrinol. Invest. 24 (11), RC31-RC33 (2001)PUBMED 11817718REFERENCE 4 (bases 1 to 438)AUTHORS de Lecea,L., Criado,J.R., Prospero-Garcia,O., Gautvik,K.M., Schweitzer,P., Danielson,P.E., Dunlop,C.L., Siggins,G.R.,Henriksen,S.J. and Sutcliffe,J.G.TITLE A cortical neuropeptide with neuronal depressant andsleep-modulating propertiesJOURNAL Nature 381 (6579), 242-245 (1996)PUBMED 8622767COMMENT PROVISIONAL REFSEQ: This record has not yet been subject to final NCBI review. The reference sequence was derived from U51919.1.Summary: inhibits growth hormone secretion; may act as aneuropeptide to mediate signaling via a somatostatin receptorsubtype [RGD].FEATURES Location/Qualifierssource 1..438/organism="Rattusnorvegicus"/mol_type="mRNA"/strain="Sprague-Dawley"/db_xref="taxon:10116"/chromosome="5"/map="5q36"gene 1..438/gene="Cort"/note="cortistatin"/db_xref="GeneID:25305"/db_xref="RATMAP:41189"/db_xref="RGD:2383"CDS 30..368/gene="Cort"/note="Preprocortistatin"/codon_start=1/product="cortistatin"/protein_id="NP_036967.1"/db_xref="GI:6978683"/db_xref="GeneID:25305"/db_xref="RATMAP:41189"/db_xref="RGD:2383"/translation="MGGCSTRGKRPSALSLLLLLLLSGIAASALPLESGPTGQDSVQDATGGRRTGLLTFLAWWHEWASQDSSSTAFEGGTPELSKRQERPPLQQPPHRDKKPC KNFFWKTFSSCK"sig_peptide 30..110/gene="Cort"mat_peptide 324..365/gene="Cort"/product="cortistatin-14"ORIGIN1 aaagcacagacttcaggtctccaaggaggatgggtggctgcagcacaagaggcaagcggc61 cgtcagccctcagtctgctgctgctgctgctgctctcggggatcgcagcctctgccctcc121 ccctggagagcggtcccaccggccaggacagtgtgcaggatgccacaggcgggaggagga181 ccggccttctgactttccttgcctggtggcatgagtgggcttcccaagacagctccagca241 ccgctttcgaagggggtaccccggagctgtctaagcggcaggaaagaccacccctccagc301 agcccccacaccgggataaaaagccctgcaagaacttcttctggaaaaccttctcctcgt361 gcaagtagcccgagcctgaccggagcctgaccggccaccctgtgaatgcagccgtggcct421 gaataaagagtgtcaagt//其中origin后面的序列，是我们用来设计的序列。

The Secrets of Good Healthneurotransmitter神经递质, 神经传递素immune system viruses and bacteriahinder newlywed cold viruspessimistic / optimistic immune cellimmunotransmitter免疫递质chronic pain relaxation therapyrespiration belly breathing energize1Exercise. Eat right. Don't smoke. These are some of the most common words of advice to people who wish to stay healthy. But a growing amount of scientific research shows that there is another, equally important, aspect to staying well, peace of mind.2Think about how your heart races while you are waiting to be called into the doctor's office or how unhappy a bad headache can make you. There is a two-way connection between mind and body. When one is bothered, the other feels it.3At the heart of the communications network are brain chemicals called neurotransmitters which communicate messages not only within the brain, but also within the body. One key receptor, the immune system, is a network of cells and organs that work to fight off viruses and bacteria.4When you experience joy, fear, or relaxation, the immune system may increase or decrease production of disease-fighting cells, thus helping or hindering you in fighting diseases such as the flu, or even some cancer.5By now, how the immune system is affected by stress has been well documented. In one study involving newlywed couples, for example, those who showed hostile behavior during a 30-minute discussion about marriage problems had lower immune functioning for the 24-hour period following the experiment than people who showed less negative behavior.6It is not just stress that can do damage. One researcher thought that if the same cold virus was put under two different noses, the person who is depressed or anxious or pessimistic would be more likely to develop the cold.7What is it about stress and related emotions that can encourage poor health? These feelings can cause the production of substances that damage or weaken our immune cells. Negative emotions can also cause our bodies to produce fewer immunotransmitters which ultimately help fight off disease.8If stress, depression, anger and other negative feelings can make you more likely to get sick, can the reverse be tree? Will you have a stronger immune response and greater health if you are happier, less stressed, and more optimistic? Experts believe that the answer is yes.9There are studies showing that by employing certain mind-body techniques that help reduce stress and improve outlook, cancer patients can live longer. But cancer patients aren't the only ones who can benefit. Certain mind-body techniques can help all of us.10Research has found that when patients with chronic pain used relaxation therapies and other behavioral techniques to manage discomfort, they reduced their visits to the doctor by 36 percent.11Relaxation produces better health through deep, rhythmic breathing, muscle loosening, anda slower heart rate. When some of the tension is taken out of the body, the strain is taken offthe entire system.12Relaxation decreases blood pressure, heart rate and respiration and increases one's sense of well-being. That is important because a body that is constantly tense will eventually give out. 13There are dozens of mind-body techniques for you to choose from. The key is to find one you're comfortable with and then do it regularly. Simply writing about negative, unpleasant events may actually boost your immunity according to researchers.14Scientists are not completely sure why it works but they know that when individuals write, it helps them organize events, which in turn gives them more understanding of the situation. 15When you can give a stressful experience meaning through writing, you don't think about it or worry about it as much. And when you reduce stress, you boost immune functioning. How much you write or how long you write depends upon how much stress you feel about the event.16One doctor suggests that people write until they are tired of writing and then read over what they have written. This helps make more sense of it. Also, just talking about a stressful experience with a friend can have the same positive effect.17Study after study has shown that people with good support systems -- caring, helpful family, friends and co-workers -- have better health. Researchers think that the understanding we get from them reduces stress, which in turn helps the immune system.18As one psychologist states, "When you have someone who loves you and cares about you to share your problems and feelings with, you don't feel you have to fight your problems, or the world, alone."19Another interesting study has shown that the more diverse your social network, the better, that people who have a number of different social relationships have a lower risk of getting colds than those with fewer.20There are other fast but effective mind-body relaxation techniques. One could be called "belly breathing". Sit in a comfortable chair in a quiet room. Close your eyes. Breathe through your nose, fill your belly with air, then slowly release the breath through your mouth.21Another technique could be called "mindfulness". Take a slow walk and be aware of exactly what is happening to you at each moment whether the wind is on your face, an insect is flying near you, or you hear birds singing.22Even if you continue thinking about problems, you will become calmer and distance yourself from your problems. If you are at home, you might dance. Put on some fast music, close the door, and let yourself go. The dancing will energize you and that alone will make you feel better.23Whichever mind-body techniques work best for you, never rely on them and them alone to keep you mentally and physically well. Like exercise, good nutrition and proper medical care, methods such as relaxation therapies are only one part of the recipe for good health. Still, they are an important ingredient.。