人工智能(第八章)(Artificial intelligence (the eighth chapter))

人工智能(第八章)（Artificial intelligence (the eighthchapter)）OneThe eighth chapter, artificial neural networkOneThe eighth chapter, artificial neural networkKey experiment of ground mechanical bionics technology of Ministry of education of Jilin University8.1 basic concepts and composition characteristics of neural networksIn a broad sense, the neural network usually includes two aspects: biological neural network and artificial neural network.A biological neural network is a complex network of nerves composed of the central nervous system and the peripheral nervous system of animalsIt is responsible for the management of the various activities of the animal body, the most important of which is the cerebral nervous system.Artificial neural network (ANN) is to simulate the structure and function of the human brain nervous system, and use a largenumber of soft and hardware processing units,A network system built manually by extensive parallel interconnection.Structure and functional characteristics of 8.1.1 biological neuronsBiological neurons, usually called nerve cells, are made up of living beingsThe basic unit of the nervous system, referred to as a neuron.Neurons mainly consist of three parts, including cell bodies,The basic structures of axons and dendrites are shown here.1. the structure of biological neuronsBiological neuron structureTwoKey experiment of ground mechanical bionics technology of Ministry of education of Jilin UniversityFunctional properties of 2. neuronsFrom the point of view of biological cybernetics, neurons, which are the basic unit of control and information processing, haveList some features and features.(1) space-time integration function;The neuron has the function of time integration for the information transmitted by the same synapse at different time;The information of different synaptic afferents has spatial integration function. The combination of the two functions makes biological neurons possessSpatio temporal integration of input information processing functions.(2) the dynamic polarization of neuronsAlthough different neurons differ markedly in shape and function, most neurons do predictIn the direction of which information flows.(3) excitatory and inhibitory states;Neurons have two conventional working states, namely excitatory state and inhibitory state.(4) plasticity of structure;The characteristics of synaptic transmission information are variable, and the transmission function can be enhanced withthe change of nerve impulse transmission modeWeak, so the connection between neurons is flexible, which is called structural plasticity.ThreeKey experiment of ground mechanical bionics technology of Ministry of education of Jilin University(5) pulse and potential signal conversionSynaptic interface has the function of pulse and potential signal conversion.(6) synaptic delay and refractory period;Synaptic transmission of information is delayed and refractory, requiring a certain time interval between two adjacent inputs,In the meantime, it does not affect incentives and does not convey information, which is called refractory period.(7) learning, forgetting and fatigueBecause of structural plasticity, synaptic transmission is enhanced, attenuated, and saturated, so nerve cells also respond accordinglyLearning, forgetting, or fatigue effects (saturation effects).The composition and structure of 8.1.2 artificial neural network1. artificial neural networkArtificial neural networks (Artificial, Neural, Nets, ANN) are composed of a large number of processing units that are extensively interconnectedArtificial neural networks are used to simulate the structure and function of the brain nervous system. These processing units are called artificial neurons.Artificial neural networks can be viewed as directed graphs with artificial neurons as nodes and connected by directed weighted arcs.In directed graphs, artificial neurons are the simulation of biological neurons, while directed arcs are simulations of axon synapse dendritic pairs,The weights of the directed arcs represent the strength of the interaction between the two artificial neurons that are interconnected.Key experiment of ground mechanical bionics technology of Ministry of education of Jilin UniversityAs shown in the figure for artificial neural network composed of sketch.It consists of multiple artificial neurons interconnected.The circle represents the cell body of the neuron; XRepresents the external input of the neuron; W is theThe connections between neurons and each input are strongDegrees are called connection weights.For an individual worker neuron in ANN, the input from other neurons is multiplied by the weights, and then the phasePlus. Compare all sums with the threshold level. When the sum is above the threshold level, the output is 1Then, the output is 0.As shown in the figure for artificial neural network composed of sketch.It consists of multiple artificial neurons interconnected.The circle represents the cell body of the neuron; XRepresents the external input of the neuron; W is theThe connections between neurons and each input are strongDegrees are called connection weights.For an individual worker neuron in ANN, the input from other neurons is multiplied by the weights, and then the phasePlus. Compare all sums with the threshold level. When the sum is above the threshold level, the output is 1Then, the output is 0.FiveKey experiment of ground mechanical bionics technology of Ministry of education of Jilin University2. the working process of artificial neuronsFor each neuron in an artificial neural network, it can accept a set of other neurons from the systemThe input signal, each input corresponds to one weight, and the weighted sum of all inputs determines the activation state of the neuron. thisInside, each right is equivalent to the synaptic strength of the synapse".For a neuron, it is assumed that information from other neurons, I, is Xi, and that they interact with the neuronWith intensity, that is, the connection weight is wi, i=0,1,... N-1, the internal threshold of the neuron is theta.Then the input of this neuron is: the output of the neuron is:.NOne)In the formula, Xi is the input of the I element, and wi is the interconnection weight between the I neuron and the neuron. F is called an excitation function or functionFunction that determines the output of a neuron.The output of the neuron is 1 or 0 depending on the sum of its inputs greater than or less than the internal threshold theta.NOneIZero.Sigma =F (.Y= Wixi Wixi =I Zero order Sigma =Wixi.NOneIs called activation value.IZero.Sigma=SixKey experiment of ground mechanical bionics technology of Ministry of education of Jilin UniversityThe excitation function generally has nonlinearcharacteristics. The commonly used nonlinear excitation functions have threshold type and piecewise linear type,The Sigmoid function type (referred to as "S") and the hyperbolic tangent type are shown in the figure.The threshold type function is also called the step function,It represents the relationship between the activation value Sigma and its output f (sigma). This two valued modelA neuron whose output is valued at 1 or 0, respectively, represents the excitatory and inhibitory states of neurons. At some point, GodThe state of the yuan is determined by the excitation function F.When the activation value is Sigma >0, the neuron is activated and excited, and its state f (sigma) is 1;When the activation value is Sigma <0, the neuron is not activated and enters a state of suppression, with the state f (sigma) of 0.F (sigma)Sigma 10F (sigma)Sigma 10F (sigma)Sigma 10F (sigma)Sigma 10-1 (a), threshold type (b), piecewise linear (c), Sigmoid function type (d), hyperbolic tangent typeThe excitation function generally has nonlinear characteristics. The commonly used nonlinear excitation functions have threshold type and piecewise linear type,The Sigmoid function type (referred to as "S") and the hyperbolic tangent type are shown in the figure.The threshold type function, also called the step function, represents the relation between the activation value Sigma and its output f (sigma). This two valued modelA neuron whose output is valued at 1 or 0, respectively, represents the excitatory and inhibitory states of neurons. At some point, GodThe state of the yuan is determined by the excitation function F.When the activation value is Sigma >0, the neuron is activated and excited, and its state f (sigma) is 1;When the activation value is Sigma <0, the neuron is not activated and enters a state of suppression, with the state f (sigma) of 0.F (sigma)Sigma 10F (sigma)Sigma 10F (sigma)Sigma 10F (sigma)Sigma 10-1 (a), threshold type (b), piecewise linear (c), Sigmoid function type (d), hyperbolic tangent typeSevenKey experiment of ground mechanical bionics technology of Ministry of education of Jilin UniversityPiecewise linear function can be regarded as one of the simplest nonlinear functions. It is characterized by limiting the range of functionsWithin a certain range, the input and output meet the linear relationship in a certain range and continue until the output is the mostDomain value. However, when the maximum value is reached, the output will no longer increase. This maximum is called saturation.The S type function is a nonlinear function with the maximum output value, whose output value is continuously valued in a certain range.Neurons whose activation functions are also saturated.The hyperbolic tangent type function is actually a special S type function whose saturation values are - 1 and 1.In artificial neural networks, different connections of neurons form different connection models of networks.3. structure of artificial neural networksCommon connection models are:Forward networkA network of feedback from the input layer to the output layerThere are interconnected networks in the layerAny two neurons in the network can be interconnected by the internet.Piecewise linear function can be regarded as one of the simplest nonlinear functions. It is characterized by limiting the range of functionsWithin a certain range, the input and output meet the linear relationship in a certain range and continue until the output is the mostDomain value. However, when the maximum value is reached, the output will no longer increase. This maximum is calledsaturation.The S type function is a nonlinear function with the maximum output value, whose output value is continuously valued in a certain range.Neurons whose activation functions are also saturated.The hyperbolic tangent type function is actually a special S type function whose saturation values are - 1 and 1.In the artificial neural network,The different connections of neurons form different connection models of networks.3. structure of artificial neural networksCommon connection models are:Forward networkA network of feedback from the input layer to the output layerThere are interconnected networks in the layerAny two neurons in the network can be interconnected by the internet.EightKey experiment of ground mechanical bionics technology of Ministry of education of Jilin University4. classification and main characteristics of artificial neural networksThe neural network model can obtain different classification results from different angles:According to the performance of the network, it can be divided into continuous and discrete networks, and can be divided into deterministic and stochastic networksAccording to the topological structure of the network, it can be divided into feedback network and no feedback network;According to the method of network learning, it can be divided into a learning network with teachers and a learning network without teachers;According to the properties of synaptic connections, they can be divided into first order linear associative networks and higher order nonlinear associated networks;Artificial neural networks have the following main characteristics:(1) it can simulate human's image thinking better;(2) large-scale parallel processing capability;(3) strong learning ability;(4) it has strong fault tolerance and association ability;(5) it is a large-scale, self-organizing and adaptive nonlinear dynamical system.NineKey experiment of ground mechanical bionics technology of Ministry of education of Jilin University8.2 perceptron model and its learning algorithmmethod8.2.1 perceptron modelPerceptron is one of the earliest artificial neural networks models designed and implemented.In 1957, American scholar Rosen Bo Larter (Rosenblatt) proposed a neural network model with self-learning abilityType perceptron model, which leads the research of neural network to engineering realization from pure theory.Perceptron model, also called single layer perceptron, consists of input part and output layer, and the output layer is its computing layer.In the single-layer perceptron model, the input part of the neuron is connected to the various neurons in the output layer. When the input section willWhen the input data is transmitted to the connected neuron, the output layer weights the input data and generates the product via the threshold functionGenerate a set of output data.As shown, a single layer perceptron model.X1x2xn.........Y1ym...Omega ijInput dataAdjustable connection weightsoutput dataMjxxynijiijnijiiji,..., 3,2,10,00111=.......<.Aged.=Sigma sigma ==Theta, Omega, theta, OmegaififThe input and output satisfy the following excitation functions:Key experiment of ground mechanical bionics technology of Ministry of education of Jilin UniversityLearning algorithm of 8.2.2 single layer perceptron modelRosenblatt proposed the learning algorithm of connection weight parameter in perceptron modelFirst, the join weights and thresholds are initialized tosmaller nonzero random numbers;Then input the input with n connection weights into the network;By weighted processing, if the output is different from the desired output, the connection weights are consideredThe number is adjusted automatically according to an algorithm;After many times of repetition, until the difference between the output and the desired output meets the requirements.Single layer perceptron specific learning algorithm (consider only one output):Let Xi (T) be the input of the time t perceptron (i=1,2,... (n), Omega I (T) is the corresponding connection weight, and Y (T) is practicalThe output of D (T) is the desired output, and the output of sensor or 1, or 1, is the single layer perceptronThe learning algorithm is:(1) initialize join weights and thresholds. Give Omega I (T) (i=1,2),... (n) and theta give a smaller non-zero sum, respectivelyRandom numbers are used as their initial values. Omega I (0) is the connection weight of the first input of the t=0 at I, and theta is the outputThreshold in a node.ElevenKey experiment of ground mechanical bionics technology of Ministry of education of Jilin University(2) enter a training parameter X = (x1 (T), X2 (t),... Xn (T) and expected output D (t).(3) calculate the actual output of the networkY (T)=F(sigma (n) Omega I(T) Xi(T) thetaI=1,2,..., n)One(4) calculate the difference between actual output and expected outputDEL=d (T) -y (T)If DEL< epsilon (epsilon is a small positive number), network training ends; otherwise, step (5).(5) adjust the connection weight asOmega = Omega I (t+1) I (T) + [d (T) -y (T)]xi (t (i=1,2),... N).The 0< is less than or equal to 1, a gain factor is used to control the speed change, also known as the gain or learning speed.(6) return (2).The above algorithm shows that the learning of network connection weights is an iterative process, which is in the first (2) step (6) stepRepeat until the error between the actual output of the network and the desired output is met. At this point, incomeThe weighted value of the network is I (i=1,2,... (n) is thenetwork connection parameter that is learned by training data.TwelveKey experiment of ground mechanical bionics technology of Ministry of education of Jilin University8.2.3 multilayer perceptionimplementOne or more layers of processing units are added between the input and output layers of the single layer perceptron to form two layers or moreLayer perceptron.The new layer is called the hidden layer, and the processing unit in the hidden layer is called the hidden unit.The hidden unit acts as a specific detector, which extracts the effective feature information contained in the input pattern, which is processed by the output unitThe patterns are linearly separable.The architecture of a two - layer perceptron is shown in the figure................Y1ynx1x2xnAdjustable weightWeight fixationoutput layerhidden layerInput partIn this figure, the perceptron has two layers of connection, the input partThe connection weights between the hidden layer and the hidden layer units are fixed randomlyFixed value, not adjustable, only output layer and hidden layer unitThe connection weight between the two layers is adjustable.ThirteenKey experiment of ground mechanical bionics technology of Ministry of education of Jilin UniversityMultilayer perceptron can take many disadvantages of single layer perceptron, and some of the problems that can not be solved by some single layer perceptron,It can be solved in a multilayer perceptron.For example, the application of two layer perceptron can solve the XOR logic problem, as shown in the figure.YOutput layer, where the threshold of the output layer neuron is assumed to be 0.5, for each deityThe link weights between the elements are shown in the figureX11-1-111x21The hidden layer is x11=1 * x10+ (-1) * x20-0.5ElevenX21=1 * X20 (-1 * x10-0.5)Input unitbranchY=1 * x11+1 * x21-0.5X10X20The corresponding identification area is shown in the figure.Because each neuron in the hidden layer can have its own half of the plane that can be identifiedAnd the output layer neuron makes the logic of the output of the hidden layerAnd "arithmetic", so its output can be used to identify the hidden layer identifiedA convex polygon formed by the intersection of half and half planes.A2B2A1B1FourteenKey experiment of ground mechanical bionics technology of Ministry of education of Jilin University8.3 back propagation model and its learning algorithmmethod8.3.1 back propagation model and its network structureThe back-propagation model, also known as the B-P(Back-Propagation) model, is used in forward multilayer neural networksThe back-propagation learning algorithm consists of Rumelhart (D.Rumelhart) and McLelland (MeClelland)Proposed 1985.The network structure of B-P algorithm is a forward multilayer network. The network contains not only input and output nodes, but also networkAnd contains one or more layers of hidden (layer) nodes, as shown.X1x2xn.........Y1ym... Y1When the information is input to the network, the information is transmitted from the input layer to the hidden layer first, and then the characteristic function is applied to it,And then spread to the next hidden layer. This layer is passed down until the output node layer is finally output.In the meantime, the excitation function of each layer is differentiable, and the S type function is generally used.Key experiment of ground mechanical bionics technology of Ministry of education of Jilin UniversityLearning algorithms for 8.3.2 backpropagation networksmethodThe purpose of the B-P algorithm is to adjust the connection weights of the network, so that the network is adjusted to any inputCapable of getting the desired output.The learning process consists of forward propagation and reverse propagation. Forward propagation is used to calculate the forward network, that is, to an input letterAfter the network calculation, the output is obtained; the back propagation is used to transfer errors by layer by layer, and the connection rights between neurons are modifiedThe output of the network can reach the desired error requirement after the input information is calculated.Sigma.Kkkyye2')Twenty-oneActivation function. Is the node, function jxfIfOOWIjjiiij here)= = (sigma)Let Oi be the output of the node i in the network, Ij is the input of the node j, and Wij is the connection weight from node i to node j,YK and YK, respectively, the actual output and expected output of the node K on the B-P network output layer.For node j, if the node I is its upper node and the I is connected to j, thenActual output and expected outputThe error E is defined as:Correction formula of connection weight:....'''=......Delta sigma = mkmmkkkkjkjkkkjkkWIfkIfyyOWIIeW When the hidden layer node isWhen the output layer node isDelta, beta, beta, beta)(())KAmong them, beta is the proportionality factor.(1)(2)(3)(4)Key experiment of ground mechanical bionics technology of Ministry of education of Jilin UniversityNYstartSelect a set of training examples,Each sample consists of two parts: the input information and the desired outputSelect a sample from the training sample set and enter the input information into the networkForward propagation: calculates the output of the neurons in each hidden layer and output layer of the networkEndThe delta values of each node in the output layer are calculated in the first half of the formula (3),The connection weights of the neurons in the hidden layer are adjusted according to the formulaBack propagation: each hidden layer is calculated layer by layer in the second half of formula (3)The delta value of the upper neuron, and the connection weight of each neuron is adjusted according to the formula (2)NDoes the error E meet the requirements?Given input vectors and output vectorsAccording to formula (2) the actual output and expected output error eSets the connection weights for the training example and initial values for the thresholdAll of the training examples are centralizedHave you finished the sample yet?Flow chart of YB-P learning algorithmNYstartSelect a set of training examples, each of which consists of two parts, the input information and the desired outputSelect a sample from the training sample set and enter the input information into the networkForward propagation: calculates the output of the neurons in each hidden layer and output layer of the networkEndThe delta values of each node in the output layer are calculated in the first half of the formula (3),The connection weights of the neurons in the hidden layer are adjusted according to the formulaBack propagation: each hidden layer is calculated layer by layer in the second half of formula (3)The delta value of the upper neuron, and the connection weight of each neuron is adjusted according to the formula (2)NDoes the error E meet the requirements?Given input vectors and output vectorsAccording to formula (2) the actual output and expected output error eSets the connection weights for the training example and initial values for the thresholdAll of the training examples are centralizedHave you finished the sample yet?Flow chart of YB-P learning algorithmKey experiment of ground mechanical bionics technology of Ministry of education of Jilin UniversityThe B-P algorithm has many advantages, such as solid theoretical foundation, rigorous derivation process, clear physical concept and good generalityIt is a better algorithm for training forward multilayer networks.B-P algorithm also has some disadvantages, mainly in:(1) the convergence speed of the learning algorithm is slow, and it usually takes thousands of iterations, and with the training sample dimensionWith the increase in number, network performance will become worse.(2) the number of nodes in the network selection there is no theoretical guidance.(3) from a mathematical point of view, the B-P algorithm is a gradient steepest descent method, which may cause local minimaQuestions. When there is a local pole hour, the error meets the requirement on the surface, but the solution is not necessarilyIs the real solution to the problem. Therefore, the B-P algorithm is incomplete.Example of 8.3.3 reverse propagation calculationThe most important step in the B-P learning algorithm is to compute the connection between neurons in the network before training to the multilayer networkThe modified weight quantity Wjk. In order to calculate the delta Wjk, calculated in the backward delta is very important.The following example illustrates the calculation of delta in reverse propagation.EighteenKey experiment of ground mechanical bionics technology of Ministry of education of Jilin UniversityExample: here is a simple forward propagation network that uses the B-P algorithm to determine the weights of each connectionThe calculation method is as followsOneFourX1x22W13W233W34W355y1y2As you can see from the diagram:The reverse propagation is calculated as follows: I3=W13x1+W23x2O3=F(I3)(1) calculation;EI=WOO=Y=F(I)FourThree hundred and forty-three Forty-oneFour .W.E.E.I3 .e I= WO O=Y=F(I)=.D = xFiveThree hundred and fifty-three Fifty-twoFiveOne hundred and thirty-one.W...IThirteenThree hundred and thirteen ThreeTwelveTwoE=([Y1'.Y1)+(Y2'.Y2) ].E.E..e Two =.D = x TwoThirty-two.W.I..ITwenty-threeThree hundred and twenty-three (2) calculation;Delta4.=..IeFour =(Y1 .Y1 ')F'(I4) .E.E.I4 .=D = OThreeForty-three.W.I..I.EThirty-fourFour hundred and thirty-four FourDelta ==(Y.Y ')F'(I)FiveTwo hundred and twenty-five .I5.EE.I5.=.D = O ThreeFifty-three Delta = (delta W Delta + W )F'(I).W.I..IThree million four hundred and thirty-four thousand five hundred and thirty-fiveThreeThirty-fiveFive hundred and thirty-fiveKey experiment of ground mechanical bionics technology of Ministry of education of Jilin University8.4 Hopfield model and its learning algorithmThe feedforward network described earlier lacks dynamic processing capability, and hence its computational power is not strong enough.。