当前位置：文档之家 › 深度学习2014-Jiang,Crookes-Deep Salience Visual Salience Modeling via Deep Belief Propagation

深度学习2014-Jiang,Crookes-Deep Salience Visual Salience Modeling via Deep Belief Propagation

  • 格式：pdf
  • 大小：1.67 MB
  • 文档页数：7

Deep Salience: Visual Salience Modeling via Deep Belief PropagationRichard Jiang Automatic Control and System Engineering The University of SheffieldSheffield, United KingdomE-mail: richardjiang@Danny CrookesECIT Institute Queen’s University Belfast Belfast, United Kingdom E-mail: d.crookes@AbstractVisual salience is an intriguing phenomenon observed in biological neural systems. Numerous attempts have been made to model visual salience mathematically using various feature contrasts, either locally or globally. However, these algorithmic models tend to ignore the problem’s biological solutions, in which visual salience appears to arise during the propagation of visual stimuli along the visual cortex. In this paper, inspired by the conjecture that salience arises from deep propagation along the visual cortex, we present a Deep Salience model where a multi-layer model based on successive Markov random fields (sMRF) is proposed to analyze the input image successively through its deep belief propagation. As a result, the foreground object can be automatically separated from the background in a fully unsupervised way. Experimental evaluation on the benchmark dataset validated that our Deep Salience model can consistently outperform eleven state-of-the-art salience models, yielding the higher rates in the precision-recall tests and attaining the best F-measure and mean-square error in the experiments.IntroductionAutomated detection of visual objects in images and videos is a subject of primary interest because of its wide application in image/video indexing, content-aware editing, medical image analysis, intelligent computer-human interface, robotic vision, and visual surveillance. Researchers in artificial intelligence and computer vision have successfully developed a number of methods for object detection, such as AdaBoost face detection (Viola and Jones 2001), SVM-based human detection (Vedalid et al2009; Dalal and Triggs 2005), and min-cut object segmentation (Rother et al 2004). These approaches usually depend on training on predefined datasets, or on user input such as scribbles or trimaps. However, when no prior knowledge of image content is available, unsupervised object detection is a hard problem, and it has attracted considerable interest from the research community.The past decade has seen consistent progress towards unsupervised image segmentation and object detection. A widely-adopted approach is to consider an image principally as a set of hierarchical contours (Arbelaez et al 2011). This assumes that semantic content and the objects usually correspond to specific image structures. Recent research (Farabet et al 2013; Kohli et al 2013) has also suggested that this hierarchical view of image content may be correlated to the deep learning of image structures (Hinton et al 2006). In a development of this approach, we propose in this paper a Deep Salience model based on successive Markov Random Fields (sMRF) for unsupervised object detection. Our work is conceptually related to the recent pioneering work on hierarchical image analysis (Farabet et al 2013; Kohli et al 2013). Unsupervised object detection usually leads to the topic of visual salience, which stems from psychological research on biological visual perception (Koch and Ullman 1985; Itti and Koch 2001). The earliest bio-inspired computational salience model was proposed by Koch and Ullman (1985), where the contrast between visual stimuli (pixels) was considered as the origin of salience awareness. A number of publications (Itti et al 1998; Ma and Zhang 2003; Harel et al 2006; Hou and Zhang 2007; Judd 2009) have followed this roadmap to develop their salience models using a variety of features. These methods are usually based on local contrast and tend to produce higher salience values near edges instead of uniformly highlighting salient objects. Cheng et al(2011) categorized these approaches as local approaches. Recent efforts have been made towards using global contrasts, where pixels or regions were evaluated with respect to the entire image. Achanta et al(2009) proposed a frequency tuned method that defines pixel salience using region-averaged contrast. Goferman et al (2012) used block-based global contrast. Cheng et al (2011) extend Achanta’s method to region-based salience estimation. Perazzi et al (2012) further extend this region-contrast approach by leveraging superpixels. Jiang and Crookes (2012) used mutual information (MI) evaluation with a center-surround a priori map for global salience estimation. Peng et al(2013) introduced the low-rank matrix computation for salience modeling. In summary, both local and global methods have been based on modeling salience using various visual contrast definitions with various features based on pixels, blocks or regions. From a biological viewpoint, we consider the conjecture that human visual salience is a consequence of the deepCopyright©2014,Association for the Advancement of ArtificialIntelligence().All rights reserved.Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligencepropagation of visual stimuli along the human visual cortex (Ekstorm et al 2008). Hence, it is a complex procedure rather than a simple visual comparison using color contrast. To emulate this biological process, we present a Deep Salience model using our proposed successive Markov Random Fields (sMRF) model.In our sMRF model, a fuzzy graph is introduced with a Markov Random Field to allow each node to maximize its belief by solving the fuzziness of edges. When applied to a cascaded multi-scale image pyramid, the pyramid structure of multiple-layer MRFs forms successive MRFs (sMRFs) which processes the input visual signals via its deep belief propagation along layers, while a belief maximization procedure is applied to detect salient objects automatically from its iterative convergence. Fig.1 shows an example of applying our Deep Salience method to a challenging image.Successive Image AnalysisIn image structure analysis, hierarchical analysis has been effective in tackling various computer vision tasks. Lowe (2004) proposed the use of a pyramid computation of DoG for multi-scale feature point detection. Sun and Pfister (2013) applied a pyramid-based coarse-to-fine approach to optical flow and achieved improved accuracy. In these methods, images were processed at different scales and the computation was carried out hierarchically. In the recent research (Arbelaez et al 2011; Farabet et al 2013; Kohli et al 2013), images have been considered as a collection of hierarchical contours. While hierarchical methods have been seen very successful in these challenging vision tasks, in this paper we apply this strategy to our visual salience analysis for unsupervised object detection.In our method, the first step is to compute a pyramid of hierarchical images. Usually this can be done by using discrete wavelet transform (DWT). Given an input image, the 2D DWT computation will result in four components, namely LL, LH, HL and HH, as shown in Fig.2-a. In our approach, we keep only the LL (low-pass) component of each level. Consequently, we obtain a hierarchical pyramid of resized images at different resolutions, as shown in Fig.2-b. Our Deep Salience method will then take these images as input, and perform hierarchical analysis successively from the top of the pyramid downwards. The proposed coarse-to-fine process is akin to an emulation of the human visual system where the input visual signals are propagated along the visual cortex from eyes to brain (Koch and Ullman 1985; Ekstorm et al 2008).Maximum Belief Propagation in MRFMarkov random fields have been a powerful tool for image analysis. Given an undirected graph G = (V , E ) and observations X={x k } with k ∈V indexed by V , a set of random variables Y ={y k } forms a conditional random fieldwith respect to G when they satisfy the local Markov properties that a variable is conditionally independent of anyvariablesthat are not its neighbors. Hence, global belief propagation over a CRF can then be simplified as local belief propagation over cliques.Our work starts from the pair-wise MRF model proposed by Jegelka and Bilmes (2011). Let V be the set of pixels in the image X we want to process, and let the set E contain all pairs of neighboring pixels that are edges in the MRF. Thea) An imageb) Our DS resultc) Ground truthd) Salience maps from several state-of-the-art methods Fig.1. A simple example that challenges most state-of-the-art methods. a) The image; b) Our Deep Salience(DS) map; c) Ground truth; d) Results by several state-of-the-art salience methods, SR (Hou & Zhang 2007), IT (Itti et al in 1998), GB (Harel et al 2006), MZ (Ma & Zhang 2003), LC (Zhai & Shah 2006), FT (Achanta et al 2009), CA (Goferman et al 2010), and RC (Cheng et al 2011). Here the ‘jet’ color map is applied to visualize the results.a) Successive DWT b) Hierarchical pyramid Fig.2. Hierarchical image analysis with wavelet pyramid.Fig.3 The proposed successive Markov random fields allow multi-layer deep belief propagation downwards along directed edges.label of each pixel in V is given by a binary random variable y i whose labels (0/1) denote the label “foreground”/“background”.Considering a clique of two vertices i and j , if the confidence on the i -th vertex is known as P i , the belief propagation along the clique edge can be inferred as a MAP (maximum a posteriori) formula:ijj i P P (1) ij denotes the conservative probability from the j th vertexto the i th vertex, which can be simply formulated with thefeature difference between the i th and j th pixels:j i ij x x (2)Here, ψ(·) denotes a penalty function when the belief is propagated across two color-different pixels. For example, given a pixel k 0 with a confidence P 0, a neighbor k 1 with exactly the same color should have a confidence P 1=P 0 with ij =0.A node in aMRFmay have multiple edges. Typical MRF methods using mean field continuum approximation (Yedidia et al 2005) may easily bleach the edges in an image and allow the belief propagation to cross region borders. To address this problem, we introduce a maximum belief propagation algorithm. We consider all edges of a node as a potential fuzzy edge in a fuzzy graph (Kosko 1986; Blue and Bush 2002; Salzenstein and Collet 2006) E V G ~,~ . Hence, the pixel may have multiple choices of likelihood P i j inferred from different edges, and the fuzziness can be described by:Cj ij j ij Cj j i ij P f P f Fi (3)Here, f ij stands for the fuzziness of the j-th fuzzy edge. Tomaximize the belief in propagation, we apply a winner-takes-all strategy in the local iteration over random fields,otherwise ,0max arg when ,1ji j ij P j f (4)In the above winner-takes-all strategy, a node always favors the edge with the maximum confidence and dynamically sets other fuzzy edges as disconnected.A major advantage of the above maximum belief propagation (MBP) algorithm is to guarantee that belief propagation in MRF will not blur the region borders in an input image. Besides, by disconnecting some edges dynamically, MBP inference problem is simplified into a local search of the winner in the fuzzy set of neighbors. In an analogy, the above MBP is more like a quantum mechanism (Birkhoff and Neumann 1936) that allows each node to switch between multiple quantum energy levels, while each fuzzy edge stands for a specific quantum energy level. For our convenience, we refer to our above simple implementation of MBP random fields as Quantum Random Fields (QRF) in this paper. Fig.5 shows the different belief propagation results in our Deep Salience model using typical MRF and our QRF. It is shown that QRF can easily converge on the salient object quickly.Deep Belief Propagation via Successive MRFSuccessive Markov Random FieldIn research in unsupervised image segmentation and object detection, a common approach is to consider an image as a set of hierarchical contours (Arbelaez et al 2011). Recenta) Sample imageb) Level 4c) Level 3d) Level 2 e) Level 1 f) Level 0 Fig.4. Deep Salience: Successive salience analysis via deep belief propagation over sMRF.Images QRF (0.1sec) MRF (3.5sec) Fig.5. DS modeling with sMRF using our QRF or standard MRF at each layer. QRF-based DS takes only 0.1s and highlights the foreground object, while typical MRF based DS needs longer time (3.5s) and blurs the region borders.Images One-layer MRF sMRFFig.6. sMRF vs one-layer MRF: sMRF helps overcome the texture problem by its deep belief propagation.research (Farabet et al 2013; Kohli et al 2013) has also suggested that this hierarchical view of image content may be correlated to the deep learning of image structures.As mentioned above, a pyramid of hierarchical images can be obtained from an input image using DWT, as shown in Fig.2. Each image in the pyramid can be mapped as an observation for each layer of successive MRF (sMRF). Fig.3 shows the concept of sMRF, where fuzzy edges not only present in one layer of random fields, but also connect a layer to the previous and next layers. Hence, the belief propagation is carried out down the pyramid of random fields. For a node in our successive QRF (sQRF), its inter-layer edges can be added into the search of its maximum belief,},{max arg ,i j i j m P P j (5)Herei P is the estimated belief propagated from the nodes in other layers. In this work, we assume that the inter-layer connection is unidirectional so that the belief can only propagate from the upper layer to its lower layer. This implies an inheritance of the confidence map hierarchically down the pyramid from coarse to fine.Starting from the top level of sMRF, we carry out the maximum belief propagation process (in Eq.(5)) iteratively over each layer of our quantum random field to solve its fuzziness. The computation on the top image (the one resized to 1/2k ) gives the result as the k th level belief map G k . Then the belief map is propagated to the next layer down the pyramid.Fig.4 shows an example of sMRF. Fig.4-a shows the sample image where, similar to Judd’s work (2009), we assume that most pixels on the borders are taken as background (P B =1). With this priori, the belief is thenpropagated based on the coarse observation from the top level, and successively downward to the lower sMRF layers. It can be seen that the resolution of the confidence maps increases and the details are gradually added via deep belief propagation. In our experiment, we use SF method (Perazzi et al 2012) to initialize the border and keep non-salient pixels as initial background.The benefits of sMRF over single-layer MRF stem from its successive analysis of hierarchical image structures. It focuses on global structures at its initial coarse level and gradually focuses on local details in the subsequent levels. Therefore, it can allow belief propagation to go across texture regions easily. Fig.6 shows several examples comparing sMRF against single-layer MRF. We can see that the belief propagation in single-layer MRF gets stuck in texture-like regions in the background, while sMRF can overcome these local textures and highlight the salient foreground objects successfully in these test images.Computing ComplexityUsually, the computing complexity of MRF is not readily predictable. However in our sQRF, the problem can become much easier. While the belief propagation is simplified as a local maximum problem in Eq.(5), the computing time is proportional to the range for a belief to propagate from the source to the pixel, which is related to the number of pixels . As shown in Fig.7, our experiment validated that the convergence time of sQRF is nearly proportional to the number of pixels, namely O(N ).It is noted that sMRF needs to converge on multiple layers. However, the number of nodes has been reduced to 1/2k , and the search space at the top level of the pyramid is then drastically reduced to (1/2k )2 as well. While the sMRF iteration moves downwards, the pre-converged confidence map is inherited by the next level and hence it can be expected to help reduce the total convergence time. Table I gives the experimental results of the computation times at different layers. The computing time was measured for a MATLAB solution. We can see that the single-level MRF took the longest time to converge (210 ms), while 5-level sMRF needs only 23 ms in total for all five levels. From this comparison we can see that successive MRF not only helps overcome texture background, but also helps reduce the computing time.ExperimentsWe tested our method on the widely used object dataset ---the EPFL object dataset (Achanta et al 2009), which is publicly available and the ground truth of foreground objects is provided as binary masks. Most state-of-the-art methods have been reported with their benchmark results on this dataset. The EPFL dataset has the same images as MSRA-1000 except that it takes segmented objects as ground truth instead of gaze points from psychologicalFig.7. Computing time versus the number of pixels N. Table I. Convergence time vs the number of layers (ms)Layers 1 Layer 2 Layer 3 Layer 4 Layer 5 Layer 1st 210 103 78 61 12 2nd -- 29 16 14 4 3rd -- -- 6 3 3 4th -- -- -- 2 2 5th -- -- -- -- 2 Total 210 132 100 80 23measurement. Hence, the dataset is well suited to our purpose of unsupervised object detection.Our algorithm was implemented in MATLAB. We ran our code on the dataset, and compared the results against 11 state-of-the-art unsupervised visual salience models. The methods for comparison include: SR (Hou & Zhang 2007), IT (Itti et al in 1998), GB (Harel et al 2006), MZ (Ma & Zhang 2003), LC (Zhai & Shah 2006), FT (Achantaet al 2009), CA (Goferman et al 2010), HC and RC (Cheng et al 2011), MI (Jiang & Crookes 2012) and SF (Perazzi et al 2012). The following are our experimental results.Visual ComparisonFig.8 provides a visual comparison of the various methods using a number of sample images. Most of the state-of-the-art methods failed to find the salient objects in these test images. In comparison, we can see that our Deep Salience method using sMRF can robustly highlight salient objects from their background.Fig.9 shows several more examples. By applying thresholds of 2*mean and 3*mean respectively to the salience maps, we have firstly identified object-like regions (shown as red rectangles) and then, even further, attempted to identify their salient parts (shown as green rectangles). The success of our sQRF-based object detection can be attributed to the use of hierarchical analysis that is able to capture the global spatial structures of an image. However, we note that we did not build any contrast-based salience model explicitly. Instead, object detection is a natural outcome of maximizing the belief iteratively over the sQRF.Statistic EvaluationFig.10-a shows the typical statistical results of precision-recall curves for the EPFL dataset. The curves are computed in the same way as reported by previous work. Here, precision corresponds to the percentage of salient pixels correctly assigned, while recall rate corresponds to the fraction of detected salient pixels in relation to the number of ground truth salient pixels. As shown in Fig.10-a, our method achieves the best overall precision and recall rates of all the compared methods, and consistently outperforms previous salience models in term of precision and recall rates. It is also observed that the DS curve goes up much steeper than other methods.Fig.8. Visual comparison of salience maps. From left to right columns: original images; ground truth; salience maps from 1) SR; 2) IT; 3) GB; 4) MZ; 5) LC; 6) FT; 7) CA; 8) HC; 9) RC; 10) SF; and our Deep Salience (DS) maps. It can be seenthat our DS model can robustly highlight salient objects in images where other algorithms failed.a) Input images b) DS maps c) Detected objects Fig.9. Unsupervised object detection with our Deep Salience model. Red rectangles denote the object-likeregions, and green ones highlight the most salient parts.Fig.10-b shows the results of F-measure , which combines both precision and recall to evaluate salience cut. Here, we use the adaptive threshold similar to that proposed by Achanta (2009), defined as twice the mean salience of the image. Appling this threshold to the salience map, wecan get the precision and recall of its binary cut, and then the F-measure can be subsequently computed by:Precision Recall Recall Precision F (6)From Fig.10-b, we can see that our Deep Salience achieves the best F-measure score than all the other methods in the comparison.Precision and recall measures do not consider true negative salience assignment, i.e., the number of pixel correctly marked as non-salient. Moreover, the quality of the weighted, continuous saliency maps may be of higher importance than the binary masks from psychological view of visual attention. To measure how successful a method is in the detection of non-salient background regions, we also carried out the comparison using mean square error (MSE ) between the continuous saliency map S (before thresholding) and the binary ground truth G . MSE measure is then defined as,x yy x G y x S H W MSE 2,,1 (7) where W and H are the width and the height of the respective saliency map and ground truth image.Figure 11 shows that our method also outperforms other approaches in terms of the MSE measure, which provides a better estimate of the dissimilarity between the saliency map and ground truth. Results have been averaged over all images in the test dataset.ConclusionIn conclusion, a new salience model has been successfully developed for unsupervised object detection via its deep belief propagation along the pyramid of successive Markov Random Fields (sMRF). The hierarchical structure of sMRF helps overcome the problem of background textures in belief propagation and also speeds up the convergence in multi-layer MRFs. It is also worth noting that our maximum belief propagation in QRF differs from typical mean-field approximation in that it employs a dynamic selection of fuzzy edges in its random fields, making it possible to guarantee that the belief propagation will not bleed over region borders in an image. Our benchmark experiments successfully validated that the proposed Deep Salience method consistently achieved better rates in precision-recall curves, and also attained better scores in both F-measure and MSE tests.This work was initially inspired by the observed fact that visual salience arises during the propagation of visual stimuli along human visual cortex. Unlike most previous methods that are based on algorithmic math computation of various global contrasts, our salience estimation is a natural outcome of visual signal propagation over sMRF, which is more like the way how our human vision system processes the input visual signals.a) Precision-recall testb) F-measure testFig.10. Evaluation on the EPFL database. a) Precision-recall test: DS model consistently achieved the best prevision/recall rates; b) F-measure test: DS attained the bestscoresin F-measure and recall, and the precision after thresholding was exceeded only by RC and SF.Fig.11. Mean square error (MSE) test.ReferencesAchanta, R.; Hemami, S.; Estrada, F.; Susstrunk, S. 2009. Frequency-tuned salient region detection. In CVPR’09, 1597–1604.Arbelaez, P.;; Maire, M.;; Fowlkes, C.;; Malik, J. 2011. Contour detection and hierarchical image segmentation. In IEEE TP AMI, 33(5): 898-916.Birkhoff, G.; Neumann, J. V. 1936. The Logic of Quantum Mechanics. Annals of Mathematics, 37: 823–843.Blue, M.;; Bush, B. 2002. Unified approach to fuzzy graph problems. Fuzzy Sets and Systems, 125(3):355–368.Cheng, M.; Zhang, G.; Mitra, N. J.; Huang, X.; Hu, S. 2011. Global Contrast based Salient Region Detection. In CVPR’11, 409-416.Dalal, N.; Triggs, B. 2005. Histograms of oriented gradients for human detection. In CVPR’05.Ekstrom, L. B.; Roelfsema, P. R.; Arsenault, J. T.; Bonmassar, G.; Vanduffel, W. 2008. Bottom-up dependent gating of frontal signals in early visual cortex. Science, 321(5887): 414-417. Farabet, C.;; Couprie, C.;; Najman L.;; LeCun, Y. 2013. Learning hierarchical features for scene labeling. In IEEE TP AMI, 35(8): 1915-1929.Goferman, S.;; Zelnik-Manor, L.;; Tal, A. 2010. Context-aware saliency detection. In CVPR’10.Harel, J.;; Koch, C.;; Perona, P. 2006. Graph-based visual saliency. In NIPS, 545–552.Hou, X.;; Zhang, L. 2007. Saliency detection: A spectral residual approach. In CVPR’07, 1–8.Hinton, G. E.;; Salakhutdinov, R. 2006. Reducing the dimensionality of data with neural networks. In Science, 313(5786): 504-507.Itti, L.; Koch, C. 2001. Computational modelling of visual attention. Nature Reviews Neuroscience, 2:194–203.Itti, L.;; Koch, C.;; Niebur, E. 1998. A model of saliency-based visual attention for rapid scene analysis. In IEEE TP AMI, 20(11):1254–1259.Jegelka, S.;; Bilmes, J. 2011. Submodularity beyond submodular energies: Coupling edges in graph cuts. In CVPR’11, 1897-1904. Jiang, R.;; Crookes, D. 2012. Visual saliency estimation through manifold learning. In AAAI’12.Judd, T. 2009. Learning to predict where humans look. In ICCV’09.Kohli, P.;; Osokin, A.;; Jegelka, S. 2013. A principled deep random field model for image segmentation. In CVPR’13.Koch, C.;; Ullman, S. 1985. Shifts in selective visual attention: towards the underlying neural circuitry. Human Neurobiology, 4:219–227.Kosko, B. 1986. Fuzzy cognitive maps. International Journal of Man-Machine Studies, 24(1):65-75.Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. In IJCV, 60(2): 91-110.Ma, Y. F.;; Zhang, H. J. 2003. Contrast-based image attention analysis by using fuzzy growing. In ACM Multimedia’03, 374-381.Perazzi, F.;; Krähenbül, P.;; Pritch, Y.;; Hornung, A. 2012. Saliency Filters: Contrast Based Filtering for Salient Region Detection. In CVPR’12.Peng, H.;; Li, B.;; Ji, R.;; Hu, W. 2013. Salient object detection via low-rank and structured sparse matrix decomposition. In AAAI-13. Rother, C.;; Kolmogorov, V.;; Blake, A. 2004. Grabcut–Interactive foreground extraction using iterated graph cuts. In ACM Trans. Graph., 23(3): 309-314.Salzenstein, F.;; Collet, C. 2006. Fuzzy Markov Random Fields versus Chains for Multispectral Image Segmentation. IEEE TP AMI, 28(11): 1753 - 1767.Sun, D.;; Wulff, J.;; Sudderth, E. B.;; Pfister, H.;; Black, M. J. 2013.A fully-connected layered model of foreground and background flow. In CVPR'13, 2451-2458.Viola, P.;; Jones, M. 2001. Robust real-time object detection. In ICCV’01.Vedaldi, A.;; Gulshan, V.;; Varma, M.;; Zisserman, A. 2009. Multiple kernels for object detection. In ICCV’09.Yedidia, J. S.;; Freeman, W. T.;; Weiss, Y. 2005. Constructing Free Energy Approximations and Generalized Belief Propagation Algorithms. IEEE Trans. Information Theory, 51:2282-2312. Zhai, Y.;; Shah, M. 2006. Visual attention detection in video sequences using spatiotemporal cues. In ACM Multimedia’06, 815-824.。

合集下载

《2024年深度学习在计算机视觉领域的若干关键技术研究》范文

《2024年深度学习在计算机视觉领域的若干关键技术研究》范文

《深度学习在计算机视觉领域的若干关键技术研究》篇一一、引言随着科技的飞速发展，深度学习在计算机视觉领域的应用越来越广泛。

计算机视觉，作为人工智能的重要分支，已经深入到我们生活的方方面面，如人脸识别、自动驾驶、医疗影像分析等。

本文将就深度学习在计算机视觉领域的若干关键技术进行深入探讨。

二、深度学习基础深度学习是机器学习的一个分支，其核心在于神经网络。

神经网络通过模拟人脑神经元的结构和工作方式，可以学习和理解复杂的模式。

在计算机视觉领域，深度学习主要通过卷积神经网络（CNN）实现图像的识别和分类。

三、关键技术研究1. 卷积神经网络（CNN）卷积神经网络是深度学习在计算机视觉领域的重要应用。

它通过卷积操作提取图像的局部特征，然后通过全连接层进行分类或回归。

CNN在图像分类、目标检测、语义分割等领域有着广泛的应用。

近年来，随着深度学习的不断发展，CNN的模型结构也在不断优化，如残差网络（ResNet）、轻量级网络（MobileNet）等。

2. 生成对抗网络（GAN）生成对抗网络是一种无监督学习的深度学习模型，由生成器和判别器组成。

GAN在计算机视觉领域的应用主要包括图像生成、图像修复、超分辨率等。

通过生成器和判别器的对抗训练，GAN 可以生成高质量的图像，为计算机视觉领域带来了新的可能性。

3. 迁移学习与微调迁移学习是利用在大数据集上预训练的模型来辅助小样本数据的训练。

在计算机视觉领域，可以利用在大规模图像数据集上预训练的模型（如ResNet），然后通过微调（fine-tuning）来适应特定的任务。

迁移学习和微调可以有效提高模型的性能，同时降低模型的训练成本。

四、应用领域分析1. 人脸识别深度学习在人脸识别领域的应用已经非常成熟。

通过卷积神经网络提取人脸特征，然后利用分类器进行人脸识别。

人脸识别技术在安防、金融、支付等领域有着广泛的应用。

2. 自动驾驶自动驾驶是计算机视觉的另一重要应用领域。

通过深度学习算法分析道路情况、行人、车辆等信息，然后对汽车进行决策和控制。

《神经网络与深度学习综述DeepLearning15May2014

《神经网络与深度学习综述DeepLearning15May2014

Draft:Deep Learning in Neural Networks:An OverviewTechnical Report IDSIA-03-14/arXiv:1404.7828(v1.5)[cs.NE]J¨u rgen SchmidhuberThe Swiss AI Lab IDSIAIstituto Dalle Molle di Studi sull’Intelligenza ArtificialeUniversity of Lugano&SUPSIGalleria2,6928Manno-LuganoSwitzerland15May2014AbstractIn recent years,deep artificial neural networks(including recurrent ones)have won numerous con-tests in pattern recognition and machine learning.This historical survey compactly summarises relevantwork,much of it from the previous millennium.Shallow and deep learners are distinguished by thedepth of their credit assignment paths,which are chains of possibly learnable,causal links between ac-tions and effects.I review deep supervised learning(also recapitulating the history of backpropagation),unsupervised learning,reinforcement learning&evolutionary computation,and indirect search for shortprograms encoding deep and large networks.PDF of earlier draft(v1):http://www.idsia.ch/∼juergen/DeepLearning30April2014.pdfLATEX source:http://www.idsia.ch/∼juergen/DeepLearning30April2014.texComplete BIBTEXfile:http://www.idsia.ch/∼juergen/bib.bibPrefaceThis is the draft of an invited Deep Learning(DL)overview.One of its goals is to assign credit to those who contributed to the present state of the art.I acknowledge the limitations of attempting to achieve this goal.The DL research community itself may be viewed as a continually evolving,deep network of scientists who have influenced each other in complex ways.Starting from recent DL results,I tried to trace back the origins of relevant ideas through the past half century and beyond,sometimes using“local search”to follow citations of citations backwards in time.Since not all DL publications properly acknowledge earlier relevant work,additional global search strategies were employed,aided by consulting numerous neural network experts.As a result,the present draft mostly consists of references(about800entries so far).Nevertheless,through an expert selection bias I may have missed important work.A related bias was surely introduced by my special familiarity with the work of my own DL research group in the past quarter-century.For these reasons,the present draft should be viewed as merely a snapshot of an ongoing credit assignment process.To help improve it,please do not hesitate to send corrections and suggestions to juergen@idsia.ch.Contents1Introduction to Deep Learning(DL)in Neural Networks(NNs)3 2Event-Oriented Notation for Activation Spreading in FNNs/RNNs3 3Depth of Credit Assignment Paths(CAPs)and of Problems4 4Recurring Themes of Deep Learning54.1Dynamic Programming(DP)for DL (5)4.2Unsupervised Learning(UL)Facilitating Supervised Learning(SL)and RL (6)4.3Occam’s Razor:Compression and Minimum Description Length(MDL) (6)4.4Learning Hierarchical Representations Through Deep SL,UL,RL (6)4.5Fast Graphics Processing Units(GPUs)for DL in NNs (6)5Supervised NNs,Some Helped by Unsupervised NNs75.11940s and Earlier (7)5.2Around1960:More Neurobiological Inspiration for DL (7)5.31965:Deep Networks Based on the Group Method of Data Handling(GMDH) (8)5.41979:Convolution+Weight Replication+Winner-Take-All(WTA) (8)5.51960-1981and Beyond:Development of Backpropagation(BP)for NNs (8)5.5.1BP for Weight-Sharing Feedforward NNs(FNNs)and Recurrent NNs(RNNs)..95.6Late1980s-2000:Numerous Improvements of NNs (9)5.6.1Ideas for Dealing with Long Time Lags and Deep CAPs (10)5.6.2Better BP Through Advanced Gradient Descent (10)5.6.3Discovering Low-Complexity,Problem-Solving NNs (11)5.6.4Potential Benefits of UL for SL (11)5.71987:UL Through Autoencoder(AE)Hierarchies (12)5.81989:BP for Convolutional NNs(CNNs) (13)5.91991:Fundamental Deep Learning Problem of Gradient Descent (13)5.101991:UL-Based History Compression Through a Deep Hierarchy of RNNs (14)5.111992:Max-Pooling(MP):Towards MPCNNs (14)5.121994:Contest-Winning Not So Deep NNs (15)5.131995:Supervised Recurrent Very Deep Learner(LSTM RNN) (15)5.142003:More Contest-Winning/Record-Setting,Often Not So Deep NNs (16)5.152006/7:Deep Belief Networks(DBNs)&AE Stacks Fine-Tuned by BP (17)5.162006/7:Improved CNNs/GPU-CNNs/BP-Trained MPCNNs (17)5.172009:First Official Competitions Won by RNNs,and with MPCNNs (18)5.182010:Plain Backprop(+Distortions)on GPU Yields Excellent Results (18)5.192011:MPCNNs on GPU Achieve Superhuman Vision Performance (18)5.202011:Hessian-Free Optimization for RNNs (19)5.212012:First Contests Won on ImageNet&Object Detection&Segmentation (19)5.222013-:More Contests and Benchmark Records (20)5.22.1Currently Successful Supervised Techniques:LSTM RNNs/GPU-MPCNNs (21)5.23Recent Tricks for Improving SL Deep NNs(Compare Sec.5.6.2,5.6.3) (21)5.24Consequences for Neuroscience (22)5.25DL with Spiking Neurons? (22)6DL in FNNs and RNNs for Reinforcement Learning(RL)236.1RL Through NN World Models Yields RNNs With Deep CAPs (23)6.2Deep FNNs for Traditional RL and Markov Decision Processes(MDPs) (24)6.3Deep RL RNNs for Partially Observable MDPs(POMDPs) (24)6.4RL Facilitated by Deep UL in FNNs and RNNs (25)6.5Deep Hierarchical RL(HRL)and Subgoal Learning with FNNs and RNNs (25)6.6Deep RL by Direct NN Search/Policy Gradients/Evolution (25)6.7Deep RL by Indirect Policy Search/Compressed NN Search (26)6.8Universal RL (27)7Conclusion271Introduction to Deep Learning(DL)in Neural Networks(NNs) Which modifiable components of a learning system are responsible for its success or failure?What changes to them improve performance?This has been called the fundamental credit assignment problem(Minsky, 1963).There are general credit assignment methods for universal problem solvers that are time-optimal in various theoretical senses(Sec.6.8).The present survey,however,will focus on the narrower,but now commercially important,subfield of Deep Learning(DL)in Artificial Neural Networks(NNs).We are interested in accurate credit assignment across possibly many,often nonlinear,computational stages of NNs.Shallow NN-like models have been around for many decades if not centuries(Sec.5.1).Models with several successive nonlinear layers of neurons date back at least to the1960s(Sec.5.3)and1970s(Sec.5.5). An efficient gradient descent method for teacher-based Supervised Learning(SL)in discrete,differentiable networks of arbitrary depth called backpropagation(BP)was developed in the1960s and1970s,and ap-plied to NNs in1981(Sec.5.5).BP-based training of deep NNs with many layers,however,had been found to be difficult in practice by the late1980s(Sec.5.6),and had become an explicit research subject by the early1990s(Sec.5.9).DL became practically feasible to some extent through the help of Unsupervised Learning(UL)(e.g.,Sec.5.10,5.15).The1990s and2000s also saw many improvements of purely super-vised DL(Sec.5).In the new millennium,deep NNs havefinally attracted wide-spread attention,mainly by outperforming alternative machine learning methods such as kernel machines(Vapnik,1995;Sch¨o lkopf et al.,1998)in numerous important applications.In fact,supervised deep NNs have won numerous of-ficial international pattern recognition competitions(e.g.,Sec.5.17,5.19,5.21,5.22),achieving thefirst superhuman visual pattern recognition results in limited domains(Sec.5.19).Deep NNs also have become relevant for the more generalfield of Reinforcement Learning(RL)where there is no supervising teacher (Sec.6).Both feedforward(acyclic)NNs(FNNs)and recurrent(cyclic)NNs(RNNs)have won contests(Sec.5.12,5.14,5.17,5.19,5.21,5.22).In a sense,RNNs are the deepest of all NNs(Sec.3)—they are general computers more powerful than FNNs,and can in principle create and process memories of ar-bitrary sequences of input patterns(e.g.,Siegelmann and Sontag,1991;Schmidhuber,1990a).Unlike traditional methods for automatic sequential program synthesis(e.g.,Waldinger and Lee,1969;Balzer, 1985;Soloway,1986;Deville and Lau,1994),RNNs can learn programs that mix sequential and parallel information processing in a natural and efficient way,exploiting the massive parallelism viewed as crucial for sustaining the rapid decline of computation cost observed over the past75years.The rest of this paper is structured as follows.Sec.2introduces a compact,event-oriented notation that is simple yet general enough to accommodate both FNNs and RNNs.Sec.3introduces the concept of Credit Assignment Paths(CAPs)to measure whether learning in a given NN application is of the deep or shallow type.Sec.4lists recurring themes of DL in SL,UL,and RL.Sec.5focuses on SL and UL,and on how UL can facilitate SL,although pure SL has become dominant in recent competitions(Sec.5.17-5.22). Sec.5is arranged in a historical timeline format with subsections on important inspirations and technical contributions.Sec.6on deep RL discusses traditional Dynamic Programming(DP)-based RL combined with gradient-based search techniques for SL or UL in deep NNs,as well as general methods for direct and indirect search in the weight space of deep FNNs and RNNs,including successful policy gradient and evolutionary methods.2Event-Oriented Notation for Activation Spreading in FNNs/RNNs Throughout this paper,let i,j,k,t,p,q,r denote positive integer variables assuming ranges implicit in the given contexts.Let n,m,T denote positive integer constants.An NN’s topology may change over time(e.g.,Fahlman,1991;Ring,1991;Weng et al.,1992;Fritzke, 1994).At any given moment,it can be described as afinite subset of units(or nodes or neurons)N= {u1,u2,...,}and afinite set H⊆N×N of directed edges or connections between nodes.FNNs are acyclic graphs,RNNs cyclic.Thefirst(input)layer is the set of input units,a subset of N.In FNNs,the k-th layer(k>1)is the set of all nodes u∈N such that there is an edge path of length k−1(but no longer path)between some input unit and u.There may be shortcut connections between distant layers.The NN’s behavior or program is determined by a set of real-valued,possibly modifiable,parameters or weights w i(i=1,...,n).We now focus on a singlefinite episode or epoch of information processing and activation spreading,without learning through weight changes.The following slightly unconventional notation is designed to compactly describe what is happening during the runtime of the system.During an episode,there is a partially causal sequence x t(t=1,...,T)of real values that I call events.Each x t is either an input set by the environment,or the activation of a unit that may directly depend on other x k(k<t)through a current NN topology-dependent set in t of indices k representing incoming causal connections or links.Let the function v encode topology information and map such event index pairs(k,t)to weight indices.For example,in the non-input case we may have x t=f t(net t)with real-valued net t= k∈in t x k w v(k,t)(additive case)or net t= k∈in t x k w v(k,t)(multiplicative case), where f t is a typically nonlinear real-valued activation function such as tanh.In many recent competition-winning NNs(Sec.5.19,5.21,5.22)there also are events of the type x t=max k∈int (x k);some networktypes may also use complex polynomial activation functions(Sec.5.3).x t may directly affect certain x k(k>t)through outgoing connections or links represented through a current set out t of indices k with t∈in k.Some non-input events are called output events.Note that many of the x t may refer to different,time-varying activations of the same unit in sequence-processing RNNs(e.g.,Williams,1989,“unfolding in time”),or also in FNNs sequentially exposed to time-varying input patterns of a large training set encoded as input events.During an episode,the same weight may get reused over and over again in topology-dependent ways,e.g.,in RNNs,or in convolutional NNs(Sec.5.4,5.8).I call this weight sharing across space and/or time.Weight sharing may greatly reduce the NN’s descriptive complexity,which is the number of bits of information required to describe the NN (Sec.4.3).In Supervised Learning(SL),certain NN output events x t may be associated with teacher-given,real-valued labels or targets d t yielding errors e t,e.g.,e t=1/2(x t−d t)2.A typical goal of supervised NN training is tofind weights that yield episodes with small total error E,the sum of all such e t.The hope is that the NN will generalize well in later episodes,causing only small errors on previously unseen sequences of input events.Many alternative error functions for SL and UL are possible.SL assumes that input events are independent of earlier output events(which may affect the environ-ment through actions causing subsequent perceptions).This assumption does not hold in the broaderfields of Sequential Decision Making and Reinforcement Learning(RL)(Kaelbling et al.,1996;Sutton and Barto, 1998;Hutter,2005)(Sec.6).In RL,some of the input events may encode real-valued reward signals given by the environment,and a typical goal is tofind weights that yield episodes with a high sum of reward signals,through sequences of appropriate output actions.Sec.5.5will use the notation above to compactly describe a central algorithm of DL,namely,back-propagation(BP)for supervised weight-sharing FNNs and RNNs.(FNNs may be viewed as RNNs with certainfixed zero weights.)Sec.6will address the more general RL case.3Depth of Credit Assignment Paths(CAPs)and of ProblemsTo measure whether credit assignment in a given NN application is of the deep or shallow type,I introduce the concept of Credit Assignment Paths or CAPs,which are chains of possibly causal links between events.Let usfirst focus on SL.Consider two events x p and x q(1≤p<q≤T).Depending on the appli-cation,they may have a Potential Direct Causal Connection(PDCC)expressed by the Boolean predicate pdcc(p,q),which is true if and only if p∈in q.Then the2-element list(p,q)is defined to be a CAP from p to q(a minimal one).A learning algorithm may be allowed to change w v(p,q)to improve performance in future episodes.More general,possibly indirect,Potential Causal Connections(PCC)are expressed by the recursively defined Boolean predicate pcc(p,q),which in the SL case is true only if pdcc(p,q),or if pcc(p,k)for some k and pdcc(k,q).In the latter case,appending q to any CAP from p to k yields a CAP from p to q(this is a recursive definition,too).The set of such CAPs may be large but isfinite.Note that the same weight may affect many different PDCCs between successive events listed by a given CAP,e.g.,in the case of RNNs, or weight-sharing FNNs.Suppose a CAP has the form(...,k,t,...,q),where k and t(possibly t=q)are thefirst successive elements with modifiable w v(k,t).Then the length of the suffix list(t,...,q)is called the CAP’s depth (which is0if there are no modifiable links at all).This depth limits how far backwards credit assignment can move down the causal chain tofind a modifiable weight.1Suppose an episode and its event sequence x1,...,x T satisfy a computable criterion used to decide whether a given problem has been solved(e.g.,total error E below some threshold).Then the set of used weights is called a solution to the problem,and the depth of the deepest CAP within the sequence is called the solution’s depth.There may be other solutions(yielding different event sequences)with different depths.Given somefixed NN topology,the smallest depth of any solution is called the problem’s depth.Sometimes we also speak of the depth of an architecture:SL FNNs withfixed topology imply a problem-independent maximal problem depth bounded by the number of non-input layers.Certain SL RNNs withfixed weights for all connections except those to output units(Jaeger,2001;Maass et al.,2002; Jaeger,2004;Schrauwen et al.,2007)have a maximal problem depth of1,because only thefinal links in the corresponding CAPs are modifiable.In general,however,RNNs may learn to solve problems of potentially unlimited depth.Note that the definitions above are solely based on the depths of causal chains,and agnostic of the temporal distance between events.For example,shallow FNNs perceiving large“time windows”of in-put events may correctly classify long input sequences through appropriate output events,and thus solve shallow problems involving long time lags between relevant events.At which problem depth does Shallow Learning end,and Deep Learning begin?Discussions with DL experts have not yet yielded a conclusive response to this question.Instead of committing myself to a precise answer,let me just define for the purposes of this overview:problems of depth>10require Very Deep Learning.The difficulty of a problem may have little to do with its depth.Some NNs can quickly learn to solve certain deep problems,e.g.,through random weight guessing(Sec.5.9)or other types of direct search (Sec.6.6)or indirect search(Sec.6.7)in weight space,or through training an NNfirst on shallow problems whose solutions may then generalize to deep problems,or through collapsing sequences of(non)linear operations into a single(non)linear operation—but see an analysis of non-trivial aspects of deep linear networks(Baldi and Hornik,1994,Section B).In general,however,finding an NN that precisely models a given training set is an NP-complete problem(Judd,1990;Blum and Rivest,1992),also in the case of deep NNs(S´ıma,1994;de Souto et al.,1999;Windisch,2005);compare a survey of negative results(S´ıma, 2002,Section1).Above we have focused on SL.In the more general case of RL in unknown environments,pcc(p,q) is also true if x p is an output event and x q any later input event—any action may affect the environment and thus any later perception.(In the real world,the environment may even influence non-input events computed on a physical hardware entangled with the entire universe,but this is ignored here.)It is possible to model and replace such unmodifiable environmental PCCs through a part of the NN that has already learned to predict(through some of its units)input events(including reward signals)from former input events and actions(Sec.6.1).Its weights are frozen,but can help to assign credit to other,still modifiable weights used to compute actions(Sec.6.1).This approach may lead to very deep CAPs though.Some DL research is about automatically rephrasing problems such that their depth is reduced(Sec.4). In particular,sometimes UL is used to make SL problems less deep,e.g.,Sec.5.10.Often Dynamic Programming(Sec.4.1)is used to facilitate certain traditional RL problems,e.g.,Sec.6.2.Sec.5focuses on CAPs for SL,Sec.6on the more complex case of RL.4Recurring Themes of Deep Learning4.1Dynamic Programming(DP)for DLOne recurring theme of DL is Dynamic Programming(DP)(Bellman,1957),which can help to facili-tate credit assignment under certain assumptions.For example,in SL NNs,backpropagation itself can 1An alternative would be to count only modifiable links when measuring depth.In many typical NN applications this would not make a difference,but in some it would,e.g.,Sec.6.1.be viewed as a DP-derived method(Sec.5.5).In traditional RL based on strong Markovian assumptions, DP-derived methods can help to greatly reduce problem depth(Sec.6.2).DP algorithms are also essen-tial for systems that combine concepts of NNs and graphical models,such as Hidden Markov Models (HMMs)(Stratonovich,1960;Baum and Petrie,1966)and Expectation Maximization(EM)(Dempster et al.,1977),e.g.,(Bottou,1991;Bengio,1991;Bourlard and Morgan,1994;Baldi and Chauvin,1996; Jordan and Sejnowski,2001;Bishop,2006;Poon and Domingos,2011;Dahl et al.,2012;Hinton et al., 2012a).4.2Unsupervised Learning(UL)Facilitating Supervised Learning(SL)and RL Another recurring theme is how UL can facilitate both SL(Sec.5)and RL(Sec.6).UL(Sec.5.6.4) is normally used to encode raw incoming data such as video or speech streams in a form that is more convenient for subsequent goal-directed learning.In particular,codes that describe the original data in a less redundant or more compact way can be fed into SL(Sec.5.10,5.15)or RL machines(Sec.6.4),whose search spaces may thus become smaller(and whose CAPs shallower)than those necessary for dealing with the raw data.UL is closely connected to the topics of regularization and compression(Sec.4.3,5.6.3). 4.3Occam’s Razor:Compression and Minimum Description Length(MDL) Occam’s razor favors simple solutions over complex ones.Given some programming language,the prin-ciple of Minimum Description Length(MDL)can be used to measure the complexity of a solution candi-date by the length of the shortest program that computes it(e.g.,Solomonoff,1964;Kolmogorov,1965b; Chaitin,1966;Wallace and Boulton,1968;Levin,1973a;Rissanen,1986;Blumer et al.,1987;Li and Vit´a nyi,1997;Gr¨u nwald et al.,2005).Some methods explicitly take into account program runtime(Al-lender,1992;Watanabe,1992;Schmidhuber,2002,1995);many consider only programs with constant runtime,written in non-universal programming languages(e.g.,Rissanen,1986;Hinton and van Camp, 1993).In the NN case,the MDL principle suggests that low NN weight complexity corresponds to high NN probability in the Bayesian view(e.g.,MacKay,1992;Buntine and Weigend,1991;De Freitas,2003), and to high generalization performance(e.g.,Baum and Haussler,1989),without overfitting the training data.Many methods have been proposed for regularizing NNs,that is,searching for solution-computing, low-complexity SL NNs(Sec.5.6.3)and RL NNs(Sec.6.7).This is closely related to certain UL methods (Sec.4.2,5.6.4).4.4Learning Hierarchical Representations Through Deep SL,UL,RLMany methods of Good Old-Fashioned Artificial Intelligence(GOFAI)(Nilsson,1980)as well as more recent approaches to AI(Russell et al.,1995)and Machine Learning(Mitchell,1997)learn hierarchies of more and more abstract data representations.For example,certain methods of syntactic pattern recog-nition(Fu,1977)such as grammar induction discover hierarchies of formal rules to model observations. The partially(un)supervised Automated Mathematician/EURISKO(Lenat,1983;Lenat and Brown,1984) continually learns concepts by combining previously learnt concepts.Such hierarchical representation learning(Ring,1994;Bengio et al.,2013;Deng and Yu,2014)is also a recurring theme of DL NNs for SL (Sec.5),UL-aided SL(Sec.5.7,5.10,5.15),and hierarchical RL(Sec.6.5).Often,abstract hierarchical representations are natural by-products of data compression(Sec.4.3),e.g.,Sec.5.10.4.5Fast Graphics Processing Units(GPUs)for DL in NNsWhile the previous millennium saw several attempts at creating fast NN-specific hardware(e.g.,Jackel et al.,1990;Faggin,1992;Ramacher et al.,1993;Widrow et al.,1994;Heemskerk,1995;Korkin et al., 1997;Urlbe,1999),and at exploiting standard hardware(e.g.,Anguita et al.,1994;Muller et al.,1995; Anguita and Gomes,1996),the new millennium brought a DL breakthrough in form of cheap,multi-processor graphics cards or GPUs.GPUs are widely used for video games,a huge and competitive market that has driven down hardware prices.GPUs excel at fast matrix and vector multiplications required not only for convincing virtual realities but also for NN training,where they can speed up learning by a factorof50and more.Some of the GPU-based FNN implementations(Sec.5.16-5.19)have greatly contributed to recent successes in contests for pattern recognition(Sec.5.19-5.22),image segmentation(Sec.5.21), and object detection(Sec.5.21-5.22).5Supervised NNs,Some Helped by Unsupervised NNsThe main focus of current practical applications is on Supervised Learning(SL),which has dominated re-cent pattern recognition contests(Sec.5.17-5.22).Several methods,however,use additional Unsupervised Learning(UL)to facilitate SL(Sec.5.7,5.10,5.15).It does make sense to treat SL and UL in the same section:often gradient-based methods,such as BP(Sec.5.5.1),are used to optimize objective functions of both UL and SL,and the boundary between SL and UL may blur,for example,when it comes to time series prediction and sequence classification,e.g.,Sec.5.10,5.12.A historical timeline format will help to arrange subsections on important inspirations and techni-cal contributions(although such a subsection may span a time interval of many years).Sec.5.1briefly mentions early,shallow NN models since the1940s,Sec.5.2additional early neurobiological inspiration relevant for modern Deep Learning(DL).Sec.5.3is about GMDH networks(since1965),perhaps thefirst (feedforward)DL systems.Sec.5.4is about the relatively deep Neocognitron NN(1979)which is similar to certain modern deep FNN architectures,as it combines convolutional NNs(CNNs),weight pattern repli-cation,and winner-take-all(WTA)mechanisms.Sec.5.5uses the notation of Sec.2to compactly describe a central algorithm of DL,namely,backpropagation(BP)for supervised weight-sharing FNNs and RNNs. It also summarizes the history of BP1960-1981and beyond.Sec.5.6describes problems encountered in the late1980s with BP for deep NNs,and mentions several ideas from the previous millennium to overcome them.Sec.5.7discusses afirst hierarchical stack of coupled UL-based Autoencoders(AEs)—this concept resurfaced in the new millennium(Sec.5.15).Sec.5.8is about applying BP to CNNs,which is important for today’s DL applications.Sec.5.9explains BP’s Fundamental DL Problem(of vanishing/exploding gradients)discovered in1991.Sec.5.10explains how a deep RNN stack of1991(the History Compressor) pre-trained by UL helped to solve previously unlearnable DL benchmarks requiring Credit Assignment Paths(CAPs,Sec.3)of depth1000and more.Sec.5.11discusses a particular WTA method called Max-Pooling(MP)important in today’s DL FNNs.Sec.5.12mentions afirst important contest won by SL NNs in1994.Sec.5.13describes a purely supervised DL RNN(Long Short-Term Memory,LSTM)for problems of depth1000and more.Sec.5.14mentions an early contest of2003won by an ensemble of shallow NNs, as well as good pattern recognition results with CNNs and LSTM RNNs(2003).Sec.5.15is mostly about Deep Belief Networks(DBNs,2006)and related stacks of Autoencoders(AEs,Sec.5.7)pre-trained by UL to facilitate BP-based SL.Sec.5.16mentions thefirst BP-trained MPCNNs(2007)and GPU-CNNs(2006). Sec.5.17-5.22focus on official competitions with secret test sets won by(mostly purely supervised)DL NNs since2009,in sequence recognition,image classification,image segmentation,and object detection. Many RNN results depended on LSTM(Sec.5.13);many FNN results depended on GPU-based FNN code developed since2004(Sec.5.16,5.17,5.18,5.19),in particular,GPU-MPCNNs(Sec.5.19).5.11940s and EarlierNN research started in the1940s(e.g.,McCulloch and Pitts,1943;Hebb,1949);compare also later work on learning NNs(Rosenblatt,1958,1962;Widrow and Hoff,1962;Grossberg,1969;Kohonen,1972; von der Malsburg,1973;Narendra and Thathatchar,1974;Willshaw and von der Malsburg,1976;Palm, 1980;Hopfield,1982).In a sense NNs have been around even longer,since early supervised NNs were essentially variants of linear regression methods going back at least to the early1800s(e.g.,Legendre, 1805;Gauss,1809,1821).Early NNs had a maximal CAP depth of1(Sec.3).5.2Around1960:More Neurobiological Inspiration for DLSimple cells and complex cells were found in the cat’s visual cortex(e.g.,Hubel and Wiesel,1962;Wiesel and Hubel,1959).These cellsfire in response to certain properties of visual sensory inputs,such as theorientation of plex cells exhibit more spatial invariance than simple cells.This inspired later deep NN architectures(Sec.5.4)used in certain modern award-winning Deep Learners(Sec.5.19-5.22).5.31965:Deep Networks Based on the Group Method of Data Handling(GMDH) Networks trained by the Group Method of Data Handling(GMDH)(Ivakhnenko and Lapa,1965; Ivakhnenko et al.,1967;Ivakhnenko,1968,1971)were perhaps thefirst DL systems of the Feedforward Multilayer Perceptron type.The units of GMDH nets may have polynomial activation functions imple-menting Kolmogorov-Gabor polynomials(more general than traditional NN activation functions).Given a training set,layers are incrementally grown and trained by regression analysis,then pruned with the help of a separate validation set(using today’s terminology),where Decision Regularisation is used to weed out superfluous units.The numbers of layers and units per layer can be learned in problem-dependent fashion. This is a good example of hierarchical representation learning(Sec.4.4).There have been numerous ap-plications of GMDH-style networks,e.g.(Ikeda et al.,1976;Farlow,1984;Madala and Ivakhnenko,1994; Ivakhnenko,1995;Kondo,1998;Kord´ık et al.,2003;Witczak et al.,2006;Kondo and Ueno,2008).5.41979:Convolution+Weight Replication+Winner-Take-All(WTA)Apart from deep GMDH networks(Sec.5.3),the Neocognitron(Fukushima,1979,1980,2013a)was per-haps thefirst artificial NN that deserved the attribute deep,and thefirst to incorporate the neurophysiolog-ical insights of Sec.5.2.It introduced convolutional NNs(today often called CNNs or convnets),where the(typically rectangular)receptivefield of a convolutional unit with given weight vector is shifted step by step across a2-dimensional array of input values,such as the pixels of an image.The resulting2D array of subsequent activation events of this unit can then provide inputs to higher-level units,and so on.Due to massive weight replication(Sec.2),relatively few parameters may be necessary to describe the behavior of such a convolutional layer.Competition layers have WTA subsets whose maximally active units are the only ones to adopt non-zero activation values.They essentially“down-sample”the competition layer’s input.This helps to create units whose responses are insensitive to small image shifts(compare Sec.5.2).The Neocognitron is very similar to the architecture of modern,contest-winning,purely super-vised,feedforward,gradient-based Deep Learners with alternating convolutional and competition lay-ers(e.g.,Sec.5.19-5.22).Fukushima,however,did not set the weights by supervised backpropagation (Sec.5.5,5.8),but by local un supervised learning rules(e.g.,Fukushima,2013b),or by pre-wiring.In that sense he did not care for the DL problem(Sec.5.9),although his architecture was comparatively deep indeed.He also used Spatial Averaging(Fukushima,1980,2011)instead of Max-Pooling(MP,Sec.5.11), currently a particularly convenient and popular WTA mechanism.Today’s CNN-based DL machines profita lot from later CNN work(e.g.,LeCun et al.,1989;Ranzato et al.,2007)(Sec.5.8,5.16,5.19).5.51960-1981and Beyond:Development of Backpropagation(BP)for NNsThe minimisation of errors through gradient descent(Hadamard,1908)in the parameter space of com-plex,nonlinear,differentiable,multi-stage,NN-related systems has been discussed at least since the early 1960s(e.g.,Kelley,1960;Bryson,1961;Bryson and Denham,1961;Pontryagin et al.,1961;Dreyfus,1962; Wilkinson,1965;Amari,1967;Bryson and Ho,1969;Director and Rohrer,1969;Griewank,2012),ini-tially within the framework of Euler-LaGrange equations in the Calculus of Variations(e.g.,Euler,1744). Steepest descent in such systems can be performed(Bryson,1961;Kelley,1960;Bryson and Ho,1969)by iterating the ancient chain rule(Leibniz,1676;L’Hˆo pital,1696)in Dynamic Programming(DP)style(Bell-man,1957).A simplified derivation of the method uses the chain rule only(Dreyfus,1962).The methods of the1960s were already efficient in the DP sense.However,they backpropagated derivative information through standard Jacobian matrix calculations from one“layer”to the previous one, explicitly addressing neither direct links across several layers nor potential additional efficiency gains due to network sparsity(but perhaps such enhancements seemed obvious to the authors).。

《2024年深度强化学习综述》范文

《2024年深度强化学习综述》范文

《深度强化学习综述》篇一一、引言深度强化学习（Deep Reinforcement Learning，简称DRL）是人工智能领域中一种重要的学习范式，它结合了深度学习和强化学习的优势，使智能体能够在复杂的动态环境中自主地学习和决策。

本文旨在全面综述深度强化学习的基本原理、发展历程、应用领域以及未来挑战与展望。

二、深度强化学习基本原理深度强化学习结合了深度学习的表示学习和强化学习的决策能力。

在深度强化学习中，智能体通过与环境的交互，在不断地试错过程中学习到如何从状态空间中提取有价值的信息，以最大化长期回报。

具体来说，深度强化学习主要包括以下几个步骤：1. 状态表示：通过深度神经网络将环境状态映射为数值型向量，以便于后续的决策过程。

2. 决策过程：利用强化学习算法，如策略梯度法或值函数法，根据当前状态和历史经验进行决策。

3. 奖励机制：通过给予智能体奖励或惩罚来指导其学习过程，使其在长期的交互过程中学会如何最大化长期回报。

4. 模型训练：通过优化神经网络参数来提高智能体的决策能力。

三、深度强化学习的发展历程深度强化学习的发展经历了从简单到复杂、从理论到实践的过程。

早期的研究主要集中在如何将深度神经网络与强化学习算法相结合，以解决复杂任务中的决策问题。

随着深度学习技术的不断发展，越来越多的研究者开始关注如何将深度强化学习应用于实际问题中。

近年来，随着计算能力的提高和大数据的积累，深度强化学习在许多领域取得了显著的成果。

四、深度强化学习的应用领域深度强化学习在许多领域都取得了广泛的应用，包括但不限于以下几个方面：1. 游戏领域：深度强化学习在游戏领域取得了巨大的成功，如AlphaGo在围棋领域的表现。

此外，它还可以应用于其他类型的游戏，如电子游戏和策略游戏等。

2. 机器人控制：通过深度强化学习，机器人可以在没有先验知识的情况下自主地学习和完成任务。

例如，通过学习模仿人类的行为和技能，机器人可以更加高效地执行各种任务。

深度学习技术介绍PPT课件

深度学习技术介绍PPT课件

根据Marr(1982)年理论，理解一个信息处理系统，具有三个被称为分析层面的内容： 计算理论(computational theory)对应计算目标和任务的抽象定义。 表示和算法(representation and algorithm)是关于输人和输出如何表示和从输入到输
出变换的算法说明。 硬件实现(hardware implementation)是系统的实物物理实现。
29
29
M40 GPU加速特性
30
GPU与CPU连接
通过PCIe与CPU连接, 最大理论带宽8GB/s(gen2.0)、16GB/s(gen3.0) CPU称为主机(host), 显卡(GPU)称为设备(device)
31
31
最优连接数量：4
32
32
目前的GPU使用方案
33
33
CPU困境
34
机器学习还可以进行压缩(compression)。用规则拟合数据，我们能得到比数据更简 单的解释，需要的存储空间更少，处理所需要的计算更少，例如，一旦你掌握了加法 规则，你就不必记忆每对可能数字的和是多少。
机器学习的另一种用途是离群点检测(outlier detection)，即发现那些不遵守规则的 例外实例。在这种情况下，学习规则之后，我们感兴趣的不是规则，而是规则未能覆 盖的例外，他们可能暗示出我们需要注意的异常，如诈骗等。
具体应用-人脸识别
对于人脸识别(face recognition)。输入是人脸 图像，类是需要识别的人，并且学习程序应当 学习人脸图像与身份之间的关联性。人脸会有 更多的类，输入图像也更大一些，并且人脸是 三维的，不同的姿势和光线等都会导致图像的 显著变化。另外，对于特定人脸的输人也会出 现问题，比如说眼镜可能会把眼睛和眉毛遮住 ，胡子可能会把下巴盖住等。