当前位置：文档之家 › A Semiparametric Model for Accurate Camera Response Function Modeling and Exposure Estimati

A Semiparametric Model for Accurate Camera Response Function Modeling and Exposure Estimati

  • 格式：pdf
  • 大小：2.76 MB
  • 文档页数：13

1138IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 8, AUGUST 2005A Semiparametric Model for Accurate Camera Response Function Modeling and Exposure Estimation From Comparametric DataFrank M. Candocia, Member, IEEE, and Daniel A. MandarinoAbstract—A fundamentally new approach that accurately estimates the camera response function from comparametric data, i.e., pixel data from two differently exposed images over a common field of view, is presented. It does so by solving for the camera response function from its associated comparametric relation. The approach offers several advantageous features, including having a complexity that is independent of the number of pixel data considered, allowing for the modeling of saturated pixels, enabling an inherently constrained optimization problem to be solved in an unconstrained manner, and the easy incorporation into an existing framework for joint image registration. This is accomplished by approximating the camera response function with a constrained piecewise linear model so that its solution, within the comparametric camera relation, can be obtained. This results in a semiparametric comparametric model, optimally determined from pixel data, which is directly parameterized in terms of the exposure parameter. Subsequently, it is shown how this semiparametric model is used for exposure estimation from captured images. Finally, we incorporate the semiparametric model within an existing and previously published framework for simultaneous and joint spatial and tonal image registration in order to illustrate the developed model’s performance. Index Terms—Camera response function, comparametric equation, exposure estimation, image registration, piecewise linear modeling, semiparametric model.I. INTRODUCTIONWE PRESENT an approach for the accurate modeling of a camera’s response function so that proper exposure estimates between multiple images can be obtained. Previous tonal alignment work has utilized optoelectronic conversion functions [1], polynomials [2], and both parametric [3], [4] and nonparametric local [5] and global [6] quantimetric [7] processing. Our approach uses the comparametric relation associated with a camera’s response function in the estimation as a convenient means of utilizing the pixel data available to us. In essence, we are solving for the camera response function from its associated comparametric relation. This comparametric relation, and quantimetric principle upon which it is based, has been well founded and established [4], [8]. A nice property of utilizing the quantimetric model is the straightforward mannerManuscript received May 10, 2004; revised September 7, 2004. This work was supported in part by the National Science Foundation under Grant HRD0317692. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Bruno Carpentieri. The authors are with the Department of Electrical and Computer Engineering, Florida International University, Miami, FL 33174 USA (e-mail: candocia@fi). Digital Object Identifier 10.1109/TIP.2005.851696in which spatial and exposure-based registration can be performed by exploiting the group structures inherent to the spatial and exposure transformation models considered. This has been done with success using comparametric models that have been directly [9], [10] and indirectly modeled in terms of the exposure parameter [11]. The advantage of the direct approach lies in its use of predefined and parameterized camera response/comparametric functions for which many have already been defined. Its disadvantages are that, though one of the predefined models might well approximate a camera’s exposure behavior, a highly accurate model—if, indeed, necessary—might not be available nor easily determined. In the indirect approach, the advantage was that a highly accurate comparametric model could be determined from image pixel data and this, in turn, allowed for the first general procedure that performed joint domain and range registration of differently exposed images [12], [13]. The disadvantages were that this model was not parameterized with respect to the exposure parameter (hence, indirect means for its estimation were required) and the associated camera response function could not be estimated from it. This work presents an approach that incorporates the advantages of the previously described direct and indirect approaches, that is, we have the ability of obtaining a camera response and corresponding comparametric function that is directly parameterized in terms of the exposure parameter as in [4] and it will be done directly from comparametric pixel data as in [10] so that a flexible and accurate modeling can be obtained. This is achieved by approximating the camera response function with a constrained, continuous, piecewise linear (PL) function and, subsequently, using this model in the comparametric relation that is directly parameterized with respect to the exposure parameter. This leads to a fundamentally different approach to camera response function estimation that offers several benefits. Specifically, the contributions of this work are in developing an approach that 1) can accurately model a camera response function from pixel data, 2) has a complexity that is independent of the number of pixel data considered, 3) allows for the modeling of saturated pixels, 4) enables an inherently constrained optimization problem to be solved in an unconstrained manner, and 5) is easily incorporated into an existing simultaneous and joint spatial and tonal image registration framework. The relation resulting from the functional composition of the PL modeled camera response function with its inverse leads to a semiparametric comparametric model that is part of an optimality measure to be minimized. Its semiparametric nature comes from1057-7149/$20.00 © 2005 IEEECANDOCIA AND MANDARINO: SEMIPARAMETRIC MODEL FOR ACCURATE CAMERA RESPONSE FUNCTION MODELING1139the model’s ability to allow for a very general class of functional forms and where the number of the model’s parameters can be increased in a systematic way to build ever more flexible models, but where the total number of parameters in the model can be varied independently from the size of the data set [14].1 The remainder of this paper discusses the camera response and exposure estimation approach. Section II describes the quantimetric exposure model and the associated comparametric relation about which this work is based. This is followed by a motivation for the piecewise linear model of the paper and its subsequent definition in Section III. In Section IV, we define our optimality measure and describe how the optimal camera response and exposure estimates are obtained from it. This is followed in Section V with experimental results on three different image sequences. Section VI presents some noteworthy issues and provides additional commentary concerning the approach before the paper concludes in Section VII with a summary and some final remarks. II. PROBLEM DESCRIPTION It is instructive to now describe and formulate the problem addressed in this paper. This brief presentation will establish the footing and motivation for the remainder of the paper. A. Quantimetric Exposure Model The exposure model upon which this work is based is generally described by (1) and is the quantimetric relation detailed in [4]. In this model, is the resulting pixel value at spatial coordinate is a dynamic range compression (DRC) function that limits the range of possible values sensed, embodies the exposure time that modifies the amount of light sensed by the camera , while and each represent pre-DRC noise related to the imaging process (electronically induced noise, lens distortion, point spread function effects, etc.) and post-DRC noise related to the final image representation (quantization, JPEG compression, is a monotonically increasing etc.), respectively. Note that function also known as the camera response function and that where in the absence of light, i.e., when no light is absorbed by the imaging sensor. The relation between and a differently exposed image (over the same field) would be as in (1), except that we would use in place of to account for a different amount of sensed light due to differences in exposure. In an ideal, noiseless setting, we can solve for in (1) and, likewise, for its counterpart , since is a monotonically increasing smooth function whose inverse exists. In the absence of noise, solving for in (1), both as related to and ,1This definition of a semiparametric model has been paraphrased from C.M. Bishop’s book [14, p. 33]. It is being used to stress the type of comparametric modeling being performed—even though the underlying model is of the camera response function.and equating these two expressions yields the comparametric description of exposure change between two images, namely (2) where and indirectly quantifies the exposure difference of image relative to image . Thus, (1) and (2) provide different, though ideally equivalent, descriptions by which the pixel values of two differently exposed images can be related, i.e., using a camera’s response function or its corresponding comparametric relation, respectively. Estimating the camera response function via (1) is not possible when only the pixel values of a captured image are available, and, it is evident from (2) that determining the camera response function from pixel values of two images is not a trivial task as the comparametric relation is composed of the functional composition of this camera response function with a scaled version of its inverse. This paper describes a semiparametric comparametric model that allows for the direct and optimal estimation of from one (or more) image pairs utilizing the relation of (2). As previously mentioned, this offers the advantage of having a model directly parameterized in terms of so that estimating the camera response function from a known value can be accomplished as easily as estimating from one or more image pairs given the camera response model. B. Motivating the Piecewise Linear Camera Response Model The decision to employ a PL camera response model approximation for our estimation problem resulted from many considerations. Initially, the fact that we needed a pixel-based model directly parameterized in terms of required the use of relation (2). Having established this, we needed a reasonable model for that could be used and where the existence of could be guaranteed without much effort. Polynomial models were considered but it became apparent that determining the inverse of such a function is not, in general, easy, and, if it could be done, would generally increase, leading to inthe model order of creased comparametric model order complexity. Further, constraints would have to be placed on the permissible values of could exist over the defined the coefficients of so that range of . Other function approximations such as linear transform decompositions were also considered, but it was not clear how could be guaranteed to be monotonically increasing via coefficient constraints. Drawing on previous experience with PL models, we found that employing such a model for the camera response function very nicely addressed many of the difficulties that seemed inexists, trinsic to the problem at hand. First, to guarantee that we need a parameterizable form for that can be constrained to enforce a monotonically increasing behavior for . Previous work had shown that this could be done by constraining the slopes of a PL function to be positive-valued [15], [16]. Second, was also PL. utilizing a PL form for , it was found that An even more welcomed result was that the functional comvia (2) was PL—hence, the comparaposition of with metric model was not “order-wise” intrinsically more complex than the camera response model. Third, by simply increasing the number of PL segments in , we could, in principle, arbitrarily1140IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 8, AUGUST 2005closely model this continuous function and, in turn, its corresponding comparametric function. Fourth, by appropriately parameterizing the PL model of , we could express the inherently constrained camera response function estimation problem as an unconstrained optimization problem. Due to these and other advantages that will soon be discussed, approximating the camera response function with a PL model so that we could solve for in (2) seemed most appropriate. III. PIECEWISE LINEAR MODELING The piecewise linear comparametric model employed in [11] for joint domain and range registration was slope parameterized, i.e., parameterized in terms of each PL segment’s slope and associated knot locations. In the work of this paper, we reparameterize this model to one that will be called intercept parameterized so that each PL segment’s “ intercept,” along with their corresponding knot locations, serve as the parameter set about which to optimize. At first, this might seem like a trivial change to our camera response function estimation problem but, as it turns out, has a very significant impact on the optimization setup and solution. This point is further addressed in Section IV-A. We now define the intercept parameterized PL camera response function that is employed throughout this paper. First, let us define an segment continuous and piecewise linear function on an interval asFig. 1. Diagram illustrating the variables used in parameterizing the piecewise linear model y (x) of (3) and (4) for the case of N = 3 piecewise linear segments. The (x) each represents a linear segment. The define linear segment intercept-related parameters and the x denote knot locations delimiting the linear segments. In general, we have a function defined over a domain [x ; x ) and, given y , we observe y = (x ); i = 1; . . . ; N .. . .(3). The significance of these relations will become evident in Section IV-A. In order to solve for the camera response in (2), we approxthat is consistent imate it with a range bounded PL function with the general increasing and bounded camera response propto the pixel range erties defined in [4]. First, by normalizing of [0, 1], as done in [4], we can extend slightly the definition of in (3) and (4) so thatwhere , enforces conti, are knots that nuity between segments and the constitute a partition over the previously given interval. Using three segments), we the diagram in Fig. 1 (illustrated for can see that, in general, depends on knots and “inwhere and where tercepts” . Given this parameterization in terms of and , we can express (3) as(6) Notice that, because is a continuous function for all , (6) imand this will be the case in the plies necessarily that remainder of the paper; hence, any use of (5) or others resulting from this point forward. Second, by from it assumes constraining the function to be increasing not only do we satisfy the camera response trait described in [4] but we also allow for the general existence of the camera response function inneeded in (2). Given this functionally increasing reverse can quirement and the definition in (6), the approximation be readily made to satisfy this criteria by constraining the inter, to be strictly positive. Enforcing cept values that is, like its this constraint leads to an inverse function , also continuous and piecewise linear over counterpart segments. It is given by. . .(4) . It is clear from (3) and (4) that, given our for parameterization, each linear segment can be expressed as (5) Further, it is apparent from Fig. 1 that . , so Likewise, we can define that and, for later convenience, we now also define the vector and vector. . .(7)CANDOCIA AND MANDARINO: SEMIPARAMETRIC MODEL FOR ACCURATE CAMERA RESPONSE FUNCTION MODELING1141where for and . Each segis found to simply be the functional inverse of its correment sponding linear segment , i.e., we must satisfy for . Using (5), we find that(8) The final and central model of this paper is the PL approximation to the general comparametric relation of (2). By piecewise linearly modeling, the camera response function, we now have a direct comparametric relation in terms of the exposure parameter that we did not have in previous PL comparametrically based work [11]. The advantages of retaining the properties inherent to PL models and of being able to accurately model the camera response function directly from observed pixel values, as previously mentioned, make this model a fundamentally new and attractive approach to, not just exposure estimation, but to the joint spatial and tonal registration of images that vary in exposure. Using the relations of (6) and (7), the approximation to (2) is given by (9), shown at the bottom of the page. Analyzing this comparametric equation approximation leads to some notable points. 1) The equation is PL. This is readily noted as the functional composition of two linear segments results in a linear segment. Using (5) and (8), we can see that for any, which resulting in a new set of knots then affects all knots in (9) by this very factor. One can then easily appreciate that multiplying all knots by in (10) does not change the equation as the s would cancel out. 3) Its evaluation requires determining which segments in and contribute to argument ’s algebraic image. The unique contributing segment in given argument is that one which satisfies and the unique contributing segment in is that one which satisfies for all . As a side note, not all PL segment will contribute to defining (9) as this depairs and , i.e., the camera pends on the exposure ratio and being modeled. This is demonstrated later in Fig. 4 and discussed in the results section. 4) The equation allows for modeling of camera response function saturation. Saturation occurs when image pormodel tions have been overexposed. The fact that the of (6) saturates (reaches the maximal range value of 1) past the largest knot value in this equation allows for this type of modeling. Given this, (9) is the central relation about which the remainder of this paper is based. It describes a semiparametric piecewise linear (SPL) relation from which we can use image pixel values to estimate both the camera response function and the exposure differences between images. Its main relevance is that it allows for a solution to in (2).2 IV. OPTIMAL PARAMETER ESTIMATION Having established our SPL comparametric model, we wish to use the pixel values of captured images in the estimation of either the camera response function or the exposure ratio between images. The former estimation requires prior knowledge of exposure ratios between the images and the latter estimation requires the established camera response function model. In either case, we will employ the standard sum of squared differences as our optimality measure as it has been deemed sufficiently simple and accurate to the estimation problem at hand.2This is somewhat analogous to numerically obtaining the solution of an ordinary differential equation, but rather than having a relation involving derivatives, we have one involving a functional composition and a functional inverse.(10) .” and this equation is in the familiar linear form “ 2) The function. results in a unique solution only up to a scale factor in . This behavior is consistent with the camera response modeling in [4] as noted therein. We can see this by scaling all knots in (6) by a constant factor ,(9)1142IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 8, AUGUST 2005Following the cost defined in [11], the optimality measure be expressed ascandefined to make expressing (13) more compact. These definitions are and (14) where the subscript “ ” in (14) denotes the summation of terms and dependence on is stressed by including the “ ” syntax in this equation. Note that the partial derivative in (13) depends on which PL segment in (9) the value of corresponds. Since this segment is defined by the composition of two PL segments from (6) and (7), respectively, we must express (13) over and . all possible The other necessary partial derivative that is needed for the camera response function modeling is(11)and that differ in exposure, , constitute comparagram pixel pairs, i.e., image pixel values of the same scene point captured at different camera exposure settings (one from image and the other from image ). Also, is the SPL comparametric model of (9) where the pixel values of and have been normalized to the interval [0, 1] according to the definition of (6). The optimum parameter values will necessarily be found at that point where the cost’s gradient with respect to each parameter value is identically 0, i.e., where . For each parameter value , this partial derivative is simply written aswhere, for two images(12)Note that, given two images, one can, in principle, attempt to solve for the and parameters simultaneously in . This, however, would make the camera response function estimation unnecessarily difficult. Instead, the two sections that follow detail how the camera response function can be estimated parameters, and once this is done, how expovia the and sure estimates from image pairs can be obtained. A. Camera Response is a vector of paIn this estimation problem, rameters that makes explicit the dependence of (9) on intercept related and knot interval-related parameter values as defined following (5). The gradient in our problem requires us to find the and . partial derivative of (11) for each Noting this and working through the differentiation, it can be shown that(15) where, similar to (13), , and where the also has been omitted for noparameter dependence of tational conciseness. The unit impulse and step function are is defined in (14) and we define as described in (13), along with (16) Equation (16) has been defined this way for notational conciseness and consistency with its counterpart in (14). We have now formulated the main derivatives necessary for our camera response modeling from comparametric data. However, as previously discussed, all parameters in the camera response function of (6) are required to be strictly positive quanti, as defined from ties. Furthermore, all knot interval lengths are, by (6)’s definition, also strictly positive. In the knots order to assure that the solution for the and parameters satisfy these constraints, we have opted to work with a parameterization of auxiliary variables that, through a nonlinear mapping, enforce the required positivity constraints. For this, we have chosen to use the softmax nonlinearity [18], common in probabilistic neural network applications, as our variable mapping. That is, we have defined (17)(13) and where the parameter dependence where of has been omitted for notational conciseness. Notice and are the discrete-time unit sample sequence that (impulse function) and discrete-time unit step sequences, and have been respectively [17]. In addition,CANDOCIA AND MANDARINO: SEMIPARAMETRIC MODEL FOR ACCURATE CAMERA RESPONSE FUNCTION MODELING1143and (18) as our nonlinear mapping where, from (17) for and (18), we can see that and are the auxiliary variables and knot interval lengths , rerelated to the intercepts spectively. The advantage of this variable transformation is that the solution for the auxiliary variables is unconstrained. This is seen by noting that the softmax function is defined over the real line in each independent variable; hence, there are no bounds on permissible values of and . Also, it is easily shown that the softmax mapping in (17) and (18) results in and , as well as and . This is, indeed, very convenient for values that sum to 1 result in two reasons. First, positive that is monotonically increasing, defined over the enan tire unit-range interval and whose functional inverse exists—all requisite conditions as previously discussed. Second, because in (9) is scale invariant, we can normalize all the solution to knots in (6) such that they form a partition on the unit interval and, thus, take advantage of an unconstrained solution to the values. It is for these reasons that the intercept-parameterized PL model being employed to approximate the camera response function, as mentioned earlier, is much preferred over its equivalent but slope parameterized version as given in [11]. That is, this straightforward analysis and unconstrained modeling is not easily possible with the slope-parameterized PL description. In order to solve the unconstrained problem, we define the parameter vector in (11) in terms of the auxiliary intercept and where knot interval parameters such that and . By noting that the softmax derivative in (17) [and similarly for (18)] is (19), the partial derivatives with respect to the for auxiliary intercept and knot interval lengths are seen to be (20) and (21) , since each and parameter depends for and values, respectively, as seen in (17) and (18). on all Finally, note that the partial derivatives in the sum of (20) are given in (13) and (19). Similarly, those in (21) are given by in (19) for and (15) and (19) [by exchanging the and , respectively]. In solving for the unconstrained and parameters, we have used a conjugate gradient approach with line search for the parameter optimization due to its well-established convergence properties and because costly Hessian matrix computations can be avoided [19]. B. Exposure Once the camera response function of (6) is estimated by determining the and parameters that minimize (11) by way of the and auxiliary variables, we can estimate the exposure ratio between two images from their pixel values using the cost function of (11). In this estimation problem, , i.e., there is only the parameter vector consists of one parameter that needs estimating. As dictated by (12), we determine the necessary derivative to be (22), shown at the bottom of the page, where we note (23), shown at the bottom of the page, and where (23) comes from utilizing (5) and (6). In arriving at the derivative in (22), note that at any given value of , there is one and one that determines the value and, similarly, one segment over which ofotherwiseotherwise (22)otherwiseotherwise otherwise (23)1144IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 8, AUGUST 2005is defined. However, given the segments each of these two product terms is defined over, we see that necessarily ; hence, the final derivative form shown in (22). Also noteworthy is that, since is a constant, evaluating it at as done in (22) does not alter its constancy; hence, the term, via (23), in the final form of (22). Of further interest in exposure estimation is when the SPL comparametric model is used in the joint domain and range registration of multiple images, as performed in [10]. Besides needing to evaluate the SPL comparametric function’s derivative with respect to the exposure parameter , we will also need to evaluate its (image ’s) derivative with respect to the other image. This derivative would supplant that of (18) in [10] as the SPL comparametric model would be substituted in place of the comparametric model of (3) in [10].3 This SPL comparametric derivative is given by (24), shown at the bottom of the page, is as given in (22) and where whereotherwise otherwise (25) and this results from the relations in (7) and (8). You will notice the result of (25) was also utilized in determining the final form of (24). In (24), as in (22), for any given value of , the product and implies that , hence, its of final form. V. EXPERIMENTAL RESULTS This section will report on results pertaining to three different image sequences. The first is an exposure sequence that will be used to estimate the camera response function and corresponding SPL comparametric relation. The second is also an exposure sequence used to test the approach’s exposure estimating3The comparametric model in [10] is written as f (g ; k ) for reasons addressed in that paper. However, in this work, it is expressed as g (f ; k ) as given by (2) and (9). By re-expressing (9) with f as the dependent variable, the form to substitute in [10] is simply r (r (g )=k ).ability. The third is a sequence captured while the camera was handheld, approximately moved about its optical center and automatically adjusted its exposure setting. Note that all images used in this paper have been resized to 240 320 pixels for generating the results reported. To model the camera response function, we first captured five images at different exposures with a Sony DCRTRV30 camcorder. This exposure sequence is depicted in Fig. 2 and will be denoted the “Flowers” sequence. Note that the five images, – , correspond to Fig. 2(a)–(e), respectively, and that there between any two successive imis an exposure ratio of and in this sequence. To estimate the camera reages sponse function, we simultaneously considered the four ordered , and in Fig. 2 image pairs , and for which exposure ratios of , respectively, are known. The comparagrams of these four image pairs are shown in the scatter plots of Fig. 2(f)–(i). These scatter plots visually depict the pixel-pair data referred to and needed for the optimization of (11). Also, note that more dense portions of the comparagrams are indicated by darker shades of gray. Examining the comparagram data in Fig. 2, we can notice a “wavy” comparametric behavior and pixel saturation resulting from those overexposed images. We can also notice that a curve has been superimposed on each of the four comparagrams of Fig. 2. This was done to illustrate how the SPL comparametric estimate has been able to model both the camera’s “wavy” comparametric behavior as well as the saturation of pixels. To arrive at this comparametric estiPL segments in the of (6), mate, we considered as well as all comparagram data from the four aforementioned image pairs—thus, processing all comparametric data simultaneously. This was used in (11) to solve for and using unconstrained conjugate gradient descent as discussed in Section IV-A. The and values of the PL camera response function were then obtained with the relations in (17) and (18). Having established our camera response function, we estimated the exposure ratio between the “training” image pairs in Fig. 2 as described in Section IV-B. These are noted in the top left corner of each of the four subplots of Fig. 2(f)–(i). We can see there is very good agreement between the estimated values of and the corresponding true values as the maximum relative error is around of 3%. The time to compute the camera response function fromotherwiseotherwise(24)。

合集下载

机器学习题库

机器学习题库

机器学习题库一、 极大似然1、 ML estimation of exponential model (10)A Gaussian distribution is often used to model data on the real line, but is sometimesinappropriate when the data are often close to zero but constrained to be nonnegative. In such cases one can fit an exponential distribution, whose probability density function is given by()1xb p x e b-=Given N observations x i drawn from such a distribution:(a) Write down the likelihood as a function of the scale parameter b.(b) Write down the derivative of the log likelihood.(c) Give a simple expression for the ML estimate for b.2、换成Poisson 分布：()|,0,1,2,...!x e p x y x θθθ-==()()()()()1111log |log log !log log !N Ni i i i N N i i i i l p x x x x N x θθθθθθ======--⎡⎤=--⎢⎥⎣⎦∑∑∑∑3、二、 贝叶斯假设在考试的多项选择中，考生知道正确答案的概率为p ，猜测答案的概率为1-p ，并且假设考生知道正确答案答对题的概率为1，猜中正确答案的概率为1，其中m 为多选项的数目。

fracdiff模块文档说明书

fracdiff模块文档说明书

Package‘fracdiff’October31,2022Version1.5-2VersionNote Released1.5-0on2019-12-09,1.5-1on2020-01-20Date2022-10-31Title Fractionally Differenced ARIMA aka ARFIMA(P,d,q)ModelsDescription Maximum likelihood estimation of the parameters of a fractionallydifferenced ARIMA(p,d,q)model(Haslett and Raftery,Appl.Statistics,1989);including inference and basic methods.Some alternative algorithms to estimate``H''.Imports statsSuggests longmemo,forecast,urcaLicense GPL(>=2)URL https:///mmaechler/fracdiffBugReports https:///mmaechler/fracdiff/issuesEncoding UTF-8NeedsCompilation yesAuthor Martin Maechler[aut,cre](<https:///0000-0002-8685-9910>), Chris Fraley[ctb,cph](S original;Fortran code),Friedrich Leisch[ctb](R port,<https:///0000-0001-7278-1983>),Valderio Reisen[ctb](fdGPH()&fdSperio()),Artur Lemonte[ctb](fdGPH()&fdSperio()),Rob Hyndman[ctb](residuals()&fitted(),<https:///0000-0002-2140-5352>)Maintainer Martin Maechler<******************.ethz.ch>Repository CRANDate/Publication2022-10-3119:50:38UTCR topics documented:confint.fracdiff (2)diffseries (3)12confint.fracdiff fdGPH (4)fdSperio (5)fracdiff (6)fracdiff-methods (9)fracdiff.sim (10)fracdiff.var (12)Index14 confint.fracdiff Confidence Intervals for Fracdiff Model ParametersDescriptionComputes(Wald)confidence intervals for one or more parameters in afitted fracdiff model,see fracdiff.Usage##S3method for class fracdiffconfint(object,parm,level=0.95,...)Argumentsobject an object of class fracdiff,typically result of fracdiff(..).parm a specification of which parameters are to be given confidence intervals,eithera vector of numbers or a vector of names.If missing,all parameters are consid-ered.level the confidence level required....additional argument(s)for methods.ValueA matrix(or vector)with columns giving lower and upper confidence limits for each parameter.These will be labelled as(1-level)/2and1-(1-level)/2in%(by default2.5%and97.5%).WarningAs these confidence intervals use the standard errors returned by fracdiff()(which are based on finite difference approximations to the Hessian)they may end up being much too narrow,see the example in fracdiff.var.Author(s)Spencer Graves posted the initial version to R-help.diffseries3 See Alsothe generic confint;fracdiff modelfitting,notably fracdiff.var()for re-estimating the variance-covariance matrix on which confint()builds entirely.Examplesset.seed(101)ts2<-fracdiff.sim(5000,ar=.2,ma=-.4,d=.3)mFD<-fracdiff(ts2$series,nar=length(ts2$ar),nma=length(ts2$ma))coef(mFD)confint(mFD)diffseries Fractionally Differenciate DataDescriptionDifferenciates the time series data using the approximated binomial expression of the long-memory filter and an estimate of the memory parameter in the ARFIMA(p,d,q)model.Usagediffseries(x,d)Argumentsx numeric vector or univariate time series.d number specifiying the fractional difference order.DetailsSince2018,we are using(an important correction of)the fast algorithm based on the discrete Fourier transform(fft)by Jensen and Nielsen which is significantly faster for large n=length(x).Valuethe fractionally differenced series x.Author(s)Valderio A.Reisen<*****************.br>and Artur J.Lemonte(first slow version),now hid-den as diffseries.0().Current version:Jensen and Nielsen(2014);tweaks by Martin Maechler,2018.4fdGPHReferencesSee those in fdSperio;additionallyReisen,V.A.and Lopes,S.(1999)Some simulations and applications of forecasting long-memory time series models;Journal of Statistical Planning and Inference80,269–287.Reisen,V.A.Cribari-Neto,F.and Jensen,M.J.(2003)Long Memory Inflationary Dynamics.The case of Brazil.Studies in Nonlinear Dynamics and Econometrics7(3),1–16.Jensen,Andreas Noack and Nielsen,MortenØrregaard(2014)A Fast Fractional Difference Algo-rithm.Journal of Time Series Analysis35(5),428–436;doi:10.1111/jtsa.12074.See Alsofracdiff.simExamplesmemory.long<-fracdiff.sim(80,d=0.3)str(mGPH<-fdGPH(memory.long$series))r<-diffseries(memory.long$series,d=mGPH$d)#acf(r)#shouldn t show structure-ideallyfdGPH Geweke and Porter-Hudak Estimator for ARFIMA(p,d,q)DescriptionEstimate the fractional(or“memory”)parameter d in the ARFIMA(p,d,q)model by the method of Geweke and Porter-Hudak(GPH).The GPH estimator is based on the regression equation using the periodogram function as an estimate of the spectral density.UsagefdGPH(x,bandw.exp=0.5)Argumentsx univariate time seriesbandw.exp the bandwidth used in the regression equationDetailsThe function also provides the asymptotic standard deviation and the standard error deviation of the fractional estimator.The bandwidth is bw=trunc(n^bandw.exp),where0<bandw.exp<1and n is the sample size.Default bandw.exp=0.5.fdSperio5Valued GPH estimatesd.as asymptotic standard deviationsd.reg standard error deviationAuthor(s)Valderio A.Reisen and Artur J.LemonteReferencessee those in fdSperio.See AlsofdSperio,fracdiffExamplesmemory.long<-fracdiff.sim(1500,d=0.3)fdGPH(memory.long$series)fdSperio Sperio Estimate for’d’in ARFIMA(p,d,q)DescriptionThis function makes use Reisen(1994)estimator to estimate the memory parameter d in the ARFIMA(p,d,q) model.It is based on the regression equation using the smoothed periodogram function as an esti-mate of the spectral density.UsagefdSperio(x,bandw.exp=0.5,beta=0.9)Argumentsx univariate time series data.bandw.exp numeric:exponent of the bandwidth used in the regression equation.beta numeric:exponent of the bandwidth used in the lag Parzen window.DetailsThe function also provides the asymptotic standard deviation and the standard error deviation of thefractional estimator.The bandwidths are bw=trunc(n^bandw.exp),where0<bandw.exp<1and n is the samplesize.Default bandw.exp=0.5;and bw2=trunc(n^beta),where0<beta<1and n is the sample size.Default beta=0.9.Valuea list with componentsd Sperio estimatesd.as asymptotic standard deviationsd.reg standard error deviationAuthor(s)Valderio A.Reisen<*****************.br>and Artur J.LemonteReferencesGeweke,J.and Porter-Hudak,S.(1983)The estimation and application of long memory time series models.Journal of Time Series Analysis4(4),221–238.Reisen,V.A.(1994)Estimation of the fractional difference parameter in the ARFIMA(p,d,q)model using the smoothed periodogram.Journal Time Series Analysis,15(1),335–350.Reisen,V.A.,B.Abraham,and E.M.M.Toscano(2001)Parametric and semiparametric estima-tions of stationary univariate ARFIMA model.Brazilian Journal of Probability and Statistics14, 185–206.See AlsofdGPH,fracdiffExamplesmemory.long<-fracdiff.sim(1500,d=0.3)spm<-fdSperio(memory.long$series)str(spm,digits=6)fracdiff ML Estimates for Fractionally-Differenced ARIMA(p,d,q)modelsDescriptionCalculates the maximum likelihood estimators of the parameters of a fractionally-differenced ARIMA (p,d,q)model,together(if possible)with their estimated covariance and correlation matrices and standard errors,as well as the value of the maximized likelihood.The likelihood is approximated using the fast and accurate method of Haslett and Raftery(1989).Usagefracdiff(x,nar=0,nma=0,ar=rep(NA,max(nar,1)),ma=rep(NA,max(nma,1)),dtol=NULL,drange=c(0,0.5),h,M=100,trace=0)Argumentsx time series(numeric vector)for the ARIMA modelnar number of autoregressive parameters p.nma number of moving average parameters q.ar initial autoregressive parameters.ma initial moving average parameters.dtol interval of uncertainty for d.If dtol is negative or NULL,the fourth root ofmachine precision will be used.dtol will be altered if necessary by the program.drange interval over which the likelihood function is to be maximized as a function ofd.h size offinite difference interval for numerical derivatives.By default(or if neg-ative),h=min(0.1,eps.5*(1+abs(cllf))),where clff:=log.max.likelihood(as returned)and eps.5:=sqrt(.Machine$double.neg.eps)(typically1.05e-8).This is used to compute afinite difference approximation to the Hessian,andhence only influences the cov,cor,and std.error computations;use fracdiff.var()to change this after the estimation process.M number of terms in the likelihood approximation(see Haslett and Raftery1989).trace optional integer,specifying a trace level.If positive,currently the“outer loop”iterations produce one line of diagnostic output.DetailsThe fracdiff package has—for historical reason,namely,S-plus arima()compatibility—used an unusual parametrization for the MA part,see also the‘Details’section in arima(in standard R’s stats package).The ARMA(i.e.,d=0)model in fracdiff()and fracdiff.sim()isX t−a1X t−1−···−a p X t−p=e t−b1e t−1−...−b q e t−q,where e i are mean zero i.i.d.,for fracdiff()’s estimation,e i∼N(0,σ2).This model indeed has the signs of the MA coefficients b j inverted,compared to other parametrizations,including Wikipedia’s https:///wiki/Autoregressive_moving-average_model and the one of arima.Note that NA’s in the initial values for ar or ma are replaced by0’s.Valuean object of S3class"fracdiff",which is a list with components:log.likelihood logarithm of the maximum likelihoodd optimal fractional-differencing parameterar vector of optimal autoregressive parametersma vector of optimal moving average parameterscovariance.dpq covariance matrix of the parameter estimates(order:d,ar,ma).stderror.dpq standard errors of the parameter estimates c(d,ar,ma).correlation.dpqcorrelation matrix of the parameter estimates(order:d,ar,ma).h interval used for numerical derivatives,see h argument.dtol interval of uncertainty for d;possibly altered from input dtol.M as input.hessian.dpq the approximate Hessian matrix H of2nd order partial derivatives of the likeli-hood with respect to the parameters;this is(internally)used to compute covariance.dpq,the approximate asymptotic covariance matrix as C=(−H)−1.MethodThe optimization is carried out in two levels:an outer univariate unimodal optimization in d over the interval drange(typically[0,.5]),usingBrent’s fmin algorithm),andan inner nonlinear least-squares optimization in the AR and MA parameters to minimize white noisevariance(uses the MINPACK subroutine lm DER).written by Chris Fraley(March1991).WarningThe variance-covariance matrix and consequently the standard errors may be quite inaccurate,seethe example in fracdiff.var.NoteOrdinarily,nar and nma should not be too large(say<10)to avoid degeneracy in the model.Thefunction fracdiff.sim is available for generating test problems.ReferencesJ.Haslett and A.E.Raftery(1989)Space-time Modelling with Long-memory Dependence:As-sessing Ireland’s Wind Power Resource(with Discussion);Applied Statistics38,1–50.R.Brent(1973)Algorithms for Minimization without Derivatives,Prentice-HallJ.J.More,B.S.Garbow,and K.E.Hillstrom(1980)Users Guide for MINPACK-1,TechnicalReport ANL-80-74,Applied Mathematics Division,Argonne National Laboratory.See Alsocoef.fracdiff and other methods for"fracdiff"objects;fracdiff.var()for re-estimation ofvariances or standard errors;fracdiff.simfracdiff-methods9Examplests.test<-fracdiff.sim(5000,ar=.2,ma=-.4,d=.3)fd.<-fracdiff(ts.test$series,nar=length(ts.test$ar),nma=length(ts.test$ma))fd.##Confidence intervalsconfint(fd.)##with iteration outputfd2<-fracdiff(ts.test$series,nar=1,nma=1,trace=1)all.equal(fd.,fd2)fracdiff-methods Many Methods for"fracdiff"ObjectsDescriptionMany“accessor”methods for fracdiff objects,notably summary,coef,vcov,and logLik;further print()methods were needed.Usage##S3method for class fracdiffcoef(object,...)##S3method for class fracdifflogLik(object,...)##S3method for class fracdiffprint(x,digits=getOption("digits"),...)##S3method for class fracdiffsummary(object,symbolic.cor=FALSE,...)##S3method for class summary.fracdiffprint(x,digits=max(3,getOption("digits")-3),correlation=FALSE,symbolic.cor=x$symbolic.cor,signif.stars=getOption("show.signif.stars"),...)##S3method for class fracdifffitted(object,...)##S3method for class fracdiffresiduals(object,...)##S3method for class fracdiffvcov(object,...)Argumentsx,object object of class fracdiff.digits the number of significant digits to use when printing....further arguments passed from and to methods.10fracdiff.sim correlation logical;if TRUE,the correlation matrix of the estimated parameters is returnedand printed.symbolic.cor logical.If TRUE,print the correlations in a symbolic form(see symnum)ratherthan as numbers.signif.stars logical.If TRUE,“significance stars”are printed for each coefficient.Author(s)Martin Maechler;Rob Hyndman contributed the residuals()and fitted()methods.See Alsofracdiff to get"fracdiff"objects,confint.fracdiff for the confint method;further,fracdiff.var. Examplesset.seed(7)ts4<-fracdiff.sim(10000,ar=c(0.6,-.05,-0.2),ma=-0.4,d=0.2)modFD<-fracdiff(ts4$series,nar=length(ts4$ar),nma=length(ts4$ma))##->warning(singular Hessian)%%FIXMEcoef(modFD)#the estimated parametersvcov(modFD)smFD<-summary(modFD)smFDcoef(smFD)#gives the whole tableAIC(modFD)#AIC works because of the logLik()methodstopifnot(exprs={})fracdiff.sim Simulate fractional ARIMA Time SeriesDescriptionGenerates simulated long-memory time series data from the fractional ARIMA(p,d,q)model.Thisis a test problem generator for fracdiff.Note that the MA coefficients have inverted signs compared to other parametrizations,see the detailsin fracdiff.Usagefracdiff.sim(n,ar=NULL,ma=NULL,d,rand.gen=rnorm,innov=rand.gen(n+q,...),n.start=NA,backComp=TRUE,allow.0.nstart=FALSE,start.innov=rand.gen(n.start,...),...,mu=0)fracdiff.sim11Argumentsn length of the time series.ar vector of autoregressive parameters;empty by default.ma vector of moving average parameters;empty by default.d fractional differencing parameter.rand.gen a function to generate the innovations;the default,rnorm generates white N(0,1) noise.innov an optional times series of innovations.If not provided,rand.gen()is used.n.start length of“burn-in”period.If NA,the default,the same value as in arima.sim is computed.backComp logical indicating if back compatibility with older versions of fracdiff.sim is desired.Otherwise,for d=0,compatibility with R’s arima.sim is achieved.allow.0.nstart logical indicating if n.start=0should be allowed even when p+q>0.This not recommended unless for producing the same series as with older versions offracdiff.sim.start.innov an optional vector of innovations to be used for the burn-in period.If supplied there must be at least n.start values....additional arguments for rand.gen().Most usefully,the standard deviation of the innovations generated by rnorm can be specified by sd.mu time series mean(added at the end).Valuea list containing the following elements:series time seriesar,ma,d,mu,n.startsame as inputSee Alsofracdiff,also for references;arima.simExamples##Pretty(too)short to"see"the long memoryfracdiff.sim(100,ar=.2,ma=.4,d=.3)##longer with"extreme"ar:r<-fracdiff.sim(n=1500,ar=-0.9,d=0.3)plot(as.ts(r$series))##Show that MA coefficients meaning is inverted##compared to stats::arima:AR<-0.7MA<--0.5n.st<-2AR<-c(0.7,-0.1)MA<-c(-0.5,0.4)n<-512;sd<-0.1n.st<-10set.seed(101)Y1<-arima.sim(list(ar=AR,ma=MA),n=n,n.start=n.st,sd=sd)plot(Y1)#For our fracdiff,reverse the MA sign:set.seed(101)Y2<-fracdiff.sim(n=n,ar=AR,ma=-MA,d=0,n.start=n.st,sd=sd)$serieslines(Y2,col=adjustcolor("red",0.5))##..no,you don t need glasses;-)Y2is Y1shifted slightly## rotate left by k(k<0:rotate right)rot<-function(x,k){stopifnot(k==round(k))n<-length(x)if(k<-k%%n)x[c((k+1):n,1:k)]else x}k<-n.st-2Y2.s<-rot(Y2,k)head.matrix(cbind(Y1,Y2.s))plot(Y1,Y2.s);i<-(n-k+1):ntext(Y1[i],Y2.s[i],i,adj=c(0,0)-.1,col=2)##With backComp=FALSE,get*the same*as arima.sim():set.seed(101)Y2.<-fracdiff.sim(n=n,ar=AR,ma=-MA,d=0,n.start=n.st,sd=sd,backComp=FALSE)$series stopifnot(all.equal(c(Y1),Y2.,tolerance=1e-15))fracdiff.var Recompute Covariance Estimate for fracdiffDescriptionAllows thefinite-difference interval to be altered for recomputation of the covariance estimate for fracdiff.Usagefracdiff.var(x,fracdiff.out,h)Argumentsx a univariate time series or a vector.Missing values(NAs)are not allowed.fracdiff.out output from fracdiff for time series x.hfinite-difference interval for approximating partial derivatives with respect to thed parameter.Valuean object of S3class"fracdiff",i.e.,basically a list with the same elements as the result from fracdiff,but with possibly different values for the hessian,covariance,and correlation matrices and for standard error,as well as for h.See Alsofracdiff,also for references.Examples##Generate a fractionally-differenced ARIMA(1,d,1)model:ts.test<-fracdiff.sim(10000,ar=.2,ma=.4,d=.3)##estimate the parameters in an ARIMA(1,d,1)model for the simulated seriesfd.out<-fracdiff(ts.test$ser,nar=1,nma=1)##Modify the covariance estimate by changing the finite-difference interval(fd.o2<-fracdiff.var(ts.test$series,fd.out,h=.0001))##looks identical as print(fd.out),##however these(e.g.)differ:vcov(fd.out)vcov(fd.o2)##A case,were the default variance is*clearly*way too small:set.seed(1);fdc<-fracdiff(X<-fracdiff.sim(n=100,d=0.25)$series)fdc#Confidence intervals just based on asymp.normal approx.and std.errors:confint(fdc)#ridiculously too narrowIndex∗modelsconfint.fracdiff,2fracdiff-methods,9∗printfracdiff-methods,9∗tsdiffseries,3fdGPH,4fdSperio,5fracdiff,6fracdiff.sim,10fracdiff.var,12arima,7arima.sim,11class,7,13coef,9coef.fracdiff,8coef.fracdiff(fracdiff-methods),9 confint,3,10confint.fracdiff,2,10 diffseries,3fdGPH,4,6fdSperio,4,5,5fft,3fitted,10fitted.fracdiff(fracdiff-methods),9 fracdiff,2,3,5,6,6,9–11,13 fracdiff-methods,9fracdiff.sim,4,7,8,10fracdiff.var,2,3,7,8,10,12 logLik,9logLik.fracdiff(fracdiff-methods),9 print,9print.fracdiff(fracdiff-methods),9print.summary.fracdiff(fracdiff-methods),9residuals,10residuals.fracdiff(fracdiff-methods),9rnorm,11summary,9summary.fracdiff(fracdiff-methods),9symnum,10vcov,9vcov.fracdiff(fracdiff-methods),914。

最小参数大语言模型-概述说明以及解释

最小参数大语言模型-概述说明以及解释

最小参数大语言模型-概述说明以及解释1.引言1.1 概述概述随着人工智能技术的快速发展，大语言模型成为了自然语言处理领域的热门研究方向。

大语言模型作为一类能够自动生成连贯、有意义的文本的模型，被广泛应用于机器翻译、问答系统、自动摘要等各种语言相关任务中。

然而，大语言模型通常需要庞大的参数规模，这给模型的训练和部署带来了巨大的计算和存储资源消耗。

为了解决这个问题，学术界提出了最小参数大语言模型的概念。

最小参数大语言模型旨在通过减少模型的参数数量来降低计算和存储开销，同时保持模型的语言生成能力和模型的泛化能力。

与传统的大语言模型相比，最小参数大语言模型具有更轻量级的模型结构，更高效的推理速度以及更低的资源消耗。

本文将深入探讨最小参数大语言模型的定义、特点以及其在各个领域的应用情况。

通过对最小参数大语言模型的分析，我们可以更好地理解这一模型的优势和发展前景，并为进一步的研究和实践提供借鉴和参考。

1.2文章结构文章结构是指文章的组织结构和框架，它对于整篇文章的逻辑性和连贯性起着至关重要的作用。

本文将按照如下结构来进行展开：1. 引言：该部分将首先对最小参数大语言模型进行概述，介绍其基本概念和特点，然后说明文章的结构和目的。

2. 正文：该部分将分为两个小节，分别讨论最小参数模型的定义与特点以及其应用领域。

2.1 最小参数模型的定义与特点：在这一小节中，将详细介绍最小参数模型的定义，即其具体的模型结构和参数设置。

同时，还会探讨最小参数模型相对于传统大语言模型的一些独特特点，如模型的简洁性、训练和推理效率等。

2.2 最小参数模型的应用领域：这一小节将具体阐述最小参数模型在实际应用中的各个领域和场景。

例如，自然语言处理领域中的文本生成、机器翻译和语言模型压缩等。

3. 结论：最后一部分将对最小参数模型的优势进行总结，并展望其未来的发展前景。

特别是在大数据时代背景下，最小参数模型将发挥出更大的应用潜力，为人工智能领域带来更多可能性。

机器学习专业词汇中英文对照

机器学习专业词汇中英文对照

机器学习专业词汇中英⽂对照activation 激活值activation function 激活函数additive noise 加性噪声autoencoder ⾃编码器Autoencoders ⾃编码算法average firing rate 平均激活率average sum-of-squares error 均⽅差backpropagation 后向传播basis 基basis feature vectors 特征基向量batch gradient ascent 批量梯度上升法Bayesian regularization method 贝叶斯规则化⽅法Bernoulli random variable 伯努利随机变量bias term 偏置项binary classfication ⼆元分类class labels 类型标记concatenation 级联conjugate gradient 共轭梯度contiguous groups 联通区域convex optimization software 凸优化软件convolution 卷积cost function 代价函数covariance matrix 协⽅差矩阵DC component 直流分量decorrelation 去相关degeneracy 退化demensionality reduction 降维derivative 导函数diagonal 对⾓线diffusion of gradients 梯度的弥散eigenvalue 特征值eigenvector 特征向量error term 残差feature matrix 特征矩阵feature standardization 特征标准化feedforward architectures 前馈结构算法feedforward neural network 前馈神经⽹络feedforward pass 前馈传导fine-tuned 微调first-order feature ⼀阶特征forward pass 前向传导forward propagation 前向传播Gaussian prior ⾼斯先验概率generative model ⽣成模型gradient descent 梯度下降Greedy layer-wise training 逐层贪婪训练⽅法grouping matrix 分组矩阵Hadamard product 阿达马乘积Hessian matrix Hessian 矩阵hidden layer 隐含层hidden units 隐藏神经元Hierarchical grouping 层次型分组higher-order features 更⾼阶特征highly non-convex optimization problem ⾼度⾮凸的优化问题histogram 直⽅图hyperbolic tangent 双曲正切函数hypothesis 估值，假设identity activation function 恒等激励函数IID 独⽴同分布illumination 照明inactive 抑制independent component analysis 独⽴成份分析input domains 输⼊域input layer 输⼊层intensity 亮度/灰度intercept term 截距KL divergence 相对熵KL divergence KL分散度k-Means K-均值learning rate 学习速率least squares 最⼩⼆乘法linear correspondence 线性响应linear superposition 线性叠加line-search algorithm 线搜索算法local mean subtraction 局部均值消减local optima 局部最优解logistic regression 逻辑回归loss function 损失函数low-pass filtering 低通滤波magnitude 幅值MAP 极⼤后验估计maximum likelihood estimation 极⼤似然估计mean 平均值MFCC Mel 倒频系数multi-class classification 多元分类neural networks 神经⽹络neuron 神经元Newton’s method ⽜顿法non-convex function ⾮凸函数non-linear feature ⾮线性特征norm 范式norm bounded 有界范数norm constrained 范数约束normalization 归⼀化numerical roundoff errors 数值舍⼊误差numerically checking 数值检验numerically reliable 数值计算上稳定object detection 物体检测objective function ⽬标函数off-by-one error 缺位错误orthogonalization 正交化output layer 输出层overall cost function 总体代价函数over-complete basis 超完备基over-fitting 过拟合parts of objects ⽬标的部件part-whole decompostion 部分-整体分解PCA 主元分析penalty term 惩罚因⼦per-example mean subtraction 逐样本均值消减pooling 池化pretrain 预训练principal components analysis 主成份分析quadratic constraints ⼆次约束RBMs 受限Boltzman机reconstruction based models 基于重构的模型reconstruction cost 重建代价reconstruction term 重构项redundant 冗余reflection matrix 反射矩阵regularization 正则化regularization term 正则化项rescaling 缩放robust 鲁棒性run ⾏程second-order feature ⼆阶特征sigmoid activation function S型激励函数significant digits 有效数字singular value 奇异值singular vector 奇异向量smoothed L1 penalty 平滑的L1范数惩罚Smoothed topographic L1 sparsity penalty 平滑地形L1稀疏惩罚函数smoothing 平滑Softmax Regresson Softmax回归sorted in decreasing order 降序排列source features 源特征sparse autoencoder 消减归⼀化Sparsity 稀疏性sparsity parameter 稀疏性参数sparsity penalty 稀疏惩罚square function 平⽅函数squared-error ⽅差stationary 平稳性（不变性）stationary stochastic process 平稳随机过程step-size 步长值supervised learning 监督学习symmetric positive semi-definite matrix 对称半正定矩阵symmetry breaking 对称失效tanh function 双曲正切函数the average activation 平均活跃度the derivative checking method 梯度验证⽅法the empirical distribution 经验分布函数the energy function 能量函数the Lagrange dual 拉格朗⽇对偶函数the log likelihood 对数似然函数the pixel intensity value 像素灰度值the rate of convergence 收敛速度topographic cost term 拓扑代价项topographic ordered 拓扑秩序transformation 变换translation invariant 平移不变性trivial answer 平凡解under-complete basis 不完备基unrolling 组合扩展unsupervised learning ⽆监督学习variance ⽅差vecotrized implementation 向量化实现vectorization ⽮量化visual cortex 视觉⽪层weight decay 权重衰减weighted average 加权平均值whitening ⽩化zero-mean 均值为零Letter AAccumulated error backpropagation 累积误差逆传播Activation Function 激活函数Adaptive Resonance Theory/ART ⾃适应谐振理论Addictive model 加性学习Adversarial Networks 对抗⽹络Affine Layer 仿射层Affinity matrix 亲和矩阵Agent 代理 / 智能体Algorithm 算法Alpha-beta pruning α-β剪枝Anomaly detection 异常检测Approximation 近似Area Under ROC Curve／AUC Roc 曲线下⾯积Artificial General Intelligence/AGI 通⽤⼈⼯智能Artificial Intelligence/AI ⼈⼯智能Association analysis 关联分析Attention mechanism 注意⼒机制Attribute conditional independence assumption 属性条件独⽴性假设Attribute space 属性空间Attribute value 属性值Autoencoder ⾃编码器Automatic speech recognition ⾃动语⾳识别Automatic summarization ⾃动摘要Average gradient 平均梯度Average-Pooling 平均池化Letter BBackpropagation Through Time 通过时间的反向传播Backpropagation/BP 反向传播Base learner 基学习器Base learning algorithm 基学习算法Batch Normalization/BN 批量归⼀化Bayes decision rule 贝叶斯判定准则Bayes Model Averaging／BMA 贝叶斯模型平均Bayes optimal classifier 贝叶斯最优分类器Bayesian decision theory 贝叶斯决策论Bayesian network 贝叶斯⽹络Between-class scatter matrix 类间散度矩阵Bias 偏置 / 偏差Bias-variance decomposition 偏差-⽅差分解Bias-Variance Dilemma 偏差 – ⽅差困境Bi-directional Long-Short Term Memory/Bi-LSTM 双向长短期记忆Binary classification ⼆分类Binomial test ⼆项检验Bi-partition ⼆分法Boltzmann machine 玻尔兹曼机Bootstrap sampling ⾃助采样法／可重复采样／有放回采样Bootstrapping ⾃助法Break-Event Point／BEP 平衡点Letter CCalibration 校准Cascade-Correlation 级联相关Categorical attribute 离散属性Class-conditional probability 类条件概率Classification and regression tree/CART 分类与回归树Classifier 分类器Class-imbalance 类别不平衡Closed -form 闭式Cluster 簇/类/集群Cluster analysis 聚类分析Clustering 聚类Clustering ensemble 聚类集成Co-adapting 共适应Coding matrix 编码矩阵COLT 国际学习理论会议Committee-based learning 基于委员会的学习Competitive learning 竞争型学习Component learner 组件学习器Comprehensibility 可解释性Computation Cost 计算成本Computational Linguistics 计算语⾔学Computer vision 计算机视觉Concept drift 概念漂移Concept Learning System /CLS 概念学习系统Conditional entropy 条件熵Conditional mutual information 条件互信息Conditional Probability Table／CPT 条件概率表Conditional random field/CRF 条件随机场Conditional risk 条件风险Confidence 置信度Confusion matrix 混淆矩阵Connection weight 连接权Connectionism 连结主义Consistency ⼀致性／相合性Contingency table 列联表Continuous attribute 连续属性Convergence 收敛Conversational agent 会话智能体Convex quadratic programming 凸⼆次规划Convexity 凸性Convolutional neural network/CNN 卷积神经⽹络Co-occurrence 同现Correlation coefficient 相关系数Cosine similarity 余弦相似度Cost curve 成本曲线Cost Function 成本函数Cost matrix 成本矩阵Cost-sensitive 成本敏感Cross entropy 交叉熵Cross validation 交叉验证Crowdsourcing 众包Curse of dimensionality 维数灾难Cut point 截断点Cutting plane algorithm 割平⾯法Letter DData mining 数据挖掘Data set 数据集Decision Boundary 决策边界Decision stump 决策树桩Decision tree 决策树／判定树Deduction 演绎Deep Belief Network 深度信念⽹络Deep Convolutional Generative Adversarial Network/DCGAN 深度卷积⽣成对抗⽹络Deep learning 深度学习Deep neural network/DNN 深度神经⽹络Deep Q-Learning 深度 Q 学习Deep Q-Network 深度 Q ⽹络Density estimation 密度估计Density-based clustering 密度聚类Differentiable neural computer 可微分神经计算机Dimensionality reduction algorithm 降维算法Directed edge 有向边Disagreement measure 不合度量Discriminative model 判别模型Discriminator 判别器Distance measure 距离度量Distance metric learning 距离度量学习Distribution 分布Divergence 散度Diversity measure 多样性度量／差异性度量Domain adaption 领域⾃适应Downsampling 下采样D-separation （Directed separation）有向分离Dual problem 对偶问题Dummy node 哑结点Dynamic Fusion 动态融合Dynamic programming 动态规划Letter EEigenvalue decomposition 特征值分解Embedding 嵌⼊Emotional analysis 情绪分析Empirical conditional entropy 经验条件熵Empirical entropy 经验熵Empirical error 经验误差Empirical risk 经验风险End-to-End 端到端Energy-based model 基于能量的模型Ensemble learning 集成学习Ensemble pruning 集成修剪Error Correcting Output Codes／ECOC 纠错输出码Error rate 错误率Error-ambiguity decomposition 误差-分歧分解Euclidean distance 欧⽒距离Evolutionary computation 演化计算Expectation-Maximization 期望最⼤化Expected loss 期望损失Exploding Gradient Problem 梯度爆炸问题Exponential loss function 指数损失函数Extreme Learning Machine/ELM 超限学习机Letter FFactorization 因⼦分解False negative 假负类False positive 假正类False Positive Rate/FPR 假正例率Feature engineering 特征⼯程Feature selection 特征选择Feature vector 特征向量Featured Learning 特征学习Feedforward Neural Networks/FNN 前馈神经⽹络Fine-tuning 微调Flipping output 翻转法Fluctuation 震荡Forward stagewise algorithm 前向分步算法Frequentist 频率主义学派Full-rank matrix 满秩矩阵Functional neuron 功能神经元Letter GGain ratio 增益率Game theory 博弈论Gaussian kernel function ⾼斯核函数Gaussian Mixture Model ⾼斯混合模型General Problem Solving 通⽤问题求解Generalization 泛化Generalization error 泛化误差Generalization error bound 泛化误差上界Generalized Lagrange function ⼴义拉格朗⽇函数Generalized linear model ⼴义线性模型Generalized Rayleigh quotient ⼴义瑞利商Generative Adversarial Networks/GAN ⽣成对抗⽹络Generative Model ⽣成模型Generator ⽣成器Genetic Algorithm/GA 遗传算法Gibbs sampling 吉布斯采样Gini index 基尼指数Global minimum 全局最⼩Global Optimization 全局优化Gradient boosting 梯度提升Gradient Descent 梯度下降Graph theory 图论Ground-truth 真相／真实Letter HHard margin 硬间隔Hard voting 硬投票Harmonic mean 调和平均Hesse matrix 海塞矩阵Hidden dynamic model 隐动态模型Hidden layer 隐藏层Hidden Markov Model/HMM 隐马尔可夫模型Hierarchical clustering 层次聚类Hilbert space 希尔伯特空间Hinge loss function 合页损失函数Hold-out 留出法Homogeneous 同质Hybrid computing 混合计算Hyperparameter 超参数Hypothesis 假设Hypothesis test 假设验证Letter IICML 国际机器学习会议Improved iterative scaling/IIS 改进的迭代尺度法Incremental learning 增量学习Independent and identically distributed/i.i.d. 独⽴同分布Independent Component Analysis/ICA 独⽴成分分析Indicator function 指⽰函数Individual learner 个体学习器Induction 归纳Inductive bias 归纳偏好Inductive learning 归纳学习Inductive Logic Programming／ILP 归纳逻辑程序设计Information entropy 信息熵Information gain 信息增益Input layer 输⼊层Insensitive loss 不敏感损失Inter-cluster similarity 簇间相似度International Conference for Machine Learning/ICML 国际机器学习⼤会Intra-cluster similarity 簇内相似度Intrinsic value 固有值Isometric Mapping/Isomap 等度量映射Isotonic regression 等分回归Iterative Dichotomiser 迭代⼆分器Letter KKernel method 核⽅法Kernel trick 核技巧Kernelized Linear Discriminant Analysis／KLDA 核线性判别分析K-fold cross validation k 折交叉验证／k 倍交叉验证K-Means Clustering K – 均值聚类K-Nearest Neighbours Algorithm/KNN K近邻算法Knowledge base 知识库Knowledge Representation 知识表征Letter LLabel space 标记空间Lagrange duality 拉格朗⽇对偶性Lagrange multiplier 拉格朗⽇乘⼦Laplace smoothing 拉普拉斯平滑Laplacian correction 拉普拉斯修正Latent Dirichlet Allocation 隐狄利克雷分布Latent semantic analysis 潜在语义分析Latent variable 隐变量Lazy learning 懒惰学习Learner 学习器Learning by analogy 类⽐学习Learning rate 学习率Learning Vector Quantization/LVQ 学习向量量化Least squares regression tree 最⼩⼆乘回归树Leave-One-Out/LOO 留⼀法linear chain conditional random field 线性链条件随机场Linear Discriminant Analysis／LDA 线性判别分析Linear model 线性模型Linear Regression 线性回归Link function 联系函数Local Markov property 局部马尔可夫性Local minimum 局部最⼩Log likelihood 对数似然Log odds／logit 对数⼏率Logistic Regression Logistic 回归Log-likelihood 对数似然Log-linear regression 对数线性回归Long-Short Term Memory/LSTM 长短期记忆Loss function 损失函数Letter MMachine translation/MT 机器翻译Macron-P 宏查准率Macron-R 宏查全率Majority voting 绝对多数投票法Manifold assumption 流形假设Manifold learning 流形学习Margin theory 间隔理论Marginal distribution 边际分布Marginal independence 边际独⽴性Marginalization 边际化Markov Chain Monte Carlo/MCMC 马尔可夫链蒙特卡罗⽅法Markov Random Field 马尔可夫随机场Maximal clique 最⼤团Maximum Likelihood Estimation/MLE 极⼤似然估计／极⼤似然法Maximum margin 最⼤间隔Maximum weighted spanning tree 最⼤带权⽣成树Max-Pooling 最⼤池化Mean squared error 均⽅误差Meta-learner 元学习器Metric learning 度量学习Micro-P 微查准率Micro-R 微查全率Minimal Description Length/MDL 最⼩描述长度Minimax game 极⼩极⼤博弈Misclassification cost 误分类成本Mixture of experts 混合专家Momentum 动量Moral graph 道德图／端正图Multi-class classification 多分类Multi-document summarization 多⽂档摘要Multi-layer feedforward neural networks 多层前馈神经⽹络Multilayer Perceptron/MLP 多层感知器Multimodal learning 多模态学习Multiple Dimensional Scaling 多维缩放Multiple linear regression 多元线性回归Multi-response Linear Regression ／MLR 多响应线性回归Mutual information 互信息Letter NNaive bayes 朴素贝叶斯Naive Bayes Classifier 朴素贝叶斯分类器Named entity recognition 命名实体识别Nash equilibrium 纳什均衡Natural language generation/NLG ⾃然语⾔⽣成Natural language processing ⾃然语⾔处理Negative class 负类Negative correlation 负相关法Negative Log Likelihood 负对数似然Neighbourhood Component Analysis/NCA 近邻成分分析Neural Machine Translation 神经机器翻译Neural Turing Machine 神经图灵机Newton method ⽜顿法NIPS 国际神经信息处理系统会议No Free Lunch Theorem／NFL 没有免费的午餐定理Noise-contrastive estimation 噪⾳对⽐估计Nominal attribute 列名属性Non-convex optimization ⾮凸优化Nonlinear model ⾮线性模型Non-metric distance ⾮度量距离Non-negative matrix factorization ⾮负矩阵分解Non-ordinal attribute ⽆序属性Non-Saturating Game ⾮饱和博弈Norm 范数Normalization 归⼀化Nuclear norm 核范数Numerical attribute 数值属性Letter OObjective function ⽬标函数Oblique decision tree 斜决策树Occam’s razor 奥卡姆剃⼑Odds ⼏率Off-Policy 离策略One shot learning ⼀次性学习One-Dependent Estimator／ODE 独依赖估计On-Policy 在策略Ordinal attribute 有序属性Out-of-bag estimate 包外估计Output layer 输出层Output smearing 输出调制法Overfitting 过拟合／过配Oversampling 过采样Letter PPaired t-test 成对 t 检验Pairwise 成对型Pairwise Markov property 成对马尔可夫性Parameter 参数Parameter estimation 参数估计Parameter tuning 调参Parse tree 解析树Particle Swarm Optimization/PSO 粒⼦群优化算法Part-of-speech tagging 词性标注Perceptron 感知机Performance measure 性能度量Plug and Play Generative Network 即插即⽤⽣成⽹络Plurality voting 相对多数投票法Polarity detection 极性检测Polynomial kernel function 多项式核函数Pooling 池化Positive class 正类Positive definite matrix 正定矩阵Post-hoc test 后续检验Post-pruning 后剪枝potential function 势函数Precision 查准率／准确率Prepruning 预剪枝Principal component analysis/PCA 主成分分析Principle of multiple explanations 多释原则Prior 先验Probability Graphical Model 概率图模型Proximal Gradient Descent/PGD 近端梯度下降Pruning 剪枝Pseudo-label 伪标记Letter QQuantized Neural Network 量⼦化神经⽹络Quantum computer 量⼦计算机Quantum Computing 量⼦计算Quasi Newton method 拟⽜顿法Letter RRadial Basis Function／RBF 径向基函数Random Forest Algorithm 随机森林算法Random walk 随机漫步Recall 查全率／召回率Receiver Operating Characteristic/ROC 受试者⼯作特征Rectified Linear Unit/ReLU 线性修正单元Recurrent Neural Network 循环神经⽹络Recursive neural network 递归神经⽹络Reference model 参考模型Regression 回归Regularization 正则化Reinforcement learning/RL 强化学习Representation learning 表征学习Representer theorem 表⽰定理reproducing kernel Hilbert space/RKHS 再⽣核希尔伯特空间Re-sampling 重采样法Rescaling 再缩放Residual Mapping 残差映射Residual Network 残差⽹络Restricted Boltzmann Machine/RBM 受限玻尔兹曼机Restricted Isometry Property/RIP 限定等距性Re-weighting 重赋权法Robustness 稳健性/鲁棒性Root node 根结点Rule Engine 规则引擎Rule learning 规则学习Letter SSaddle point 鞍点Sample space 样本空间Sampling 采样Score function 评分函数Self-Driving ⾃动驾驶Self-Organizing Map／SOM ⾃组织映射Semi-naive Bayes classifiers 半朴素贝叶斯分类器Semi-Supervised Learning 半监督学习semi-Supervised Support Vector Machine 半监督⽀持向量机Sentiment analysis 情感分析Separating hyperplane 分离超平⾯Sigmoid function Sigmoid 函数Similarity measure 相似度度量Simulated annealing 模拟退⽕Simultaneous localization and mapping 同步定位与地图构建Singular Value Decomposition 奇异值分解Slack variables 松弛变量Smoothing 平滑Soft margin 软间隔Soft margin maximization 软间隔最⼤化Soft voting 软投票Sparse representation 稀疏表征Sparsity 稀疏性Specialization 特化Spectral Clustering 谱聚类Speech Recognition 语⾳识别Splitting variable 切分变量Squashing function 挤压函数Stability-plasticity dilemma 可塑性-稳定性困境Statistical learning 统计学习Status feature function 状态特征函Stochastic gradient descent 随机梯度下降Stratified sampling 分层采样Structural risk 结构风险Structural risk minimization/SRM 结构风险最⼩化Subspace ⼦空间Supervised learning 监督学习／有导师学习support vector expansion ⽀持向量展式Support Vector Machine/SVM ⽀持向量机Surrogat loss 替代损失Surrogate function 替代函数Symbolic learning 符号学习Symbolism 符号主义Synset 同义词集Letter TT-Distribution Stochastic Neighbour Embedding/t-SNE T – 分布随机近邻嵌⼊Tensor 张量Tensor Processing Units/TPU 张量处理单元The least square method 最⼩⼆乘法Threshold 阈值Threshold logic unit 阈值逻辑单元Threshold-moving 阈值移动Time Step 时间步骤Tokenization 标记化Training error 训练误差Training instance 训练⽰例／训练例Transductive learning 直推学习Transfer learning 迁移学习Treebank 树库Tria-by-error 试错法True negative 真负类True positive 真正类True Positive Rate/TPR 真正例率Turing Machine 图灵机Twice-learning ⼆次学习Letter UUnderfitting ⽋拟合／⽋配Undersampling ⽋采样Understandability 可理解性Unequal cost ⾮均等代价Unit-step function 单位阶跃函数Univariate decision tree 单变量决策树Unsupervised learning ⽆监督学习／⽆导师学习Unsupervised layer-wise training ⽆监督逐层训练Upsampling 上采样Letter VVanishing Gradient Problem 梯度消失问题Variational inference 变分推断VC Theory VC维理论Version space 版本空间Viterbi algorithm 维特⽐算法Von Neumann architecture 冯 · 诺伊曼架构Letter WWasserstein GAN/WGAN Wasserstein⽣成对抗⽹络Weak learner 弱学习器Weight 权重Weight sharing 权共享Weighted voting 加权投票法Within-class scatter matrix 类内散度矩阵Word embedding 词嵌⼊Word sense disambiguation 词义消歧Letter ZZero-data learning 零数据学习Zero-shot learning 零次学习Aapproximations近似值arbitrary随意的affine仿射的arbitrary任意的amino acid氨基酸amenable经得起检验的axiom公理，原则abstract提取architecture架构，体系结构；建造业absolute绝对的arsenal军⽕库assignment分配algebra线性代数asymptotically⽆症状的appropriate恰当的Bbias偏差brevity简短，简洁；短暂broader⼴泛briefly简短的batch批量Cconvergence 收敛，集中到⼀点convex凸的contours轮廓constraint约束constant常理commercial商务的complementarity补充coordinate ascent同等级上升clipping剪下物；剪报；修剪component分量；部件continuous连续的covariance协⽅差canonical正规的，正则的concave⾮凸的corresponds相符合；相当；通信corollary推论concrete具体的事物，实在的东西cross validation交叉验证correlation相互关系convention约定cluster⼀簇centroids 质⼼，形⼼converge收敛computationally计算(机)的calculus计算Dderive获得，取得dual⼆元的duality⼆元性；⼆象性；对偶性derivation求导；得到；起源denote预⽰，表⽰，是…的标志；意味着，[逻]指称divergence 散度；发散性dimension尺度，规格；维数dot⼩圆点distortion变形density概率密度函数discrete离散的discriminative有识别能⼒的diagonal对⾓dispersion分散，散开determinant决定因素disjoint不相交的Eencounter遇到ellipses椭圆equality等式extra额外的empirical经验；观察ennmerate例举，计数exceed超过，越出expectation期望efficient⽣效的endow赋予explicitly清楚的exponential family指数家族equivalently等价的Ffeasible可⾏的forary初次尝试finite有限的，限定的forgo摒弃，放弃fliter过滤frequentist最常发⽣的forward search前向式搜索formalize使定形Ggeneralized归纳的generalization概括，归纳；普遍化；判断（根据不⾜）guarantee保证；抵押品generate形成，产⽣geometric margins⼏何边界gap裂⼝generative⽣产的；有⽣产⼒的Hheuristic启发式的；启发法；启发程序hone怀恋；磨hyperplane超平⾯Linitial最初的implement执⾏intuitive凭直觉获知的incremental增加的intercept截距intuitious直觉instantiation例⼦indicator指⽰物，指⽰器interative重复的，迭代的integral积分identical相等的；完全相同的indicate表⽰，指出invariance不变性，恒定性impose把…强加于intermediate中间的interpretation解释，翻译Jjoint distribution联合概率Llieu替代logarithmic对数的，⽤对数表⽰的latent潜在的Leave-one-out cross validation留⼀法交叉验证Mmagnitude巨⼤mapping绘图，制图；映射matrix矩阵mutual相互的，共同的monotonically单调的minor较⼩的，次要的multinomial多项的multi-class classification⼆分类问题Nnasty讨厌的notation标志，注释naïve朴素的Oobtain得到oscillate摆动optimization problem最优化问题objective function⽬标函数optimal最理想的orthogonal(⽮量，矩阵等)正交的orientation⽅向ordinary普通的occasionally偶然的Ppartial derivative偏导数property性质proportional成⽐例的primal原始的，最初的permit允许pseudocode伪代码permissible可允许的polynomial多项式preliminary预备precision精度perturbation 不安，扰乱poist假定，设想positive semi-definite半正定的parentheses圆括号posterior probability后验概率plementarity补充pictorially图像的parameterize确定…的参数poisson distribution柏松分布pertinent相关的Qquadratic⼆次的quantity量，数量；分量query疑问的Rregularization使系统化；调整reoptimize重新优化restrict限制；限定；约束reminiscent回忆往事的；提醒的；使⼈联想…的（of）remark注意random variable随机变量respect考虑respectively各⾃的；分别的redundant过多的；冗余的Ssusceptible敏感的stochastic可能的；随机的symmetric对称的sophisticated复杂的spurious假的；伪造的subtract减去；减法器simultaneously同时发⽣地；同步地suffice满⾜scarce稀有的，难得的split分解，分离subset⼦集statistic统计量successive iteratious连续的迭代scale标度sort of有⼏分的squares平⽅Ttrajectory轨迹temporarily暂时的terminology专⽤名词tolerance容忍；公差thumb翻阅threshold阈，临界theorem定理tangent正弦Uunit-length vector单位向量Vvalid有效的，正确的variance⽅差variable变量；变元vocabulary词汇valued经估价的；宝贵的Wwrapper包装分类:。