Kernel Independent Fast Multipole Boundary Element Method and Its Applications in Engineering

Fisher KernelMartin Sewell2007Jaakkola and Haussler(1999a)introduced the Fisher kernel(named in hon-our of Sir Ronald Fisher),thus creating a generic mechanism for incorporat-ing generative probability models into discriminative classifiers such as SVMs. Jaakkola and Haussler(1999b)introduced a generic class of probabilistic re-gression models and a parameter estimation technique that can make use of arbitrary kernel functions.Jaakkola,Diekhans and Haussler(1999)applied the Fisher kernel method to detecting remote protein homologies which performed well in classifying protein domains by SCOP superfamily.Jaakkola,Diekhans and Haussler(2000)found that using the Fisher kernel significantly improved on previous methods for the classification of protein domains based on remote homologies.Moreno and Rifkin(2000)used the Fisher kernel method for large scale Web audio classification.Mika,Smola and Sch¨o lkopf(2001)presented a fast training algorithm for the kernel Fisher discriminant classifier.It improved upon the state of the art by more than an order of magnitude,thus rendering the kernel Fisher algorithm a viable option also for large datasets.Vinokourov and Girolami(2001)successfully employed the Fisher kernel for document classifi-cation.Saunders,Shawe-Taylor and Vinokourov(2003)showed how the string kernel can be thought of as a k-stage Markov process,and as a result interpreted as a Fisher kernel.Tsuda,et al.(2004)analyzed the statistical properties of the Fisher kernel.Nicotra,Micheli and Starita(2004)extended the Fisher kernel to deal with tree structured data.Kersting and G¨a rtner(2004)extend the Fisher kernel to logical sequences(sequences over an alphabet of logical atoms).Their experiments showed a considerable improvement over results achieved without Fisher kernels for logical sequences.Holub,Welling and Perona(2005)suc-cessfully combined generative models with Fisher kernels to realize performance gains on standard object recognition data-sets.The log-likelihood of a data item x with respect to the model m(θ0)for a given setting of the parametersθ0is defined to belog Lθ0(x).Consider the vector gradient of the log-likelihoodg(θ,x)=∂log Lθ∂θiNi=1.1The Fisher score of a data item x with respect to the model m(θ0)for a given setting of the parametersθ0isg(θ0,x).The Fisher information matrix with respect to the model m(θ0)for a given setting of the parametersθ0is given byI M=E[g(θ0,x)g(θ0,x) ],where the expectation is over the generation of the data point x according to the data generating distribution.The Fisher score gives us an embedding into the feature space R N and hence immediately suggests a possible kernel.The matrix I M can be used to define a non-standard inner product in that feature space.The invariant Fisher kernel with respect to the model m(θ0)for a given setting of the parametersθ0is defined asg(θ0,z).κ(x,z)=g(θ0,x) I−1MThe practical Fisher kernel is defined asκ(x,z)=g(θ0) g(θ0,z).The Fisher kernel gives a“natural”similarity measure that takes into ac-count an underlying probability distribution.It seems natural to compare two data points through the directions in which they‘stretch’the parameters of the model,that is by viewing the score function at the two points as a function of the parameters and comparing the two gradients.If the gradient vectors are similar it means that the two data items would adapt the model in the same way,that is from the point of view of the given parametric model at the current parameter setting they are similar in the sense that they would require similar adaptations to the parameters.ReferencesHOLUB,Alex D.,Max WELLING,and Pietro PERONA,bining generative models andfisher kernels for object recognition.In:Proceedings of the Tenth IEEE International Conference on Computer Vision(ICCV’05), Volume1.Washington,DC,USA:IEEE,pp.136–143.JAAKKOLA,Tommi,Mark DIEKHANS,and David HAUSSLER,ing the Fisher kernel method to detect remote protein homologies.In:Thomas LENGAUER,et al.,eds.Proceedings of the Seventh International Conference on Intelligent Systems for Molecular Biology.Menlo Park,CA:AAAI Press, pp.149–158.JAAKKOLA,Tommi,Mark DIEKHANS,and David HAUSSLER,2000.A discriminative framework for detecting remote protein homologies.Journal of Computational Biology,7(1–2),95–114.2JAAKKOLA,Tommi S.,and David HAUSSLER,1999a.Exploiting genera-tive models in discriminative classifiers.In:Michael S.KEARNS,Sara A. SOLLA,and David A.COHN,eds.Advances in Neural Information Pro-cessing Systems11,Bradford Books.Cambridge,MA,USA:The MIT Press, pp.487–493.JAAKKOLA,Tommi S.,and David HAUSSLER,1999b.Probabilistic kernel regression models.In:David HECKERMAN and Joe WHITTAKER,eds. Proceedings of the1999Conference on AI and Statistics.San Mateo,CA: Morgan Kaufmann.KERSTING,Kristian,and Thomas G¨ARTNER,2004.Fisher kernels for logical sequences.In:Jean-Fran¸c ois BOULICAUT,et al.,eds.Machine Learning: ECML2004:15th European Conference on Machine Learning,Pisa,Italy, September2004,Proceedings.Berlin/Heidelberg:Springer,pp.205–216. MIKA,Sebastian,Alexander SMOLA,and Bernhard SCH¨OLKOPF,2001.An improved training algorithm for kernelfisher discriminants.In:Tommi JAAKKOLA and Thomas RICHARDSON,eds.Artificial Intelligence and Statistics2001:Proceedings of the Eighth International Workshop:January 4-7,2001Key West,Florida.San Francisco,CA:Morgan Kaufmann,pp.98–104.MORENO,Pedro J.,and Ryan RIFKIN,ing the Fisher kernel method for Web audio classification.In:2000IEEE International Conference on Acoustics,Speech,and Signal Processing:Proceedings,Volume IV.IEEE, pp.2417–2420.NICOTRA,Luca,Alessio MICHELI,and Antonina STARITA,2004.Fisher kernel for tree structured data.In:Proceedings:2004IEEE International Joint Conference on Neural Networks,Volume3.IEEE,pp.1917–1922. SAUNDERS,Craig,John SHAWE-TAYLOR,and Alexei VINOKOUROV, 2003.String kernels,fisher kernels andfinite state automata.In:Suzanna BECKER,Sebastian THRUN,and Klaus OBERMAYER,eds.Advances in Neural Information Processing Systems15,Bradford Books.Cambridge,MA: The MIT Press,pp.633–640.TSUDA,Koji,et al.,2004.Asymptotic properties of the Fisher kernel.Neural Computation,16(1),115–137.VINOKOUROV,Alexei,and Mark GIROLAMI,2001.Document classifica-tion employing the Fisher kernel derived from probabilistic hierarchic corpus representations.In:Proceedings of ECIR-01,23rd European Colloquium on Information Retrieval Research.British Computer Society Information Re-trieval Specialist Group.Berlin:Springer Verlag,pp.24–40.3。

Package‘fastICA’November27,2023Version1.2-4Date2023-11-27Title FastICA Algorithms to Perform ICA and Projection PursuitAuthor J L Marchini,C Heaton and B D Ripley<***************>Maintainer Brian Ripley<***************>Depends R(>=4.0.0)Suggests MASSDescription Implementation of FastICA algorithm to perform IndependentComponent Analysis(ICA)and Projection Pursuit.License GPL-2|GPL-3NeedsCompilation yesRepository CRANDate/Publication2023-11-2708:34:50UTCR topics documented:fastICA (1)ica.R.def (5)ica.R.par (6)Index7 fastICA FastICA algorithmDescriptionThis is an R and C code implementation of the FastICA algorithm of Aapo Hyvarinen et al.(https: //www.cs.helsinki.fi/u/ahyvarin/)to perform Independent Component Analysis(ICA)and Projection Pursuit.1UsagefastICA(X,p,alg.typ=c("parallel","deflation"),fun=c("logcosh","exp"),alpha=1.0,method=c("R","C"),row.norm=FALSE,maxit=200,tol=1e-04,verbose=FALSE,w.init=NULL)ArgumentsX a data matrix with n rows representing observations and p columns representing variables.p number of components to be extractedalg.typ if alg.typ=="parallel"the components are extracted simultaneously(the default).if alg.typ=="deflation"the components are extracted one at atime.fun the functional form of the G function used in the approximation to neg-entropy (see‘details’).alpha constant in range[1,2]used in approximation to neg-entropy when fun== "logcosh"method if method=="R"then computations are done exclusively in R(default).The code allows the interested R user to see exactly what the algorithm does.ifmethod=="C"then C code is used to perform most of the computations,whichmakes the algorithm run faster.During compilation the C code is linked to anoptimized BLAS library if present,otherwise stand-alone BLAS routines arecompiled.row.norm a logical value indicating whether rows of the data matrix X should be standard-ized beforehand.maxit maximum number of iterations to perform.tol a positive scalar giving the tolerance at which the un-mixing matrix is considered to have converged.verbose a logical value indicating the level of output as the algorithm runs.w.init Initial un-mixing matrix of dimension c(p,p).If NULL(default) then a matrix of normal r.v.’s is used.DetailsIndependent Component Analysis(ICA)The data matrix X is considered to be a linear combination of non-Gaussian(independent)compo-nents i.e.X=SA where columns of S contain the independent components and A is a linear mixing matrix.In short ICA attempts to‘un-mix’the data by estimating an un-mixing matrix W where XW =S.Under this generative model the measured‘signals’in X will tend to be‘more Gaussian’than the source components(in S)due to the Central Limit Theorem.Thus,in order to extract the independent components/sources we search for an un-mixing matrix W that maximizes the non-gaussianity of the sources.In FastICA,non-gaussianity is measured using approximations to neg-entropy(J)which are more robust than kurtosis-based measures and fast to compute.The approximation takes the formJ(y)=[E{G(y)}−E{G(v)}]2where v is a N(0,1)r.v.log cosh(αu)and G(u)=−exp(u2/2).The following choices of G are included as options G(u)=1αAlgorithmFirst,the data are centered by subtracting the mean of each column of the data matrix X.The data matrix is then‘whitened’by projecting the data onto its principal component directionsi.e.X->XK where K is a pre-whitening matrix.The number of components can be specified bythe user.The ICA algorithm then estimates a matrix W s.t XKW=S.W is chosen to maximize the neg-entropy approximation under the constraints that W is an orthonormal matrix.This constraint en-sures that the estimated components are uncorrelated.The algorithm is based on afixed-point iteration scheme for maximizing the neg-entropy.Projection PursuitIn the absence of a generative model for the data the algorithm can be used tofind the projection pursuit directions.Projection pursuit is a technique forfinding‘interesting’directions in multi-dimensional datasets.These projections and are useful for visualizing the dataset and in density estimation and regression.Interesting directions are those which show the least Gaussian distribu-tion,which is what the FastICA algorithm does.ValueA list containing the following componentsX pre-processed data matrixK pre-whitening matrix that projects data onto thefirst p principal compo-nents.W estimated un-mixing matrix(see definition in details)A estimated mixing matrixS estimated source matrixAuthor(s)J L Marchini and C HeatonReferencesA.Hyvarinen and E.Oja(2000)Independent Component Analysis:Algorithms and Applications,Neural Networks,13(4-5):411-430See Alsoica.R.def,ica.R.parExamples#---------------------------------------------------#Example1:un-mixing two mixed independent uniforms#---------------------------------------------------S<-matrix(runif(10000),5000,2)A<-matrix(c(1,1,-1,3),2,2,byrow=TRUE)X<-S%*%Aa<-fastICA(X,2,alg.typ="parallel",fun="logcosh",alpha=1,method="C",row.norm=FALSE,maxit=200,tol=0.0001,verbose=TRUE)par(mfrow=c(1,3))plot(a$X,main="Pre-processed data")plot(a$X%*%a$K,main="PCA components")plot(a$S,main="ICA components")#--------------------------------------------#Example2:un-mixing two independent signals#--------------------------------------------S<-cbind(sin((1:1000)/20),rep((((1:200)-100)/100),5))A<-matrix(c(0.291,0.6557,-0.5439,0.5572),2,2)X<-S%*%Aa<-fastICA(X,2,alg.typ="parallel",fun="logcosh",alpha=1,method="R",row.norm=FALSE,maxit=200,tol=0.0001,verbose=TRUE)par(mfcol=c(2,3))plot(1:1000,S[,1],type="l",main="Original Signals",xlab="",ylab="")plot(1:1000,S[,2],type="l",xlab="",ylab="")plot(1:1000,X[,1],type="l",main="Mixed Signals",xlab="",ylab="")plot(1:1000,X[,2],type="l",xlab="",ylab="")plot(1:1000,a$S[,1],type="l",main="ICA source estimates",xlab="",ylab="")plot(1:1000,a$S[,2],type="l",xlab="",ylab="")#-----------------------------------------------------------#Example3:using FastICA to perform projection pursuit on a#mixture of bivariate normal distributions#-----------------------------------------------------------if(require(MASS)){x<-mvrnorm(n=1000,mu=c(0,0),Sigma=matrix(c(10,3,3,1),2,2)) x1<-mvrnorm(n=1000,mu=c(-1,2),Sigma=matrix(c(10,3,3,1),2,2)) X<-rbind(x,x1)a<-fastICA(X,2,alg.typ="deflation",fun="logcosh",alpha=1,ica.R.def5 method="R",row.norm=FALSE,maxit=200,tol=0.0001,verbose=TRUE)par(mfrow=c(1,3))plot(a$X,main="Pre-processed data")plot(a$X%*%a$K,main="PCA components")plot(a$S,main="ICA components")}ica.R.def R code for FastICA using a deflation schemeDescriptionR code for FastICA using a deflation scheme in which the components are estimated one by one.This function is called by the fastICA function.Usageica.R.def(X,p,tol,fun,alpha,maxit,verbose,w.init)ArgumentsX data matrixp number of components to be extractedtol a positive scalar giving the tolerance at which the un-mixing matrix is consideredto have converged.fun the functional form of the G function used in the approximation to negentropy.alpha constant in range[1,2]used in approximation to negentropy when fun=="logcosh"maxit maximum number of iterations to performverbose a logical value indicating the level of output as the algorithm runs.w.init Initial value of un-mixing matrix.DetailsSee the help on fastICA for details.ValueThe estimated un-mixing matrix W.Author(s)J L Marchini and C HeatonSee AlsofastICA,ica.R.par6ica.R.par ica.R.par R code for FastICA using a parallel schemeDescriptionR code for FastICA using a parallel scheme in which the components are estimated simultaneously.This function is called by the fastICA function.Usageica.R.par(X,p,tol,fun,alpha,maxit,verbose,w.init)ArgumentsX data matrix.p number of components to be extracted.tol a positive scalar giving the tolerance at which the un-mixing matrix is consideredto have converged.fun the functional form of the G function used in the approximation to negentropy.alpha constant in range[1,2]used in approximation to negentropy when fun=="logcosh".maxit maximum number of iterations to perform.verbose a logical value indicating the level of output as the algorithm runs.w.init Initial value of un-mixing matrix.DetailsSee the help on fastICA for details.ValueThe estimated un-mixing matrix W.Author(s)J L Marchini and C HeatonSee AlsofastICA,ica.R.defIndex∗multivariatefastICA,1∗utilitiesica.R.def,5ica.R.par,6fastICA,1,5,6ica.R.def,3,5,6ica.R.par,3,5,67。

《二维弹性快速多极边界元算法及截断误差分析》篇一一、引言随着计算科学技术的不断进步，有限元法已经成为一种解决各种工程问题的重要工具。

边界元法是其中一种有效的技术，特别是针对复杂的边界问题，如弹性力学问题。

在本文中，我们将介绍一种高效的二维弹性快速多极边界元算法，并对其截断误差进行分析。

二、二维弹性快速多极边界元算法2.1 算法基础多极边界元算法（Multiple Boundary Element Method）的基本原理在于其采用边界的多个多项式作为逼近基底来对复杂的区域进行逼近和解析。

而在弹性力学中，这一原理则应用在力场和位移场的求解上。

在二维空间中，我们可以通过定义边界的节点和相应的多项式来描述物体的形状和状态。

2.2 快速多极算法快速多极算法是一种高效的计算方法，它通过将计算过程分解为多个子过程，并利用并行计算技术来提高计算效率。

在二维弹性快速多极边界元算法中，我们通过构建一个层次化的计算结构，使得每个子过程都能独立地并行进行，从而大大提高了计算速度。

三、截断误差分析在数学计算中，我们往往只能获得近似的结果，因为各种因素的影响会导致我们无法得到完全精确的结果。

这就产生了截断误差。

对于二维弹性快速多极边界元算法来说，截断误差主要来源于以下几个方面：3.1 逼近基底的选择我们采用的多项式逼近基底只是一种近似逼近方法，并不能完全精确地描述所有物体的形状和状态。

因此，我们选择的逼近基底的数量和类型都会对结果产生一定的影响，从而产生截断误差。

3.2 计算过程中的舍入误差由于计算机的精度限制，计算过程中可能无法得到完全精确的解。

例如，对于非常小的数或者需要进行多次乘除法操作的数，可能会产生舍入误差。

这些舍入误差会在计算过程中累积，最终对结果产生影响。

3.3 截断的边界条件在处理实际问题时，我们往往需要对问题进行截断处理。

例如，在处理无限大的区域时，我们只能选择一个有限大的区域进行计算。

这种截断处理会导致一些边界条件无法得到满足，从而产生截断误差。

Kernel Sparse Representation-BasedClassifier一、问题：1、针对不同类别的样本相互融合的分布情况，或者不能用线性的方法将它们有效分开的一般分类问题，SRC 失去了分类能力，如何选择一个有效的方法实现有效分类；2、KSR 虽然是SRC 的非线性扩展，但是它不能使用用于稀疏信号重构的方法，并且在实验中，测试时间较长，如何缩短测试时间；3、在KSRC 中，我们需要选择一个参数内核，例如一个RBF 内核，则必须选择有效的方法来确定相应的参数，使得效果优于SRC 的分类效果。

二、解决方法：为了解决以上的问题，在SRC 的基础上，文章引入了核函数。

核函数定义为： 1212(,)()()T k x x x x φφ=。

最常用的核函数分别有：高斯核径向基函数212122(,)()k x x exp x x γ=--，其中0γ>，还有线性核函数：1212(,)T k x x x x =等。

文章定义一个训练数据集：()(){}{},|,1,2,...,,1,2,...,m i i i i x y x X R y c i n ∈⊆∈=，其中，c 是类别的数目，m 是数据输入空间X 的维数，i y 是和i x 相对应的类标。

给定一个测试数据集x X ∈，我们的目标是从给定的c 类训练样本中预测出它的实际类标y 。

现在定义第j 类训练样本作为矩阵,1,[,...,],1,...,j j m n j j j n X x x R j c ⨯=∈=的各个列，其中，,j i x 被定义为第j 类样本，j n 是第j 类训练样本的数目。

下面定义新的训练样本矩阵来表示所有的样本数据：12c [,,...,]m n X X X X R ⨯=∈其中，1c j j n n ==∑.根据映射，将输入空间X （低维空间）的数据映射到核特征空间F （高维空间）中，有：12:()[(),(),...,()]T D x X x x x x F φφφφφ∈→=∈，其中，D m 是核特征空间F 的维度。