当前位置：文档之家› Multiview Spectral Embedding

Multiview Spectral Embedding

Tian Xia,Dacheng Tao,Member,IEEE,Tao Mei,Member,IEEE,and Yongdong Zhang,Member,IEEE

Abstract—In computer vision and multimedia search,it is com-mon to use multiple features from different views to represent an object.For example,to well characterize a natural scene image,it is essential to?nd a set of visual features to represent its color, texture,and shape information and encode each feature into a vector.Therefore,we have a set of vectors in different spaces to represent the image.Conventional spectral-embedding algorithms cannot deal with such datum directly,so we have to concatenate these vectors together as a new vector.This concatenation is not physically meaningful because each feature has a speci?c statis-tical property.Therefore,we develop a new spectral-embedding algorithm,namely,multiview spectral embedding(MSE),which can encode different features in different ways,to achieve a physically meaningful embedding.In particular,MSE?nds a low-dimensional embedding wherein the distribution of each view is suf?ciently smooth,and MSE explores the complementary prop-erty of different views.Because there is no closed-form solution for MSE,we derive an alternating optimization-based iterative algorithm to obtain the low-dimensional embedding.Empirical evaluations based on the applications of image retrieval,video an-notation,and document clustering demonstrate the effectiveness of the proposed approach.

Index Terms—Dimensionality reduction,multiple views, spectral embedding.

I.I NTRODUCTION

I N COMPUTER vision and multimedia search[5],[6],

objects are usually represented in several different ways. This kind of data is termed as the multiview data.A typical example is a color image,which has different views from dif-ferent modalities,e.g.,color,texture,and shape.Different views form different feature spaces,which have particular statistical properties.

Manuscript received May14,2009;revised August31,2009and November 18,2009;accepted December6,2009.Date of publication February17,2010; date of current version November17,2010.This work was supported in part by the National Basic Research Program of China(973Program)under Grant 2007CB311100;by the National High-Technology Research and Development Program of China(863Program)under Grant2007AA01Z416;by the National Natural Science Foundation of China under Grants60873165,60802028,and 60902090;by the Beijing New Star Project on Science and Technology under Grant2007B071;by the Co-building Program of Beijing Municipal Education Commission;by the Nanyang Technological University Nanyang SUG Grant under Project M58020010;by the Microsoft Operations PTE LTD-NTU Joint R&D under Grant M48020065;and by the K.C.Wong Education Foundation Award.This paper was recommended by Associate Editor S.Sarkar.

T.Xia and Y.Zhang are with the Center for Advanced Computing Tech-nology Research,Institute of Computing Technology,Chinese Academy of Sciences,Beijing100190,China(e-mail:txia@https://www.doczj.com/doc/bd10424597.html,;zhyd@https://www.doczj.com/doc/bd10424597.html,). D.Tao is with the School of Computer Engineering,Nanyang Technological University,Singapore639798(e-mail:dctao@https://www.doczj.com/doc/bd10424597.html,.sg).

T.Mei is with Microsoft Research Asia,Beijing100190,China(e-mail: tmei@https://www.doczj.com/doc/bd10424597.html,).

Color versions of one or more of the?gures in this paper are available online at https://www.doczj.com/doc/bd10424597.html,.

Digital Object Identi?er10.1109/TSMCB.2009.2039566

Because of the popularity of multiview data in practical applications,particularly in the multimedia domain,learning from multiview data,which is also known as multiple-view learning,has attracted more and more attentions.Although a great deal of efforts have been carried out on multiview data learning[1],including classi?cation[21],clustering[4],[19], and feature selection[20],little progress has been made in dimensionality reduction,whereas it has many applications in multimedia[28],e.g.,image retrieval and video annotation. Multimedia data generally have multiple modalities,and each modality is usually represented in a high-dimensional feature space which frequently leads to the“curse of dimensional-ity”problem.In this case,multiview dimensionality reduction provides an effective solution to solve or at least reduce this problem.

In this paper,we consider the problem of spectral em-bedding for multiple-view data based on our previous patch alignment framework[29].The major challenge is learning a low-dimensional embedding to effectively explore the comple-mentary nature of multiple views of a data set.The learned low-dimensional embedding should be better than a low-dimensional embedding learned by each single view of the data set.

Existing spectral-embedding algorithms assume that sam-ples are drawn from a vector space and thus cannot deal with multiview data directly.A possible solution is to con-catenate vectors from different views together as a new vec-tor and then apply spectral-embedding algorithms directly on the concatenated vector.However,this concatenation is not physically meaningful because each view has a speci?c sta-tistical property.This concatenation ignores the diversity of multiple views and thus cannot ef?ciently explore the com-plementary nature of different views.Another solution is the distributed spectral embedding(DSE)proposed in[3].DSE performs a spectral-embedding algorithm on each view in-dependently,and then based on the learned low-dimensional representations,it learns a common low-dimensional embed-ding which is“close”to each representation as much as possible.Although DSE allows selecting different spectral-embedding algorithms for different views,the original multiple-view data are invisible to the?nal learning process,and thus,it cannot well explore the complementary nature of dif-ferent views.Moreover,its computational cost is dense be-cause it conducts spectral-embedding algorithms for each view independently.

To effectively and ef?ciently learn the complementary nature of different views,we propose a new algorithm,i.e.,multiview spectral embedding(MSE),which learns a low-dimensional and suf?ciently smooth embedding over all views simultane-ously.Empirical evaluations based on image retrieval,video

1083-4419/$26.00?2010IEEE

annotation,and document clustering show the effectiveness of the proposed approach.

The rest of this paper is organized as follows.In Section II, we provide a short review on related works.In Section III, we present the proposed MSE and the solution of MSE.Ex-perimental results are shown in Section IV,and Section V concludes.

II.R ELATED W ORKS

In this section,?rst,we provide a short review on con-ventional spectral-embedding algorithms,which are all for single-view data.Although there are some previous works on multiview spectral methods,they are about multiview learning for clustering[4]and classi?cation[21]but not for dimension-ality reduction.As for spectral embedding for multiple-view data,only some preliminary effort[3]is known to us;thus, second,we give a brief introduction to the distributed method proposed in[3].

A.Spectral Embedding

The task of dimensionality reduction is to?nd a low-dimensional representation for high-dimensional observations. It generally falls into two classes:linear methods,e.g.,prin-ciple component analysis and multidimensional scaling;and nonlinear methods,e.g.,locally linear embedding(LLE)[8]and Laplacian eigenmaps(LE)[9].

Spectral methods for dimensionality reduction?nd the low-dimensional representations by using eigenvectors of specially constructed matrices[29].Since,in the traditional problem set-ting of dimensionality reduction,it is assumed that the data are represented in a single vector space,the conventional spectral-embedding algorithms can all be regarded as methods with a single view.

Existing algorithms can be classi?ed into two groups based on whether they are supervised or unsupervised.The focus of this paper is the latter.Representative algorithms include Isomap[7],LLE[8],LE[9],Hessian eigenmaps[10],local tangent space alignment[11],transductive component analysis [2],discriminative locality alignment[30],and DLLE[27]. They perform well for single-view data but cannot deal with multiview data directly.

B.Distributed Approach for Spectral Embedding With Multiple Views

As mentioned earlier,spectral embedding with multiple views is a new topic;it is?rst proposed in[3],and a distributed approach,i.e.,DSE,is proposed in it.In the following is a brief summary of DSE.

Given a multiple-view datum with n objects having m views, i.e.,a set of matrices X={X(i)∈ m i×n}m i=1,each represen-tation X(i)is a feature matrix from view i.DSE assumes that the low-dimensional embedding of each view X(i)is already known,i.e.,A={A(i)∈ n×k i}m i=1,k i

min

B,P

i=1

A(i)?BP(i)

2s.t.B T B=I(1)

where P={P(i)∈ k×k i}m i=1is a set of mapping ma-trices.The global optimal solution to DSE is given by performing eigendecomposition of the matrix CC T,C= [A(1),...,A(m)].

III.MSE

In this section,we introduce a new spectral-embedding algo-rithm,i.e.,MSE,which?nds a low-dimensional and suf?ciently smooth embedding over all views simultaneously.To better present the technique details of the proposed MSE,we provide important notations used in the rest of this paper.Capital letters, e.g.,X,represent matrices or sets,and[X]ij is the(i,j)th entry of X.Lower case letters,e.g.,x,represent vectors,and(x)i is the i th element of x.Superscript(i),e.g.,X(i)and x(i), represents data from the i th view.

Based on the aforementioned notations,MSE can be de-scribed as follows according to our previous patch alignment framework[29].Given a multiview data set with n objects and m representations,i.e.,a set of matrices X={X(i)∈ m i×n}m i=1,wherein X(i)is the feature matrix for the i th view representation,MSE?nds a low-dimensional and suf?ciently smooth embedding of X,i.e.,Y∈ d×n,wherein d

Fig.1shows the working principle of MSE.MSE?rst builds a patch for a sample on a view.Based on the patches from different views,the part optimization can be performed to get the optimal low-dimensional embedding for each view.After-ward,all low-dimensional embeddings from different patches are uni?ed as a whole one by global coordinate alignment. Finally,the solution of MSE is derived by using the alternating optimization.

A.Part Optimization

Given the i th view X(i)=[x(i)1,...,x(i)n]∈ m i×n,con-sider an arbitrary point x(i)j and its k related ones in the same view(e.g.,nearest neighbors)x(i)j

,...,x(i)

j k

;the patch of x(i)j is de?ned as X(i)j=[x(i)j,x(i)j

,...,x(i)

j k

]∈ m i×(k+1).For X(i)j, there is a part mapping f(i)j:X(i)j→Y(i)j,wherein Y(i)j=

[y(i)

,y(i)

,...,y(i)

j k

]∈ d×(k+1).To preserve the locality in the projected low-dimensional space,the part optimization for the j th patch on the i th view is

arg min

Y(i)

l=1

y(i)j?y(i)j

w(i)

(2)

Fig.1.Working?ow of MSE:MSE?rst builds a patch for a sample on a view.Based on the patches from different views,the part optimization can be performed to get the optimal low-dimensional embedding for each view.Then,all low-dimensional embeddings from different patches are uni?ed together as a whole one by global coordinate alignment.Finally,the solution of MSE is derived by using the alternating optimization.

where w(i)j is a k-dimensional column vector weighted by

(w(i)

j )l=exp(? x(i)

?x(i)j

2/t).Therefore,(2)can be refor-

mulated to

arg min

Y(i)

j tr

y(i)

?y(i)j

...

y(i)

?y(i)j

y(i)

?y(i)j

,...,y(i)

?y(i)j

diag

w(i)

=arg min

Y(i)

j tr

Y(i)

?e T k

I k

diag

w(i)

×[?e k I k]

Y(i)

=arg min

Y(i)

j tr

Y(i)

L(i)

Y(i)

(3)

where e k=[1,...,1]T,I k is a k×k identity matrix,and tr(·) is the trace operator.L(i)j∈ (k+1)×(k+1)encodes the objective function for the j th patch on the i th view.According to(3),it is

L(i) j =

?e T k

I k

diag

w(i)

[?e k I k]

l=1

w(i)

?w(i)j diag

w(i)

??.(4)

Therefore,the part optimization for X(i)j is

arg min

Y(i)

j tr

Y(i)

L(i)

Y(i)

.(5)

Based on the locality information encoded in L(i)j,(5)?nds

a suf?ciently smooth low-dimensional embedding Y(i)

j by pre-

serving the intrinsic structure of the j th patch on the i th view.

Because of the complementary property of multiple views to each other,different views de?nitely have different contribu-tions to the?nal low-dimensional embedding.In order to well explore the complementary property of different views,a set of nonnegative weightsα=[α1,...,αm]is imposed on part optimizations of different views independently.The largerαi is,the more important role the view X(i)j plays in learning to obtain the low-dimensional embedding Y(i)

.By summing over all views,the multiview part optimization for the j th patch is

arg min

Y(i)

i=1

,α

i=1

αi tr

Y(i)

L(i)

Y(i)

.(6)

B.Global Coordinate Alignment

For each patch X(i)j,there is a low-dimensional embedding

Y(i)

.All Y(i)

can be uni?ed together as a whole one by

assuming that the coordinate for Y(i)

=[y(i)

,y(i)

,...,y(i)

j k

] is selected from the global coordinate Y=[y1,...,y n],i.e.,

Y(i)

=Y S(i)

,wherein S(i)j∈ n×(k+1)is the selection matrix to encode the spatial relationship of samples in a patch in the original high-dimensional space.That is,low-dimensional embeddings in different views are consistent with each other globally.Therefore,(6)can be equivalently rewritten as

arg min

Y,α

i=1

αi tr

Y S(i)

L(i)

S(i)

Y T

.(7)

By summing over all part optimizations de?ned by(7),the global coordinate alignment is given by

arg min

Y,α

j=1

i=1

αi tr

Y S(i)

L(i)

S(i)

Y T

=arg min

Y,α

i=1

αi tr

Y L(i)Y T

(8)

where L(i)∈ n×n is the alignment matrix for the i th view,and it is de?ned as

L(i)=

j=1

S(i)

L(i)

S(i)

.(9)

Putting (4)into (9),we have

L (i )=D (i )?W (i )

(10)

where W (i )∈ n ×n and [W (i )]pq =exp(? x (i )

p ?x (i )

q 2/t )

if x (i )p is among the k -nearest neighbors of x (i )

q or vice

versa;[W (i )]pq =0otherwise.In addition,D (i )is diagonal,and [D (i )]jj =

l [W (i )]jl .Therefore,L (i )is an unnormalized graph Laplacian matrix [12].In MSE,we adopt a normalized graph Laplacian matrix by performing a normalization on L (i )

L (i )n =

D (i ) ?1/2L

(i ) D (i )

?1/2=I ?

(i )

?1/2

(i )

?1/2

(11)

where L (i )

n is symmetric and positive semide?nite;the proof is

given in Appendix A.The constraint Y Y T =I is imposed on (8)to uniquely determine the low-dimensional embedding Y ,i.e.,

arg min

Y,α

i =1αi tr Y L (i )n Y T

s.t.Y Y T

=I ;

m i =1

αi =1,αi ≥0.(12)

The solution to αin (12)is αk =1corresponding to the

minimum tr(Y L (i )Y T )over different views,and αk =0other-wise.This solution means that only one view is ?nally selected by this method.Therefore,the performance of this method is equivalent to the one from the best view.This solution does not meet our objective on exploring the complementary property of multiple views to get a better embedding than based on a single view.

In this paper,we adopt a trick utilized in [13]to avoid

this phenomenon,i.e.,we set αi ←αr

with r >1.In this condition, m i =1αr

i achieves its minimum when αi =1/m with respect to

m i =1αi =1,αi >0.Similar αi for different views will be obtained by setting r >1,so each view has a particular contribution to the ?nal low-dimensional embedding Y .Therefore,the new objective function is de?ned as

arg min

Y,α

i =1αr i tr Y L (i )n Y T s .t .Y Y T

=I ;

m i =1

αi =1,αi ≥0(13)

where r >1.According to (13)and related discussions,MSE

?nds a low-dimensional suf?ciently smooth embedding Y by preserving the locality of each view simultaneously.The solu-tion of (13)is given in the next section.C.Alternating Optimization

In this section,we derive the solution of MSE de?ned in (13),which is a nonlinearly constrained nonconvex optimization problem.To the best of our knowledge,there is no direct way to ?nd its global optimal solution.In this paper,we derive an

iterative algorithm by using the alternating optimization [14]

to obtain a local optimal solution.The alternating optimization iteratively updates Y and αin an alternating fashion.First,we ?x Y to update α.By using a Lagrange multiplier λto take the constraint m i =1αi =1into consideration,we get the Lagrange function

L (α,λ)=m i =1

αr i tr Y L (i )n Y

?λ m

i =1

αi ?1 .(14)By setting the derivative of L (α,λ)with respect to αi and λ

to zero,we have ??????L (α,λ)?αi =rαr ?1i tr Y L (i )n Y

?λ=0,i =1,...,m ?L (α,λ)?λ=

m i =1

αi ?1=0.(15)

Therefore,αi can be obtained

αi = 1/tr

Y L (i )n Y

1/(r ?1)m i =1

1/tr Y L (i )n Y

1/(r ?1).(16)

The alignment matrix L (i )

n is positive semide?nite,so we

have αi ≥0naturally.When Y is ?xed,(16)gives the global optimal α.

According to (16),we have the following understanding for r in controlling αi .If r →∞,different αi will be close to each other.If r →1,only αi =1corresponding to the mini-mum tr(Y L (i )

n Y T )over different views,and αi =0otherwise.Therefore,the selection of r should be based on the comple-mentary property of all views.Rich complementary prefers large r ;otherwise,r should be small.The effect of parameter r is discussed in our experiments in Section IV-E.

Second,we ?x αto update Y .The optimization problem in (13)is equivalent to

min Y

tr(Y LY T )s .t Y Y T =I

(17)

where L = m i =1αr i L (i )

n .Because L (i )n is symmetric,L is symmetric.Based on the Ky-Fan theorem (the details of the Ky-Fan theorem are given in Appendix B),(17)has a global optimal solution Y when αis ?xed.The optimal Y is given as the eigenvectors associated with the smallest d eigenvalues of the matrix L .

According to the aforementioned descriptions,we can form an alternating optimization procedure,depicted in Algorithm 1,to obtain a local optimal solution of MSE.

Algorithm 1:MSE Algorithm

Input :A multiple-view datum X ={X (i )∈ m i ×n }m i =1,the dimension of the low-dimensional embedding d (d 1.

Output :A spectral embedding Y ∈ d ×n .Method :

1:Calculate L (i )

n for each view according to the patch alignment.

2:Initialize α=[1/m,...,1/m ].

3:Repeat

4:Y =U T ,where U =[u 1,...,u d ]are the eigenvectors associated with the smallest d eigenvalues of the matrix L de?ned in (17).5:αi =((1/tr(Y L (i )Y T ))1/(r ?1))/(

m i =1(1

/tr(Y L (i )Y T ))1/(r ?1))6:Until convergence The algorithm converges because the objective function m i =1αr i tr(Y L (i )n Y T

)reduces with the increasing of the iter-ation numbers.In particular,with ?xed α,the optimal Y can reduce the value of the objective function,and with ?xed Y ,the optimal αwill also reduce the value of the objective function.D.Time-Complexity Analysis

The time complexity of MSE contains two parts.One is for the constructions of alignment matrices for different views,i.e.,

the computation of L (i )

n .From (11),the time complexity of this part is O (( m

i =1m i )×n 2).The other is for the alternating optimization.The update of Y has the time complexity of the eigenvalue decomposition of an n ×n matrix.It is O (n 3).The update of αhas the time complexity of O ((m +d )×n 2).

Therefore,the entire time complexity of MSE is O ((n 3

+(m +d )×n 2)×T +( m

i =1m i )×n 2

),where T is the number of training iterations and T is around three (always less than ?ve)in all experiments.

IV .E XPERIMENTAL R ESULTS

Images and videos are usually represented by multiview features,and each view is represented in a high-dimensional space.In this paper,we compare the effectiveness of the proposed MSE with the conventional feature concatenation-based spectral embedding (CSE),the DSE [3],the average per-formance of the single-view-based spectral embedding (ASE),and the best performance of the single-view-based spectral embedding (BSE)in both image retrieval and video annotation.We also show the performance comparison in the application of document clustering,which shows the results for multiview data with poor complementary.Finally,we give our analysis on parameter r and the dimensionality of the low-dimensional embedding d .

In ASE,BSE,CSE,and DSE,the LE is adopted.The CSE is performed based on the Gaussian normalized concatenated vector from different views.For all experiments,the value of k for patch construction in MSE and that of k -nearest neighbor construction in LE are ?xed at 30.In LE and MSE,an unweighted graph as de?ned in [9]is adopted.A.Toy Data Set Test

In this section,we use a toy data set to illustrate the ef-fectiveness of MSE in comparing with CSE-LE and DSE-LE.The toy data set,which is a subset of Corel image gallery,consists of three semantic categories,i.e.,bus,ship,and train.Each category includes 100images.Fig.2shows some example images.For each image,we extract two kinds of low-level visual features,i.e.,a 64-D HSV color histogram

(HSVCH)

Fig.2.Example images in the toy data set.

and a 75-D edge directional histogram (EDH),to represent two different views.Because the two views for an image are generally complementary to each other,we empirically set r in MSE as ?ve.

Fig.3(a)and (b)shows the low-dimensional embeddings obtained by LE performed on HSVCH and EDH independently,and Fig.3(c)–(e)shows the embeddings obtained by CSE,DSE,and MSE,respectively.The results shown in Fig.3(a)–(d)demonstrate that the existing algorithms merge different cat-egories in the low-dimensional space.On the contrary,the proposed MSE can well separate different categories because MSE takes the complementary property of different views into consideration for embedding.B.Image Retrieval

In this section,we show the effectiveness of MSE in image retrieval.The procedure for performance evaluation is as fol-lows:1)The low-dimensional embedding of an image retrieval data set is learned by an embedding algorithm,e.g.,MSE,and 2)based on a low-dimensional embedding,a standard image retrieval procedure is conducted for all images in the data set.In detail,for each category,one image is selected as a query,and then,all the other images in the data set (including other categories)are ranked according to the Euclidean distance to the query computed in the low-dimensional embedding.The retrieval performance is evaluated through the average preci-sion (AP)based on the top N images.The mean AP (MAP)is computed by averaging all APs for different categories.In this paper,we use MAP to compare different embedding algorithms.

Corel 2000and Caltech256-2045are utilized independently for an image-retrieval test.Corel 2000is a subset of the Corel photo gallery.There are 20image categories,and each category contains 100images.These 20categories are the following:balloon,beach,bird,bobsled,bonsai,building,bus,butter?y,car,cat,cougar,dessert,dog,eagle,elephant,?rework,?tness,?ag,foliage,and fox.Fig.2shows some example images from Corel 2000.Caltech256-2045is a subset of the Caltech256data set [15];it contains 2045images from 15categories,including

Fig.3.Low-dimensional embeddings of different spectral-embedding algorithms.(a)LE on HSVCH.(b)LE on EDH.(c)CSE.(d)DSE.(e)

MSE.

Fig.4.Example images in Caltech256-2045.

AK-47,American?ag,backpack,baseball bat,baseball glove, baseball hoop,bat,bathtub,bear,bear mug,billiards,binocu-lars,birdbath,blimp,and bonsai.Fig.4shows some example images from Caltech256-2045.

For each image,we extract?ve kinds of low-level visual features to represent?ve different views.These?ve features are color moment,color correlogram,HSVCH,edge directional histogram,and wavelet texture.The color moment consists of5×5blockwise Lab color moments,and each block is represented by three statistical moments:mean,variance,and skewness over three color channels.Therefore,the color mo-ment is225-D.The color correlogram is extracted according to[16]and is114-D.The HSVCH contains64bins.A75-D edge directional histogram is extracted for edge representation. A128-D wavelet texture feature is extracted for texture rep-resentation.Table I summarizes the dimensionality for every modality.Because different views for an image are generally complementary to each other,we empirically set r in MSE as ?ve.The dimensionality of the low-dimensional embedding d is set as30.

The results for the performance comparison on Corel2000 and Caltech256-2045are shown in Fig.6.MSE achieves the best performance on both two data sets.

C.Video Annotation

In this section,we show the effectiveness of MSE in video annotation.We adopt a standard video annotation testbed,the TRECVID2008training set[17],which consists of39674 shots from20concepts.A key frame is extracted from each shot for representation.We select ten concepts for performance eval-uation.They are classroom,bridge,emergency vehicle,dog, airplane?ying,kitchen,bus,harbor,telephone,and demonstra-tion or pretest.We use all positive samples of the ten selected concepts to construct our data set,which has1179key frames. Fig.5shows some example key frames from TRECVID2008. Similar to the performance-evaluation procedure used in the im-age retrieval,MAP is applied to measure the performance of an embedding algorithm.Because different views for a key frame are generally complementary to each other,we empirically set r in MSE as?ve.The dimensionality of the low-dimensional embedding d is set as30.

For each key frame,we extract?ve kinds of low-level visual features,which are also used for image retrieval as shown in Table I,to represent?ve views.Fig.6shows that the proposed MSE achieves the best performance on this data set.

D.Document Clustering

In the aforementioned experiments,the data sets are multi-media data(e.g.,an image or a video)which generally have a complementary among different features;however,the com-plementary is not very rich since the different features of a sample are relatively related to each other.In this section,we consider a text data set to show the performance on multiview data with very rich complementary,in which the different views of a sample are nearly independent.We utilize the application of document clustering to perform evaluation.The procedure for evaluation is as follows:1)The low-dimensional embed-ding of a text data set is learned by an embedding algorithm,

D IMENSIONALITY OF F IV

F EATURES FOR I MAGE R

ETRIEVAL

Fig.5.Example images in TRECVID

2008.

Fig.6.Performance comparison on the three data sets measured by MAP of top 100samples.

e.g.,MSE,and 2)based on a low-dimensional embedding,a K -means algorithm [24]is performed to cluster the data set.As for cluster evaluation,following [25],we use the Rand index (RI)de?ned as

RI =

#CD N (N ?1)/2

(18)

where N denotes the number of samples,#CD denotes the number of correct decisions,and a decision is considered correct if the clustering algorithm agrees with the real clus-tering.In our experiments,K -means adopts random sampling to select the initial cluster centroids,and we compute the average RI for evaluation after having performed K -means 50times.

TABLE II

P ERFORMANCE C OMPARISON ON THE 20-N EWSGROUP D ATA S ET

M EASURED BY

The text data set in this experiment is a modi?ed 20-newsgroup data set downloaded from [22];it is a tiny version of the 20-newsgroup data set [23]with binary occur-rence data for 100selected keywords across 16242postings.Although the data are organized into 20different newsgroups,we can separate them into four clusters according to the highest level of the topic naming,i.e.,comp,rec,sci,and talk.We randomly select 500samples from each cluster,and thus,our data set consists of 2000samples from four clusters;each sample has a dimensionality of 100.

We generate two views on the text data set following the same way in [19];we randomly draw 50words from the 100keywords and regard the term vector on the 50words as view 1and the term vector on the remaining 50words as view 2.The two views are nearly independent since they are represented in two separated term spaces.Because the two views for a document have rich complementary among them,we empirically set r in MSE as nine.The dimensionality of the low-dimensional embedding d is set as ten.

Table II shows the results for performance comparison.“Non-DR”denotes the result of performing K -means on the original 100-D data;it is the result without performing di-mensionality reduction.“LE on View 1”denotes the result by LE performed on view 1.We can see that MSE achieves the best https://www.doczj.com/doc/bd10424597.html,pared with the multimedia data set,the performance of MSE has limited advantage on the 20-newsgroup data set,e.g.,the RI of CSE is very close to that of MSE.That is because the two views of the text data set are nearly independent,and each single view has very low performance for clustering (i.e.,0.521and 0.492);in such a case,it is very dif?cult to handle the disagreements from the two views.

E.Effect of Parameter r

In this section,we investigate the effect of parameter r in MSE.Table III illustrates the performance variation of MSE with respect to r ;the dimensionality of the low-dimensional embedding d is set as 30for the Corel 2000data set and as 10for the 20-newsgroup data set.As discussed in Section III-C,rich complementary prefers large r .It is known that the 20-newsgroup data set has richer complementary among dif-ferent views than Corel 2000does.We can see that the

P ERFORMANCE C OMPARISON OF MSE W ITH D IFFERENT

TABLE IV

P ERFORMANCE C OMPARISON ON THE C OREL 2000D ATA S ET W ITH

D IFFERENT d M EASURED BY

20-newsgroup data set performs the best around r =9;the

Corel 2000data set performs the best around r =6.F .Performance Under Different Values of d

In this section,we show the performance comparison un-der different dimensions of the low-dimensional embedding.Table IV illustrates the performance comparison on the Corel 2000data set with respect to d ;parameter r is empirically set as ?ve in MSE.We can see that MSE achieves the best performance on all the settings of d except when d =50,and also,we can ?nd that the optimal value of d for the Corel 2000data set is between 10and 20.

V .C ONCLUSION

In many applications,objects are represented by different views.However,existing spectral-embedding algorithms can-not deal with multiview data directly.In this paper,we have proposed a novel MSE,which learns a low-dimensional and suf?ciently smooth embedding over all views simultaneously.From the experimental results,we can conclude that MSE can effectively explore the complementary property of different views to obtain an effective low-dimensional embedding for multiview data sets.We also ?nd the following:1)MSE has promising results on multimedia data (e.g.,an image or a video),which generally have complementary among different features,and 2)MSE achieves limited success in the case when different views are nearly independent,and each single view is not informative enough,e.g.,the text data set in Section IV-D;in such a case,the performances of CSE and MSE are very close.

There are,however,some open problems in MSE:1)One is how to select the optimal dimension of the low-dimensional embedding;from Table IV,we can ?nd that there exists an optimal value of d for some data set,and MSE achieves limited success under inappropriate setting of d ;and 2)MSE is a spec-tral method which needs to perform eigenspace decomposition of matrices of size N ×N (N is the number of samples in the data set);this generally takes O (N 3)time,and it will be very time consuming when N is very large.In the future,we will consider how to utilize the sampling technique [26]in MSE to handle a large-scale data set.

A PPENDIX I

A.Proof of L (i )

n Being Symmetric and Positive Semide?nite According to (11),the symmetry of L (i )

n follows directly from the symmetry of W (i )and D (i ).As for positive semide?-niteness,given a vector f ∈ n ,we have

f T L (i )n f =f T f ?f T D (i ) ?1/2W (i ) D (i )

?1/2

n k =1

f 2

k ?

p,q =1

f p D (i ) pp f q

D (i )

W (i )

=12?

?n k =1

f 2

k ?2

n p,q =1

f p D (i ) pp ×f q

D (i )

(i )

n k =1

f 2k

12n

p,q =1

W (i ) pq ??f p D (i ) pp

?f q

D (i )

≥0

thus,L (i )

n is positive semide?nite.B.Ky-Fan Theorem

Let M ∈ n ×n be a symmetric matrix with the smallest k eigenvalues λ1≤···≤λk ,and the

corresponding eigenvectors U =[u 1,...,u k ].

Then, k

i =1λi =min X ∈ n ×k ,X T X =I k tr(X T

MX ).

Moreover,the optimal X is given by UQ ,where Q is an arbitrary orthogonal matrix.Please refer to [18]for the detailed proof.

A CKNOWLEDGMENT

The authors would like to thank the handling editor and all anonymous reviewers for their insightful comments.

R EFERENCES

[1]S.Ruping and T.Scheffer,in Proc.Workshop Learning With Multiple

Views,22nd ICML ,Bonn,Germany,Aug.2005.

[2]W.Liu,D.Tao,and J.Liu,“Transductive component analysis,”in Proc.

IEEE Int.Conf.Data Mining ,2008,pp.433–442.

[3]B.Long,P.S.Yu,and Z.Zhang,“A general model for multiple view un-supervised learning,”in Proc.8th SIAM Int.Conf.Data Mining ,Atlanta,GA,Apr.2008,pp.822–833.

[4]D.Zhou and C.Burges,“Spectral clustering and transductive learning

with multiple views,”in Proc.24th Int.Conf.Mach.Learn.,Corvallis,OR,Jun.2007,pp.1159–1166.

[5]D.Tao,X.Tang,X.Li,and X.Wu,“Asymmetric bagging and ran-dom subspace for support vector machines-based relevance feedback in

image retrieval,”IEEE Trans.Pattern Anal.Mach.Intell.,vol.28,no.7, pp.1088–1099,Jul.2006.

[6]W.Bian and D.Tao,“Biased discriminant Euclidean embedding for con-

tent based image retrieval,”IEEE Trans.Image Process.,vol.19,no.2, pp.545–554,Feb.2010.

[7]J.Tenenbaum,V.de Silva,and https://www.doczj.com/doc/bd10424597.html,ngford,“A global geometric frame-

work for nonlinear dimensionality reduction,”Science,vol.290,no.5500, pp.2319–2323,Dec.2000.

[8]S.T.Roweis and L.K.Saul,“Nonlinear dimensionality reduction by

locally linear embedding,”Science,vol.290,no.5500,pp.2323–2326, Dec.2000.

[9]M.Belkin and P.Niyogi,“Laplacian eigenmaps and spectral techniques

for embedding and clustering,”in Proc.Adv.Neural Inf.Process.Syst., Vancouver,BC,Canada,Dec.2002,pp.585–591.

[10]D.L.Donoho and C.Grimes,“Hessian eigenmaps:New locally linear

embedding techniques for high-dimensional data,”Proc.Nat.Acad.Sci., vol.100,no.10,pp.5591–5596,2003.

[11]Z.Zhang and H.Zha,“Principle manifolds and nonlinear dimension re-

duction via local tangent space alignment,”SIAM https://www.doczj.com/doc/bd10424597.html,put.,vol.26, no.1,pp.313–338,2005.

[12]U.V.Luxburg,“A tutorial on spectral clustering,”Max Planck Inst.Biol.

Cybern.,Tübingen,Germany,Tech.Rep.TR-149,Aug.2006.

[13]M.Wang,X.S.Hua,X.Yuan,Y.Song,and L.R.Dai,“Optimizing multi-

graph learning:Towards a uni?ed video annotation scheme,”in Proc.

ACM Multimedia,Augsburg,Germany,Sep.2007,pp.862–870. [14]J.C.Bezdek and R.J.Hathaway,“Some notes on alternating optimiza-

tion,”in Proc.AFSS Int.Conf.Fuzzy Syst.,vol.2275,LNAI,2002, pp.288–300.

[15]G.Grif?n, A.Holub,and P.Perona,“Caltech-256object category

dataset,”California Inst.Technol.,Pasadena,CA,Tech.Rep.TR-7694, Mar.2007.

[16]F.Long,H.Zhang,and D.Feng,“Fundamental of content-based im-

age retrieval,”in Multimedia Information Retrieval and Management.

New York:Springer-Verlag,2002.

[17]TRECVID,Trec Video Retrieval Evaluation2008.[Online].Available:

https://www.doczj.com/doc/bd10424597.html,/projects/trecvid/

[18]R.Bhatia,Matrix Analysis.New York:Springer-Cerlag,1997.

[19]S.Bickel and T.Scheffer,“Multi-view clustering,”in Proc.Int.Conf.Data

Mining,2004,pp.19–26.

[20]Z.Zhao and H.Liu,“Multi-source feature selection via geometry-

dependent covariance analysis,”in Proc.JMLR Workshop Conf.,2008, vol.4,pp.36–47.

[21]A.Zien and C.S.Ong,“Multiclass multiple kernel learning,”in Proc.

24th ICML,Corvallis,OR,2007,pp.1191–1198.

[22][Online].Available:https://www.doczj.com/doc/bd10424597.html,/~roweis/data.html

[23][Online].Available:https://www.doczj.com/doc/bd10424597.html,/jrennie/20Newsgroups/

[24]S.P.Lloyd,“Least square quantization in PCM,”IEEE Trans.Inf.Theory,

vol.IT-28,no.2,pp.129–137,Mar.1982.

[25]W.M.Rand,“Objective criteria for the evaluation of clustering methods,”

J.Amer.Stat.Assoc.,vol.66,no.336,pp.846–850,Dec.1971.

[26]A.Talwalkar, A.Kumar,and H.Rowley,“Large-scale manifold

learning,”in Proc.IEEE https://www.doczj.com/doc/bd10424597.html,put.Vis.Pattern Recog.,Anchorage, AK,Jun.2008,pp.1–8.

[27]X.Li,S.Lin,S.Yan,and D.Xu,“Discriminant locally linear embedding

with high-order tensor data,”IEEE Trans.Syst.,Man,Cybern.B,Cybern., vol.38,no.2,pp.342–352,Apr.2008.

[28]R.Datta,D.Joshi,J.Li,and J.Z.Wang,“Image retrieval:Ideas,in?u-

ences,and trends of the new age,”ACM Comput.Surv.,vol.40,no.2, pp.1–60,Apr.2008.

[29]T.Zhang,D.Tao,X.Li,and J.Yang,“Patch alignment for dimensionality

reduction,”IEEE Trans.Knowl.Data Eng.,vol.21,no.9,pp.1299–1313, Sep.2009.

[30]T.Zhang,D.Tao,and J.Yang,“Discriminative locality alignment,”in

Proc.10th https://www.doczj.com/doc/bd10424597.html,put.Vis.,2008,pp.

725–738.

Tian Xia received the B.E.degree in electrical engi-neering from the University of Science and Technol-ogy of China,Hefei,China,in2004and the Ph.D. degree in computer science from the Institute of Computing Technology,Chinese Academy of Sci-ences,Beijing,China,in2009.

He is currently with the Institute of Computing Technology,Chinese Academy of Sciences.His re-search interests focus on multimedia retrieval and machine

learning.

Dacheng Tao(M’07)received the B.Eng.degree

from the University of Science and Technology of

China,Hefei,China,the M.Phil.degree from the

Chinese University of Hong Kong,Hong Kong,

China,and the Ph.D degree from the University of

London,London,U.K.

He is currently a Nanyang Assistant Professor

with the School of Computer Engineering in the

Nanyang Technological University.His research is

mainly on applying statistics and mathematics for

data analysis problems in computer vision,multi-media,machine learning,data mining,and video surveillance.He has au-thored/edited six books and eight journal special issues.He has published more than100scienti?c papers including IEEE T RANSACTIONS ON P ATTERN A NALYSIS AND M ACHINE I NTELLIGENCE(T-PAMI)Neural Information Processing Systems,with best paper awards.One of his T-PAMI papers was listed as a“New Hot Paper”in https://www.doczj.com/doc/bd10424597.html,(Thomson Scienti?c).His H-Index in google scholar is17+and his Erd?s number is3.

Dr.Tao is an Associate Editor of IEEE T RANSACTIONS ON K NOWLEDGE AND D ATA E NGINEERING and the Computational Statistics&Data Analysis (Elsevier).He has(co)chaired for special sessions,invited sessions,workshops, panels,and conferences.He has served with more than80major international conferences and more than30prestigious international journals.He is a member of the IEEE Computer Society and IEEE Signal Processing

Society.

Tao Mei(M’07)received the B.E.degree in automa-

tion and the Ph.D.degree in pattern recognition and

intelligent systems from the University of Science

and Technology of China,Hefei,China,in2001and

2006,respectively.

In2006,he joined Microsoft Research Asia,

Beijing,China,as a Researcher Staff Member.His

current research interests include multimedia content

analysis,computer vision,and Internet multimedia

applications such as search,advertising,manage-

ment,social network,and mobile applications.He is the author of one book,?ve book chapters,and over80journal and conference papers in these areas and is the holder of more than20?led international and U.S.patents or pending applications.

Dr.Mei is a member of the Association for Computing Machinery(ACM). He serves as an Editorial Board member for the Journal of Multimedia and a Guest Editor for IEEE M ULTIMEDIA for the Special Issue on“Knowledge Discovery Over Community-Contributed Multimedia Data:Opportunities and Challenges,”for ACM/Springer Multimedia Systems for the Special Issue on“Multimedia Intelligent Services and Technologies,”and for the Elsevier Journal of Visual Communication and Image Representation for the Special Issue on“Large-Scale Image and Video Search:Challenges,Technologies,and Trends.”He was the Principle Designer of the automatic video search system that achieved the best performance in the worldwide TRECVID evaluation in2007.He was the recipient of the Best Paper and Best Demonstration Awards in the2007ACM International Conference on Multimedia,the Best Poster Paper Award in the2008IEEE International Workshop on Multimedia Signal Processing,and the Best Paper Award in the2009ACM International Conference on

Multimedia.

Yongdong Zhang(M’08)received the Ph.D.degree

from Tianjin University,Tianjin,China,in2002.

He is currently a Professor with the Institute of

Computing Technology,Chinese Academy of Sci-

ences,Beijing,China.His research interests focus on

image processing and video processing.

自动化常用英文术语

有限自动机finite automaton 工厂信息协议FIP (factory information protocol) 一阶谓词逻辑first order predicate logic 固定顺序机械手fixed sequence manipulator 定值控制fixed set point control 流量传感器flow sensor/transducer 流量变送器flow transmitter 涨落fluctuation 柔性制造系统FMS (flexible manufacturing system) 强迫振荡forced oscillation 形式语言理论formal language theory 形式神经元formal neuron 正向通路forward path 正向推理forward reasoning 分形体，分维体fractal 变频器frequency converter 频域模型降阶法frequency domain model reduction method 频域响应frequency response 全阶观测器full order observer 功能分解functional decomposition 功能相似functional simularity 模糊逻辑fuzzy logic 对策树game tree 闸阀gate valve 一般均衡理论general equilibrium theory 广义最小二乘估计generalized least squares estimation 生成函数generation function 地磁力矩geomagnetic torque 几何相似geometric similarity 框架轮gimbaled wheel 全局渐进稳定性global asymptotic stability 全局最优global optimum 球形阀globe valve 目标协调法goal coordination method 文法推断grammatical inference 图搜索graphic search 重力梯度力矩gravity gradient torque 成组技术group technology 制导系统guidance system 陀螺漂移率gyro drift rate 陀螺体gyrostat

rfc3956.Embedding the Rendezvous Point (RP) Address in an IPv6 Multicast Address

Network Working Group P. Savola Request for Comments: 3956 CSC/FUNET Updates: 3306 B. Haberman Category: Standards Track JHU APL November 2004 Embedding the Rendezvous Point (RP) Address in an IPv6 Multicast Address Status of this Memo This document specifies an Internet standards track protocol for the Internet community, and requests discussion and suggestions for improvements. Please refer to the current edition of the "Internet Official Protocol Standards" (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited. Copyright Notice Copyright (C) The Internet Society (2004). Abstract This memo defines an address allocation policy in which the address of the Rendezvous Point (RP) is encoded in an IPv6 multicast group address. For Protocol Independent Multicast - Sparse Mode (PIM-SM), this can be seen as a specification of a group-to-RP mapping mechanism. This allows an easy deployment of scalable inter-domain multicast and simplifies the intra-domain multicast configuration as well. This memo updates the addressing format presented in RFC 3306. Table of Contents 1. Introduction (2) 1.1. Background (2) 1.2. Solution (2) 1.3. Assumptions and Scope (3) 1.4. Terminology (4) 1.5. Abbreviations (4) 2. Unicast-Prefix-based Address Format (4) 3. Modified Unicast-Prefix-based Address Format (5) 4. Embedding the Address of the RP in the Multicast Address (5) 5. Examples (7) 5.1. Example 1 (7) 5.2. Example 2 (7) 5.3. Example 3 (8) 5.4. Example 4 (8) Savola & Haberman Standards Track [Page 1]

Embedding CSR values

Embedding CSR Values:The Global Footwear Industry’s Evolving Governance Structure Suk-Jun Lim Joe Phillips ABSTRACT.Many transnational corporations and international organizations have embraced corporate social responsibility(CSR)to address criticisms of working and environmental conditions at subcontractors’factories.While CSR‘codes of conduct’are easy to draft, supplier compliance has been elusive.Even third-party monitoring has proven an incomplete solution.This article proposes that an alteration in the supply chain’s governance,from an arms-length market model to a collaborative partnership,often will be necessary to effectuate CSR.The market model forces contractors to focus on price and delivery as they compete for the lead firm’s business,rendering CSR observance secondary,at best.A collaborative partnership where the lead firm gives select suppliers secure product orders and other benefits removes disincentives and adds incentives for CSR compliance.In time,the suppliers’CSR habit should shift their business philosophy toward pursuing CSR as an end in itself,regardless of buyer incentives and monitoring. This article examines these hypotheses in the context of the athletic footwear sector with Nike,Inc.and its suppliers as the specific case study.The data collected and conclusions reached offer strategies for advancing CSR beyond the superficial and often ineffectual‘code of conduct’stage. KEY WORDS:corporate social responsibility,global value chains,Nike,athletic footwear industry,gover-nance Introduction The governance of global value chains(GVCs)has occupied political economy and management liter-ature for more than a decade.Studies reveal that major retailers no longer own manufacturing facili-ties but contract with multiple suppliers,usually in less developed countries(LDCs),and often play them against each other to achieve the best price, highest quality,and quickest delivery.While eco-nomically bene?ting the buyer and subsequent consumers,this competitive,market-oriented GVC usually drives suppliers toward lower wages and sweatshop-like labor/environmental standards. One supposed route to improved conditions at the factories is‘corporate social responsibility’(CSR). CSR is sometimes presented as an enlightened approach by transnational corporations(TNCs);it is sometimes criticized as a public relations cover for business-as-usual.Stages of CSR development in a corporation’s culture have been chronicled(Zadeck, 2004)and the business case for CSR analyzed (Capaldi,2005;Vogel,2005).Less discussed and understood are CSR’s role in the evolution of GVC governance and how governance changes promote CSR compliance among contractors. Our insight is that a global buyer’s CSR concerns can trigger a transformation in GVCs from the competitive,arms-length market structure to a ‘collaborative partnership’where suppliers have a more layered and economically securer relationship with the buyer.This deeper,securer relationship should lead not only to superior compliance with the TNC’s code of conduct,but,eventually,to suppliers developing an independent ethical commitment to CSR. The traditional market-oriented governance model has been inhospitable to buyer’s CSR codes because the suppliers’performance is judged entirely on price,quality,and delivery.These economic pressures encourage suppliers to cheat on CSR to Journal of Business Ethics(2008)81:143–156óSpringer2007 DOI10.1007/s10551-007-9485-2

C-C计算延迟时间和嵌入维数3

function LCE1=lyapunov_rosenstein(x,m,t,P,p,ts) % Syntax:LCE1=lyapunov_rosenstein(x,m,t,P,p) % Calculate maximum lyapunov exponent from small sets % "A practical method for calculating largest lyapunov % exponents from small sets"--Michael T.Rosenstein %Input: % x----time series coloum format % m----embedding dimension % t----time delay % P----mean perieod time % p----p norm % ts---Sampling frequence %Output: % LCE1-Dominant lyapunov exponents % %Usage: % x=[];!!!!Remember: in coloum format % m=30; % t=10; % P=20;%gained by FFT % p=inf; % ts=0.001; % LCE1=lyapunov_rosenstein(x,m,t,P,p,ts); %Related routine: % phasespace.m %Author:skyhawk % Created on 2004.3.8 % Modified on 2004.3.23 % Air Force Engineering University % Air Force Engineering Institute % Dept.1, Shaan Xi, Xi'an 710038, PR China. % Email:xunkai_wei@https://www.doczj.com/doc/bd10424597.html, % [Y,M]=phasespace(x,m,t); %calculate two points' distance in phase space and determine the nearest neighbor %count computation times for i=1:M %Calculate original distance count_refer=1; Refer=Y(i,:); for j=1:M if abs(j-i)

Fixation, Embedding, and Sectioning of Plant Tissues

2008; doi: 10.1101/pdb.prot4941 Cold Spring Harb Protoc Detlef Weigel and Jane Glazebrook Fixation, Embedding, and Sectioning of Plant Tissues Service Email Alerting click here.Receive free email alerts when new articles cite this article - Categories Subject Cold Spring Harbor Protocols. Browse articles on similar topics from (114 articles) Visualization of Gene Expression (35 articles)Plant Cell Culture (118 articles)Plant Biology, general (98 articles)Plant (991 articles)Molecular Biology, general (876 articles)Laboratory Organisms, general (80 articles)In Situ Hybridization (71 articles)Immunohistochemistry (995 articles)Cell Biology, general (66 articles)Arabidopsis (19 articles)Analysis of Gene Expression in Plants (139 articles)Analysis of Gene Expression https://www.doczj.com/doc/bd10424597.html,/subscriptions go to: Cold Spring Harbor Protocols To subscribe to

Feature Extraction based on Sparsity embedding with manifold information for face recognition

Feature Extraction based on Sparsity Embedding with Manifold Information for Face Recognition Shi-Qiang Gao1, Xiao-Yuan Jing1,2,*, Chao Lan1, Yong-Fang Yao1, Zai-Juan Sui1 1School of Automation, Nanjing University of Posts and Telecommunications, Nanjing, 210003, China 2State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210093, China *Corresponding author, Email: jingxy_2000@https://www.doczj.com/doc/bd10424597.html, Abstract—In the past few years, manifold learning and sparse representation have been widely used for feature extraction and dimensionality reduction. The sparse representation technique shows that one sample can be linearly recovered by the others in a data set. Based on this, sparsity preserving projections (SPP) has recently been proposed, which simply minimizes the sparse reconstructive errors among training samples in order to preserve the sparse reconstructive relations among data. However, SPP does not investigate the inherent manifold structure of the data set, which may be helpful for the recognition task. Motivated by this, we present in this paper a novel feature extraction approach named sparsity embedding with manifold information (SEMI), which not only preserves the sparse reconstructive relations, but also maintains the manifold structure of the reconstructed data. Specifically, for a sparse reconstructed sample, we minimize both its difference to the corresponding original sample as SPP does, and its distance to the original intra-class samples. Provided that this sample lies on different submanifolds from other samples, we additionally maximize, in the objective function, its distance to the original inter-class samples. Experimental results on two public ORL and AR face databases demonstrate that SEMI outperforms related methods in classification performance. Keywords-sparse representation; suprivised manifold learning; feature extraction; dimensionality reduction; face recognition I.I NTRODUCTION Feature extraction has been widely used for face and palm recognition [1][2]. As a feature extraction and dimensionality reduction technique, manifold learning has attracted more and more research interest. It tends to preserve the manifold structure of a given data set in a low-dimensional subspace. Traditional manifold learning algorithms only define nonlinear mapping on training data, and cannot deal with new test data. Linear manifold embedding methods are developed to address this issue, such as locality preserving projection (LPP) [3] and neighborhood preserving embedding (NPE) [4]. LPP seeks a linear embedded space where local relationships of samples are preserved, while NPE seeks an embedded space where reconstructive relations of samples by their k nearest neighbors can be preserved. While training, all abovementioned manifold learning methods do not consider the class label information, which has been used to extract face features in [5]. To take advantage of the class separability, some supervised manifold learning methods have been proposed. Local discriminant embedding (LDE) maintains the intrinsic neighbor relations of the intra-class samples by setting the affinity weights [6]. Marginal fisher analysis (MFA) constructs the intra-class compactness and inter-class separability graphs by using the class label information [7]. As described in [8][9], the discriminant information can be extracted to enhance the classification performance. As another technique of pattern recognition, sparse representation shows that one sample can be recovered by the others. Recently, a new sparse representation method, namely sparsity preserving projections (SPP), has been proposed, which tries to preserve the sparse reconstructive relations among data by minimizing sparse reconstructive errors [10]. However, in real-world application like face recognition, face images often lie on a manifold structure, and preserving the manifold structure may enhance the recognition performance. Motivated by this, we proposed a novel feature extraction method named sparsity embedding with manifold information (SEMI), which aims at preserving both the sparse reconstructive relations and the manifold structure of the data set. Explicitly, to preserve the sparse relations as SPP does, we minimize the difference between original data and their corresponding reconstructive samples. Then, presuming that a reconstructed sample is close not only to the corresponding original sample but also to its neighbors, SEMI minimizes the distance between the original data and the reconstructed intra-class samples. Furthermore, providing that samples from different classes lie on separate submanifolds, SEMI also maximizes the distance between original data and reconstructed inter-class samples. By this means, SEMI not only preserves the sparse reconstructive relations but also the manifold structure of the data set. The rest of this paper is organized as follows: in Section 2, we briefly review two feature extraction methods, i.e., LDE and SPP. In Section 3, we present the sparsity embedding with manifold information (SEMI) approach. Experimental results on the public ORL and AR face databases are provided in Section 4 and conclusions are given in Section 5. II.R ELEATED W ORK A.Local Discriminant Embedding (LDE) LDE tries to not only preserve the neighbor relations of intra-class samples, but also enhance the separability between inter-class neighbor samples. Suppose a sample set 12n [,,...,]n m X x x x R× =∈ and let W and W′ be the affinity matrices which are defined as follows:

De-embedding技术

De-embedding技术总结 RF工程师们依赖于矢量网络分析仪(VNA)来测量RF元件的S参数，从而进行描述以及后续的设计。测量中会出现一个问题，即这些元件往往是表贴式，因此不能与VNA直接相连。简单的印刷版(PCB)测试装置往往都会表贴上待测器件(DUT)以便与VNA相连接，如图1所示。然而，这些测试装置本身会为S参数测量带来一些寄生效应，因此，需要De-embedding技术进行消除。图1 一、De-embedding技术定义微波元器件的测量必须包含两部分：第一部分是把实际被测器件（DUT）加上焊点（对于在片测量系统）或者测试夹具（对于同轴测量系统）当做一个被测器件；另一部分是空的焊点或者测试夹具作为被测器件。然后用第一部分测得的S参数扣除第二部分（焊点或者测试夹具部分产生的寄生效应），从而可以得到被测器件的真实性能。我们把这个步骤就叫做De-embedding。 De-embedding技术是微波测量的重要技术之一，主要目的是消除寄生元件、部件对实际被测器件（DUT）的影响。一般分为四大类：串联技术、并联技术、级联技术及混合技术。为了确保测量值为实际被测器件的本身特性，必须进行De-embedding。二、De-embedding技术理论简介由上面对于De-embedding技术的定义我们知道，我们测量的S参数实际是DUT、寄生元件(如探针)、其他元件(如焊点)综合下的S参数，因此，我们需要对这些因素进行消除，以达到精确测量的目的。对于任何一个被测器件DUT，我们都会使用到探针，而探针跟DUT是级联关系，我们采用A参数处理；对于焊点来说，有短路焊点与开路焊点，开路焊点相当于在DUT并联，短路焊点相当于串联器件，对于并联与串联，我们分别使用导纳参数与阻抗参数进行运算。

Efficient Steganographic Embedding by Exploiting Modification Direction

Ef?cient Steganographic Embedding by Exploiting Modi?cation Direction Xinpeng Zhang and Shuozhong Wang Abstract—A novel method of steganogrphic embedding in digital images is described,in which each secret digit in a(2n+1)-ary notational system is carried by n cover pixels and,at most, only one pixel is increased or decreased by1.In other words, the(2n+1)different ways of modi?cation to the cover pixels correspond to(2n+1)possible values of a secret digit.Because the directions of modi?cation are fully exploited,the proposed method provides high embedding ef?ciency that is better than previous techniques. Index Terms—Covert communication,steganography,embed-ding ef?ciency,embedding rate. I.I NTRODUCTION D IGITAL steganography is a technique for sending secret messages under the cover of a carrier signal.Despite that steganographic techniques only alter the most insigni?cant components,they inevitably leave detectable traces so that successful attacks are often possible.The primary goal of attack on steganographic systems,termed steganalysis,is to detect the presence of hidden data by?nding statistical abnor-mality of a stego-media caused by data embedding[1],[2]. Generally speaking,the more the secret data are embedded,the more vulnerable is the steganographic system to steganalytic attempts. One way of improving security of a steganographic system is to reduce the amount of alterations necessary to be intro-duced into the cover signal for data hiding when the number of secret bits is signi?cantly less than that of available host samples.For example,the method proposed in[3]can conceal as many as log2(mn+1) bits of data in a binary image block sized m×n by changing,at most,two bits in the block. Matrix encoding[4],on the other hand,uses less than one change of the least signi?cant bit(LSB)in average to embed l bits into2l?1pixels.In this way,the introduced distortion is signi?cantly lowered compared to a plain LSB replacement technique in which secret bits are used to simply replace the LSB plane.Further,some effective encoding methods derived from the cyclic coding have been described[5],and the matrix encoding can be viewed as a special case.In[6], two steganographic encoding methods based on random linear Manuscript received June5,2006.The associate editor coordinating the review of this manuscript and approving it for publication was Prof.Deepa Kundur.This work was supported in part by the National Natural Science Foundation of China under Grants60372090and60502039,in part by the Key Project of Shanghai Municipality for Basic Research under Grant04JC14037, and in part by Shanghai Leading Academic Discipline Project under Grant T0102. The authors are with the School of Communication and Information Engineering,Shanghai University,Shanghai200072China(e-mail:{xzhang, shuowang}@https://www.doczj.com/doc/bd10424597.html,). Digital Object Identi?er10.1109/LCOMM.2006.060863.codes and simplex codes are developed for large payloads. The stego-coding technique can also be performed on a data stream derived from the host in a dynamically running manner, so that insertion and extraction of each secret bit are carried out in a series of consecutive cover bits[7].All the above-mentioned stego-coding techniques are independent of various cover-bit-modi?cation approaches.For example,if the stego-coding methods are used in the LSB plane of an image,adding 1to a pixel is equivalent to subtracting1from the pixel to ?ip its LSB for carrying the secret message. In[8],another type of steganographic method for improving embedding ef?ciency is presented in which the choice of whether to add or subtract one to/from a pixel depends both on the original gray value and a pair of two consecutive secret bits.In other words,the direction of modi?cation to the cover pixels is exploited for data hiding.However,there exist two different modi?cation-directions corresponding to a same pair of secret bits to be embedded,meaning that the exploitation is incomplete. This letter proposes a novel steganographic embedding method that fully exploits the modi?cation directions,called the EMD embedding for short.In this method,modi?cations in different directions are used to represent different secret data,leading to a higher embedding ef?ciency. II.EMD E MBEDDING The main idea of the proposed steganographic method is that each secret digit in a(2n+1)-ary notational system is carried by n cover pixels,where n is a system parameter, and,at most,only one pixel is increased or decreased by1. Actually,for each group of n pixels,there are2n possible ways of modi?cation.The2n different ways of alteration plus the case in which no pixel is changed form(2n+1)different values of a secret digit. Before data-embedding,a data-hider can conveniently con-vert a secret message into a sequence of digits in the notational system with an odd base(2n+1).If the secret message is a binary stream,it can be segmented into many pieces with L bits,and the decimal value of each secret piece is represented by K digits in a(2n+1)-ary notational system,where L= K·log2(2n+1) (1) For example,the binary sequence(110101101001)can be expressed as(231114)in a5-ary notational system where L=4and K=2.Thus,the redundancy rate in the(2n+1)-ary sequence is R R=1? L K log2(2n+1) < 1 L+1 (2) 1089-7798/06$20.00c 2006IEEE