Learning Analysis by Reduction from Positive Data1

Professions for WomenI. Fill in the blanks with proper prepositions or adverbs.1. He speaks _______ an accent peculiar ______ the north of the region.2. ______ the exception ______ Mary, all have seen the film.3. He couldn’t get out _______ the depth ________ despair.4. She was given the credit _______ what I had done.5. You’ve done so much work that you’re bound ______ pass the exam.6. _______ _______ respect, he took off his hat.7. He feels that there is not enough stress ______ drama at the school.8. She excels _______ foreign language.9. It is necessary to discuss the ends and aims ______ which we are fighting.10. We are still ______ the process ______moving the house.II. Choose the appropriate word or phrase for each blank.1. He considered himself a great singer, and _______(in the way, in a way) it is true.2. Move the chair, it is _____(in the way, in a way).3. He’s a (an) ______ (professional, occupational) basketball player, for he makes money byplaying the games.4. Skin disorders are common ________ (professional, occupational)diseases among factory workers.5. In our fast-paced world nothing seems ______(constant, consistent).6. His actions are always ______(constant, consistent) with his words.7. The man to whom we handed the forms pointed out that they had not been ___ (properly,regularly) filled in.8. Take this medicine _______ (properly, regularly)three times a day.9. More than one thirds of the Chinese in the United States live in California,_______(predominantly, practically) in San Francisco.10. His works is ______(predominantly, practically)unknown here.11. The final document was of course, supposed to mend the damage _______(imposed,impeded) upon the world by the war.12. The development of the project was seriously_______(imposed, impeded) by reduction infunds.13. As a _______ (versatile, productive) actor, he can perform, sing, dance and play severalkinds of musical instruments.14. They work hard, but their efforts are not very______ (versatile, productive) .15. My supply of confidence slowly _______(diminished, eliminated) as the deadlineapproached.16. The teacher is trying to ______(diminish, eliminate) mistakes from the students’ writing.17. One of the striking facts of nature is the ability of living things to ______(adapt, adopt)themselves to practically any environment on earth.18. I hate those officials who _____(adapt, adopt) an air of importance.19. He gave a (an) _______(acute, vivid) analysis of the current events.20. She gave a (an) _______(acute, vivid) account of that accident.21. I ______ (advised, proposed) them to go by train, but they went by bus after all.22. I ______ (advised, proposed) that we start early.III. Choose the appropriate rhetorical device for each sentence.a. onomatopoeiab. metaphorc. metonymyd. antithesise. parallelismf. oxymorong. synecdoche1. She would have plucked the heart out of my writing.2. I heard the rustling of her skirts in the room.3. There is still much of a schoolboy in him.4. Whenever my husband has a bad day at the office, he hits the bottle.5. She has a photographic memory for detail.6. While most of us are only too ready to apply to others the cold wind of criticism, we aresomewhat reluctant to give our fellow the warm sunshine of praise.7. Have you got any coppers?8. A pessimist is one who makes difficulties of his opportunities; and an optimist is one whomakes opportunities of his difficulties.9. They vanished from a world where they were of no consequences, where they achievednothing; where they were a mistake and failure and a foolishness; where they left no sign that they had existed.10. I despise its very vastness and power. It has the poorest millionaires, the littlest great men,the haughtiest beggars, the plainest beauties, the lowest skyscrapers, the dolefulest pleasures of any town I ever saw.IV. Cloze Test.Some persons say that 1 makes the world go round. Others of a less romantic and more practical turn of mind say that it isn’t love; it’s 2 . But the truth is that it is 3 that makes the world go round. Energy is the currency of the ecological system and 4 becomes possible only when 5 is converted into energy, which in turn is used to seek more food to grow, to reproduce and to survive. On this cycle all life 6 .It is fairly well known that wild animals survive from year to year by 7 as much as they can during times of plenty, the summer and fall, storing the excess, usually in the form of 8 , and then using these reserves of fat to survive during the hard times in winter when food is 9 . But, it is probably less well known that even with their stored fat, wild animals spend 10 energy to live in winter than in summer.( ) 1. A. money B. love C. air D. water( ) 2. A. money B. human C. earth D. sun( ) 3. A. gravity B. energy C. oxygen D. air( ) 4. A. mankind B. people C. man D. life( ) 5. A. plant B. water C. oxygen D. food( ) 6. A. works B. moves C. depends D. carries( ) 7. A. storing B. eating C. planting D. destroying( ) 8. A. food B. raw material C. fat C. oil( ) 9. A. hard B. cold C. scarce D. plenty( ) 10. A. less B. more C. little D. muchV. Do as you are required.(1) Find the topic sentence of the paragraph.(5%)Spare-time learners are usually the best learners. Their rate of learning is helped, of course,by the fact that they want to learn and consequently try to learn. but they are also helped by circumstances—they are forced to take their learning by easy stages.(2) Tell the main idea of the following paragraph.The second advantage of most paid work and of some unpaid work is that it gives chances of success and opportunities for ambition. In most work success is measured by income, and while our capitalistic society continues, this is inevitable. it is only where the best work is concerned that this measure ceases to be the natural one to apply. The desire that men feel to increase their income is quite as much a desire for success as for the extra comforts that a higher income can obtain. However dull work may be, it becomes bearable if it is a means of building up reputation, whether in the world at large or only in one’s own circle. Continuity of purpose is one of the most essential ingredients of happiness in the long run, and for most men this comes chiefly through their work. In this respect those women whose lives are occupied with housework are much less fortunate than men, or than women who work outside the home. The domesticated wife does not receive wages, has no means of bettering herself, is taken for granted by her husband (who sees practically nothing of what she does), and is valued by him not for her housework but for quite other qualities. Of course this does not apply to those women who are sufficiently well-to-do to make beautiful houses and beautiful gardens and become the envy of their neighbors; but such women are comparatively few, and for the great majority housework cannot bring as much satisfaction as work of other kinds brings to men and to professional women.Keys to exercisesI. Fill in the blanks with proper prepositions or adverbs.1. with, to2. With, of3. in, of4. for5. to6. Out of7. on/upon8. in9. for 10. in,ofII. Choose the appropriate word or phrase for each blank.1. in a way2. in the way3. professional4. occupational5. constant6. consistent7. properly8. regularly9. predominantly 10. practically 11. imposed 12. impeded 13. versatile 14. productive 15. diminished 16. eliminate 17. adapting 18.adopt 19. acute 20. vivid 21. advised 22. proposedIII. Choose the appropriate rhetorical device for each sentence.a. onomatopoeiab. metaphorc. metonymyd. antithesise. parallelismf. oxymorong. synecdoche1. b. (metaphor)2. a (onomatopoiea)3. c. (metonymy)4. c. (metonymy)5. b. (metaphor)6. b, d. (metaphor and antithesis)7. g. (synecdoche)8. d. (antithesis)9. e. (parallelism)10. f. (Oxymoron)IV. Cloze Test 1-5 BABDD 6-10 CBCCAV. Do as you are required.(1) Find the topic sentence of the paragraph.The first sentence of the paragraph(2) Tell the main idea of the following paragraph.The second advantage of most paid work and of some unpaid work is that it gives chances of success and opportunities for ambition. A person’s success is usually measured by his income no matter how dull his work may be, since his work may bring him reputation. Housewives are not as happy as their husbands because the great majority of housework cannot bring as much satisfaction as work of other kinds brings to men and to professional women.。

Supervised learning is a type of machine learning where the algorithm is provided with labeled training data The goal is to learn a function that maps input data to desired outputs based on the provided labels Common examples include classification and regression tasks
Deep learning is a type of machine learning that uses neural networks with multiple layers of hidden units to learn complex patterns and representations from data It is based on biomimetic neural networks and self-organizing mapping networks.
Machine translation is the process of automatically translating text or speech from one language to another using computer algorithms and language data banks This technology has identified the need for human translators in many scenarios
Some challenges associated with deep learning include the requirement for large amounts of labeled data, the complexity of explaining the learned patterns or representations, and the potential for overflow or poor generalization to unseen data

Learning Spatially Localized,Parts-Based Representation Stan Z.Li,XinWen Hou,HongJiang Zhang,QianSheng Cheng.Microsoft Research China,Beijing Sigma Center,Beijing100080,ChinaInstitute of Mathematical Sciences,Peking University,Beijing,ChinaContact:szli@,/szliAbstractIn this paper,we propose a novel method,called local non-negative matrix factorization(LNMF),for learning spa-tially localized,parts-based subspace representation of vi-sual patterns.An objective function is defined to impose lo-calization constraint,in addition to the non-negativity con-straint in the standard NMF[1].This gives a set of bases which not only allows a non-subtractive(part-based)repre-sentation of images but also manifests localized features. An algorithm is presented for the learning of such basis components.Experimental results are presented to compare LNMF with the NMF and PCA methods for face represen-tation and recognition,which demonstrates advantages of LNMF.1IntroductionSubspace analysis helps to reveal low dimensional struc-tures of patterns observed in high dimensional spaces.A specific pattern of interest can reside in a low dimensional sub-manifold in the original input data space of an unnec-essarily high dimensionality.Consider the case ofimage pixels,each taking a value in;there is a huge number of possible configurations:.This space is capable of describing a wide variety of patterns or visual object classes.However,for a specific pattern,such as the human face,the number of admissible configurations is a only tiny fraction of that.In other words,the intrinsic dimension is much lower than.An observation can be considered as a consequence of linear or nonlinear fusion of a small number of hidden or latent variables.Subspace analysis is aimed to derive a rep-resentation that results in such a fusion.In fact,the essence of feature extraction in pattern analysis can be considered as discovering and computing intrinsic low dimension of the pattern from the observation.For these reasons,subspace analysis has been a major research issue in appearance based imaging and vision,such The work presented in the paper was carried out at Microsoft Re-search,China.as object detection and recognition[2,3,4,5,6,7].The significance of is twofold:effective characterization of the pattern and dimension reduction.One approach for learning a subspace representation fora class of image patterns involves deriving a set of basiscomponents for construction of the subspace.The eige-niamge method[2,3,4]uses principal component analy-sis(PCA)[8]performed on a set of representative training data to decorrelate second order moments corresponding to low frequency property.Any image can be represented as a linear combination of these bases.Dimension reduction is achieved by discarding least significant components.Due to the holistic nature of the method,the resulting components are global interpretations,and thus PCA is unable to extract basis components manifesting localized features.However,in many applications,localized features offer advantages in object recognition,including stability to lo-cal deformations,lighting variations,and partial occlusion.Several methods have been proposed recently for localized (spatially),parts-based(non-subtractive)feature extraction.Local feature analysis(LFA)[9],also based on second order statistics,is a method for extracting,from the holis-tic(global)PCA basis,local topographic representation in terms of local features.Independent component analysis [10,11]is a linear non-orthogonal transform.It yields a representation in which unknown linear mixtures of multi-dimensional random variables are made as statistically in-dependent as possible.It not only decorrelates the second order statistics but also reduces higher-order statistical de-pendencies.It is found that independent component of nat-ural scenes are localized edge-likefilters[12].The projection coefficients for the linear combinations in the above methods can be either positive or negative,and such linear combinations generally involve complex cancel-lations between positive and negative numbers.Therefore, these representations lack the intuitive meaning of adding parts to form a whole.Non-negative matrix factorization(NMF)[1]imposes the non-negativity constraints in learning basis images.The pixel values of resulting basis images,as well as coeffi-cients for reconstruction,are all non-negative.This way, 1only non-subtractive combinations are allowed.This en-sures that the components are combined to form a whole in the non-subtractive way.For this reason,NMF is consid-ered as a procedure for learning a parts-based representa-tion[1].However,the additive parts learned by NMF are not necessarily localized,and moreover,we found that the original NMF representation yields low recognition accu-racy,as will be shown.In this paper,we propose a novel subspace method, called local non-negative matrix factorization(LNMF),for learning spatially localized,parts-based representation of visual patterns.Inspired by the original NMF[1],the aim of this work to impose the locality of features in basis com-ponents and to make the representation suitable for tasks where feature localization is important.An objective func-tion is defined to impose the localization constraint,in addi-tion to the non-negativity constraint of[1].A procedure is presented to optimize the objective to learn truly localized, parts-based components.A proof of the convergence of the algorithm is provided.The rest of the paper is organized as follows:Section2 introduces NMF in contrast to PCA.This is followed by the formulation of LNMF.A LNMF learning procedure is presented and its convergence proved.Section3presents experimental results illustrating properties of LNMF and its performance in face recognition as compared to PCA and NMF.2Constrained Non-Negative Matrix FactorizationLet a set of training images be given as an matrix,with each column consisting of the non-negative pixel values of an image.Denote a set of basis images by an matrix.Each image can be represented as a linear combination of the basis images using the approximate factorization(1)where is the matrix of coefficients or weights. Dimension reduction is achieved when.The PCA factorization requires that the basis images (columns of be orthonormal and the rows of be mu-tually orthogonal.It imposes no other constraints than the orthogonality,and hence allows the entries of and to be of arbitrary sign.Many basis images,or eigenfaces in the case of face recognition,lack intuitive meaning,and a linear combination of the bases generally involves complex cancellations between positive and negative numbers.The NMF and LNMF representations allow only positive coef-ficients and thus non-subtractive combinations.2.1NMFNMF imposes the non-negativity constraints instead of the orthogonality.As the consequence,the entries of and are all non-negative,and hence only non-subtractive combi-nations are allowed.This is believed to be compatible to the intuitive notion of combining parts to form a whole,and is how NMF learns a parts-based representation[1].It is alsoconsistent with the physiological fact that thefiring rate are non-negative.NMF uses the divergence of from,defined as(2)as the measure of cost for factorizing into.An NMF factorization is defined as(3)where means that all entries of and are non-negative.reduces to Kullback-Leibler di-vergence when.The above optimization can be done by using multiplicative updaterules[13],for which a matlab program is available at under the“Compu-tational Neuroscience”discussion category.2.2LNMFThe NMF model defined by the constrained minimization of(2)does not impose any constraints on the spatial locality and therefore minimizing the objective function can hardly yield a factorization which reveals local features in the data .Letting,,both being,LNMF is aimed at learning local features by imposing the following three additional constraints on the NMF basis:1.A basis component should not be further decomposedinto more components,so as to minimize the num-ber of basis components required to represent.Letbe a basis vector.Given the existing constraints for all,we wish thatshould be as small as possible so that contains asmany non-zero elements as possible.This can be im-posed by.2.Different bases should be as orthogonal as possible,so as to minimize redundancy between different bases.This can be imposed by.23.Only components giving most important informationshould be retained.Given that every image inhas been normalized into a certain range such as,the total“activity”on each retained com-ponent,defined as the total squared projection coef-ficients summed over all training images,should bemaximized.This is imposed by.The incorporated of the above constraints leads the fol-lowing constrained divergence as the objective function for LNMF:(4)where are some constants.A LNMF factorization is defined as a solution to the constrained minimization of(5).A local solution to the above constrained minimization can be found by using the following three step update rules:(5)(6)(7) 2.3Convergence ProofThe learning algorithm(5)-(7)alternates between updating and updating,which is derived based on a technique in which an objective function is minimized by using an auxiliary function.is said to be an auxiliary function for if andare satisfied.If is an auxiliary function, then is non-increasing when is updated using[14].This is because.Updating:is updated by minimizingwithfixed.An auxiliary function is con-structed for as(8)It is easy to verify.The following proves.Because is a convex function,the following holds for all and:(9)Let(10)Then(11)which is.To minimize w.r.t.,we can update using(12)Such an can be found by letting for all.Because(13)wefind(14)There exists such that(15)where(16)and is a function of and.The result we want to derive from the LNMF learning is the basis,and itself is not so important.Becausewill be normalized by Eq.7,the normalized is regardless of the value as long as.Therefore,we simply replace Eq.(15)by(5).3Updating:is updated by minimizingwithfixed.The auxiliary function foris(17)We can prove andlikewise.By letting,wefind(18)Because and is an approximately orthog-onal basis,and,there must be.Therefore,we can always set the ratio to be not too large(e.g.)so that and thus.Therefore,we have Eq.(6).From the above analysis,we conclude that the three step update rules(5)-(7)results in a sequence of non-increasing values of,and hence converges to a local mini-mum of it.2.4Face Recognition in SubspaceFace recognition in the PCA,NMF or LNMF linear sub-space is performed as follows:1.Feature extraction.Let be the mean of training im-ages.Each training face image is projected into thelinear space as a feature vectorwhich is then used as a prototype feature point.Aquery face image to be classified is represented byits projection into the space as.2.Nearest neighbor classification.The Euclidean dis-tance between the query and each prototype,,is calculated.The query is classified to the class towhich the closest prototype belongs.3Experiments3.1Data PreparationThe Cambridge ORL face database is used for deriving PCA,NMF and LNMF bases.There are400images ()of40persons,10images per person(Fig.1shows the10images of one person).The images are taken at different times,varying lighting slightly,facial expressions (open/closed eyes,smiling/non-smiling)and facial details (glasses/no-glasses).All the images are taken against a dark homogeneous background.The faces are in up-right posi-tion of frontal view,with slight left-right out-of-plane rota-tion.Each image is linearly stretched to the full range of pixel values of[0,255].Figure1:Face examples from ORL database.The set of the10images for each person is randomly partitioned into a training subset of5images and a test set of the other5.The training set is then used to learn basis components,and the test set for evaluate.All the compared methods take the same training and test data.3.2Learning Basis ComponentsLNMF,NMF and PCA representations withbasis components are com-puted from the training set.The matlab package from is used for NMF. NMF converges about5-times faster than LNMF.Fig.2 shows the resulting LNMF and NMF components for subspaces of dimensions25,49and81.Higher pixel values are in in darker color;the components in each LNMF basis set have been ordered(left-right then top-down) according to the significance value.The NMF bases are as holistic as the PCA basis(eigenfaces)for the training set.We notice the result presented in[1]does not appear so,perhaps because the faces used for producing that result are well aligned.The LNMF procedure learns basis components which not only lead to non-subtractive representations,but also manifest localized features and thus truly parts-based representations.Also,we see that as the dimension(number of components)increases,the features formed in the LNMF components become more localized.4Figure2:LNMF(left)and NMF(right)bases of dimen-sions25(row1),49(row2)and81(row3).Every basis component is of size and the displayed images are re-sized tofit the paper format.The LNMF representation is both parts-based and local,whereas NMF is parts-based but holistic.3.3ReconstructionFig.3shows reconstructions in the LNMF,NMF and PCA subspaces of various dimensions for a face image in the test set which corresponds to the one in the middle of row1 of Fig.1.As the dimension is increased,more details are recovered.We see that while NMF and PCA reconstruc-tions look similar in terms of the smoothness and texture of the reconstructed images,with PCA presenting better reconstruction quality than NMF.Surprisingly the LNMF representation,which is based on more localized features, provides smoother reconstructions than NMF and PCA. 3.4Face RecognitionThe LNMF,NMF and PCA representations are compara-tively evaluated for face recognition using the images from the test set.The recognition accuracy,defined as the per-centage of correctly recognized faces,is used as the per-formance measure.Tests are done with varying numberofFigure3:Reconstructions of the face image in the(left to right)25,49,81and121dimensional(top-down)LNMF, NMF,and PCAsubspaces.Figure4:Examples of random occluding patches of sizes (from left to right)10x10,20x20,...,50x50,60x60.basis components,with or without occlusion.The occlu-sion is simulated in an image by using a white patch of size with at a random location;see Fig.4for examples.Figs.5and6show recognition accuracy curves under various conditions.Fig.5compares the three representa-tions in terms of the recognition accuracies versus the num-ber of basis components for.The LNMF yields the best recognition accuracy,slightly better than PCA whereas the original NMF gives very low accuracy.Fig.6compares the three representations un-der varying degrees of occlusion and with varying num-ber of basis components,in terms of the recognition ac-curacies versus the size of occluding patch for.As we see,although PCA yields more favorable results than LNMF when the patch size is small, the better stability of the LNMF representation under partial occlusion becomes clear as the patch size increases.4ConclusionIn this paper,we have proposed a new method,local non-negative matrix factorization(LNMF),for learning spatially5Figure5:Recognition accuracies as function of the num-ber(in5x5,6x6,...,11x11)of basis components used,for the LNMF(solid)and NMF(dashed)and PCA(dot-dashed) representations.Figure6:Recognition accuracies versus the size(in10x10, 20x20,...,60x60)of occluding patches,with25,49,81,121 basis components(left-right,then top-down),for the LNMF (solid)and NMF(dashed)and PCA(dot-dashed)represen-tations.localized,part-based subspace representation of visual pat-terns.The work is aimed to learn localized of features in NMF basis components suitable for tasks such as face recognition.An algorithms is presented for the learning and its convergence proved.Experimental results have shown that we have achieved our objectives:LNMF derives bases which are better suited for a localized representation than PCA and NMF,and leads to better recognition results than the existing methods.The LNMF and NMF learning algorithms are local min-imizers.They give different basis components from differ-ent initial conditions.We will investigate how this affects the recognition rate.Further future work includes the fol-lowing topics.Thefirst is to develop algorithms for faster convergence and better solution in terms of minimizing theobjective function.The second is to investigate the abil-ity of the model to generalize,i.e.how the constraints,the non-negativity and others,are satisfied for data not seen in the training set.The third is to compare with other methods for learning spatially localized features such as LFA[9]and ICA[12].References[1] D.D.Lee and H.S.Seung,““Learning the parts of objectsby non-negative matrix factorization”,”Nature,vol.401,pp.788–791,1999.[2]L.Sirovich and M.Kirby,““Low-dimensional procedurefor the characterization of human faces”,”Journal of theOptical Society of America A,vol.4,no.3,pp.519–524,March1987.[3]M.Kirby and L.Sirovich,““Application of the Karhunen-Loeve procedure for the characterization of human faces”,”IEEE Transactions on Pattern Analysis and Machine Intelli-gence,vol.12,no.1,pp.103–108,January1990.[4]Matthew A.Turk and Alex P.Pentland,““Face recognitionusing eigenfaces.”,”in Proceedings of IEEE Computer Soci-ety Conference on Computer Vision and Pattern Recognition,Hawaii,June1991,pp.586–591.[5]David Beymer,Amnon Shashua,and Tomaso Poggio,““Ex-ample based image analysis and synthesis”,” A.I.Memo1431,MIT,1993.[6] A.P.Pentland,B.Moghaddam,and T.Starner,““View-based and modular eigenspaces for face recognition”,”inProceedings of IEEE Computer Society Conference on Com-puter Vision and Pattern Recognition,1994,pp.84–91.[7]H.Murase and S.K.Nayar,““Visual learning and recogni-tion of3-D objects from appearance”,”International Journalof Computer Vision,vol.14,pp.5–24,1995.[8]K.Fukunaga,Introduction to statistical pattern recognition,Academic Press,Boston,2edition,1990.[9]P.Penev and J.Atick,““Local feature analysis:A generalstatistical theory for object representation”,”Neural Systems,vol.7,no.3,pp.477–500,1996.[10] C.Jutten and J.Herault,““Blind separation of sources,partI:An adaptive algorithm based on neuromimetic architec-ture”,”Signal Processing,vol.24,pp.1–10,1991.[11]on,““Independent component analysis-a new con-cept?”,”Signal Processing,vol.36,pp.287–314,1994.[12] A.J.Bell and T.J.Sejnowski,““The‘independent compo-nents’of natural scenes are edgefilters”,”Vision Research,vol.37,pp.3327–3338,1997.[13] D.D.Lee and H.S.Seung,““Algorithms for non-negativematrix factorization”,”in Proceedings of Neural InformationProcessing Systems,2000.[14] A.P.Dempster,ird,and D.B.Bubin,““Maxi-mum likelihood from imcomplete data via EM algorithm”,”Journal of the Royal Statistical Society,Series B,vol.39,pp.1–38,1977.6。

11.2.2 Learner’s factors
• Learner’s factors mainly cover the following aspects:
• Motivation • Language aptitude • Age • Learning strategy
11.2.1 Social factors
Discussing Task
Group work: Have a discussion on the following questions.
1. How does (second) language acquisition take place?
2. How is foreign language learning different from second language acquisition?
The Symbolic Function of Words
Teaching Aims
1. To know what SLA is, and how the theories account for SLA. 2. To understand different factors affecting SLA 3. To know how learner’s language is analyzed 4. To cultivate students’ research awareness and innovative spirit in discovering and solving problems by analyzing the different kinds of errors and individual differeneces in SLA.