下载文档原格式
机器学习经典书目汇总本文总结了机器学习的经典书籍，包括数学基础和算法理论的书籍。
入门书单《数学之美》作者吴军大家都很熟悉。
以极为通俗的语言讲述了数学在机器学习和自然语言处理等领域的应用。
《Programming Collective Intelligence》（《集体智慧编程》）作者Toby Segaran也是《BeautifulData : The Stories Behind Elegant Data Solutions》（《数据之美：解密优雅数据解决方案背后的故事》）的作者。
这本书最大的优势就是里面没有理论推导和复杂的数学公式，是很不错的入门书。
目前中文版已经脱销，对于有志于这个领域的人来说，英文的pdf是个不错的选择，因为后面有很多经典书的翻译都较差，只能看英文版，不如从这个入手。
还有，这本书适合于快速看完，因为据评论，看完一些经典的带有数学推导的书后会发现这本书什么都没讲，只是举了很多例子而已。
《Algorithms of the Intelligent Web》（《智能web算法》）作者Haralambos Marmanis、Dmitry Babenko。
这本书中的公式比《集体智慧编程》要略多一点，里面的例子多是互联网上的应用，看名字就知道。
不足的地方在于里面的配套代码是BeanShell而不是python或其他。
总起来说，这本书还是适合初学者，与上一本一样需要快速读完，如果读完上一本的话，这一本可以不必细看代码，了解算法主要思想就行了。
《统计学习方法》作者李航，是国内机器学习领域的几个大家之一，曾在MSRA 任高级研究员，现在华为诺亚方舟实验室。
书中写了十个算法，每个算法的介绍都很干脆，直接上公式，是彻头彻尾的“干货书”。
每章末尾的参考文献也方便了想深入理解算法的童鞋直接查到经典论文；本书可以与上面两本书互为辅助阅读。
《Machine Learning》（《机器学习》）作者Tom Mitchell是CMU的大师，有机器学习和半监督学习的网络课程视频。
Introduction to Algorithms October 29, 2005 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik D. Demaine and Charles E. Leiserson Handout 18Problem Set 4 SolutionsProblem 4-1. TreapsIf we insert a set of n items into a binary search tree using T REE-I NSERT, the resulting tree may be horribly unbalanced. As we saw in class, however, we expect randomly built binary search trees to be balanced. (Precisely, a randomly built binary search tree has expected height O(lg n).) Therefore, if we want to build an expected balanced tree for a fixed set of items, we could randomly permute the items and then insert them in that order into the tree.What if we do not have all the items at once? If we receive the items one at a time, can we still randomly build a binary search tree out of them?We will examine a data structure that answers this question in the affirmative. A treap is a binary search tree with a modified way of ordering the nodes. Figure 1 shows an example of a treap. As usual, each item x in the tree has a key key[x]. In addition, we assign priority[x], which is a random number chosen independently for each x. We assume that all priorities are distinct and also that all keys are distinct. The nodes of the treap are ordered so that (1) the keys obey the binary-search-tree property and (2) the priorities obey the min-heap order property. In other words,•if v is a left child of u, then key[v]<key[u];•if v is a right child of u, then key[v]>key[u]; and•if v is a child of u, then priority(v)>priority(u).(This combination of properties is why the tree is called a “treap”: it has features of both a binary search tree and a heap.)Figure 1: A treap. Each node x is labeled with key[x]:p riority[x]. For example, the root has key G and priority 4.It helps to think of treaps in the following way. Suppose that we insert nodes x1,x2,...,x n, each with an associated key, into a treap in arbitrary order. Then the resulting treap is the tree that wouldhave been formed if the nodes had been inserted into a normal binary search tree in the order given by their (randomly chosen) priorities. In other words, priority[x i]<priority[x j]means that x i is effectively inserted before x j.(a) Given a set of nodes x1,x2,...,x n with keys and priorities all distinct, show that thereis a unique treap with these nodes.Solution:Prove by induction on the number of nodes in the tree. The base case is a tree withzero nodes, which is trivially unique. Assume for induction that treaps with k−1orfewer nodes are unique. We prove that a treap with k nodes is unique. In this treap, theitem x with minimum priority must be at the root. The left subtree has items with keys<key[x]and the right subtree has items with keys >key[x]. This uniquely defines theroot and both subtrees of the root. Each subtree is a treap of size ≤k−1, so they areunique by induction.Alternatively, one can also prove this by considering a treap in which nodes are inserted in order of their priority. Assume for induction that the treap with the k−1nodes with smallest priority is unique. For k=0t his is trivially true. Now considerthe treap with the k nodes with smallest priority. Since we know that the structureof a treap is the same as the structure of a binary search tree in which the keys areinserted in increasing priority order, the treap with the k nodes with smallest priorityis the same as the treap with the k−1nodes with smallest priority after inserting thek-th node. Since BST insert is a deterministic algorithm, there is only one place thek-th node could be inserted. Therefore the treap with k nodes is also unique, provingthe inductive hypothesis.(b) Show that the expected height of a treap is O(lg n), and hence the expected time tosearch for a value in the treap is O(lg n).Solution: The idea is to realize that a treap on n nodes is equivalent to a randomlybuilt binary search tree on n nodes. Recall that assigning priorities to nodes as theyare inserted into the treap is the same as inserting the n nodes in the increasing orderdefined by their priorities. So if we assign the priorities randomly, we get a randomorder of n priorities, which is the same as a random permutation of the n inputs, sowe can view this as inserting the n items in random order.The time to search for an item is O(h)where h is the height of the tree. As we saw inlecture, E[h]=O(lg n). (The expectation is taken over permutations of the n nodes,i.e., the random choices of the priorities.)Let us see how to insert a new node x into an existing treap. The first thing we do is assign x a random priority priority[x]. Then we call the insertion algorithm, which we call T REAP-I NSERT, whose operation is illustrated in Figure 2.(e) (f)Figure 2: Operation of T REAP-I NSERT. As in Figure 1, each node x is labeled with key[x]: priority[x]. (a) Original treap prior to insertion. (b) The treap after inserting a node with key C and priority 25. (c)–(d) Intermediate stages when inserting a node with key D and priority 9.(e) The treap after insertion of parts (c) and (d) is done. (f) The treap after inserting a node with key F and priority 2.(c) Explain how T REAP-I NSERT works. Explain the idea in English and give pseudocode.(Hint: Execute the usual binary search tree insert and then perform rotations to restorethe min-heap order property.)Solution: The hint gives the idea: do the usual binary search tree insert and thenperform rotations to restore the min-heap order property.T REAP-I NSERT (T,x)inserts x into the treap T(by modifying T). It requires that xhas defined key and priority values. We have used the subroutines T REE-I NSERT,R IGHT-R OTATE, and R IGHT-R OTATE as defined in CLRS.T REAP-I NSERT (T,x)1T REE-I NSERT (T,x)2 while x =root[T]and priority[x]<priority[p[x]]3 do if x=left[p[x]]4 then R IGHT-R OTATE (T,p[x])5 else L EFT-R OTATE (T,p[x])Note that parent pointers simplify the code but are not necessary. Since we only needto know the parent of each node on the path from the root to x(after the call toT REE-I NSERT), we can keep track of these ourselves.(d) Show that the expected running time of T REAP-I NSERT is O(lg n). Solution:T REAP-I NSERT first inserts an item in the tree using the normal binary search treeinsert and then performs a number of rotations to restore the min-heap property.The normal binary-search-tree insertion algorithm T REE-I NSERT always places thenew item at a new leaf of tree. Therefore the time to insert an item into a treap isproportional to the height of a randomly built binary search tree, which as we saw inlecture is O(lg n)in expectation.The maximum number of rotations occurs when the new item receives a priority lessthan all priorities in the tree. In this case it needs to be rotated from a leaf to the root.An upper bound on the number of rotations is therefore the height of a randomly builtbinary search tree, which is O(lg n)in expectation. (We will see that this is a fairlyloose bound.) Because each rotation take constant time, the expected time to rotate isO(lg n).Thus the expected running time of T REAP-I NSERT is O(lg n+lg n)=O(lg n).T REAP-I NSERT performs a search and then a sequence of rotations. Although searching and rotating have the same asymptotic running time, they have different costs in practice. A search reads information from the treap without modifying it, while a rotation changes parent and child pointers within the treap. On most computers, read operations are much faster than write operations. Thus we would like T REAP-I NSERT to perform few rotations. We will show that the expected number of rotations performed is bounded by a constant (in fact, less than 2)!(a) (b)Figure 3: Spines of a binary search tree. The left spine is shaded in (a), and the right spine is shaded in (b).In order to show this property, we need some definitions, illustrated in Figure 3. The left spine of a binary search tree T is the path which runs from the root to the item with the smallest key. In other words, the left spine is the maximal path from the root that consists only of left edges. Symmetrically, the right spine of T is the maximal path from the root consisting only of right edges. The length of a spine is the number of nodes it contains.(e) Consider the treap T immediately after x is inserted using T REAP-I NSERT. Let Cbe the length of the right spine of the left subtree of x. Let D be the length of theleft spine of the right subtree of x. Prove that the total number of rotations that wereperformed during the insertion of x is equal to C+D.Solution: Prove the claim by induction on the number of rotations performed. Thebase case is when x is the parent of y. Performing the rotation so that y is the new rootgives y exactly one child, so C+D=1.Assume for induction that the number of rotations k performed during the insertionof x equals C+D. The base case is when 0 rotations are necessary and x is insertedas a leaf. Then C+D=0. To prove the inductive step, we show that if after k−1rotations C+D=k−1, then after k rotations C+D=k. Draw a picture of aleft and right rotation and observe that C+D increases by 1 in each case. Let y bethe parent of x, and suppose x is a left child of y. After performing a right rotation, ybecomes the right child of x, and the previous right child of x becomes the left childof y. That is, the left spine of the right subtree of x before the rotation is tacked onto y, so the length of that spine increases by one. The left subtree of x is unaffectedby a right rotation. The case of a left rotation is symmetric. Therefore after one morerotation C+D increases by one and k=C+D, proving the inductive hypothesis.We will now calculate the expected values of C and D. For simplicity, we assume that the keys are 1,2,...,n. This assumption is without loss of generality because we only compare keys.�For two distinct nodes x and y , let k = key [x ]and i = key [y ], and define the indicator random variableX 1 if y is a node on the right spine of the left subtree of x (in T ),i,k =0 otherwise .(f) Show that X i,k =1if and only if (1) priority [y ]> priority [x ], (2) key [y ]< key [x ], and(3) for every z such that key [y ]< key [z ]< key [x ],we have p riority [y ]< priority [z ].Solution:To prove this statement, we must prove both directions of the “if and only if”. Firstwe prove the “if” direction. We prove that if (1) priority [y ]> priority [x ], (2) key [y ]<key [x ], and (3) for every z such that key [y ]< key [z ]< key [x ]are true, priority [y ]< priority [z ], then X i,k =1. We prove this by contradiction. Assume that X i,k =0. That is, assume that y is not on the right spine of the left subtree of x . We show thatthis leads to a contradiction. If y is not on the right spine of the left subtree of x ,it could be in one of three places:1. Suppose y is in the right subtree of x . This contradicts condition (2) becausekey [y ]< key [x ].2. Suppose y is not in one of the subtrees of x . Then x and y must share somecommon ancestor z . Since key [y ]< key [x ], we know that y is in the left subtreeof z and x is in the right subtree of z and key [y ]< key [z ] < key [x ]. Since y isbelow z in the tree, priority [z ]< priority [x ]and priority [z ]< priority [y ]. Thiscontradicts condition (3).3. Suppose that y is in the left subtree of x but not on the right spine of the leftsubtree of x . This implies that there exists some ancestor z of y in the left subtreeof x such that y is in the left subtree of z . Hence key [y ]< key [z ]< key [x ]. Sincez is an ancestor of y , priority [z ]< priority [y ], which contradicts condition (3).All possible cases lead to contradictions, and so X i,k =1.Now for the “only if” part. We prove that if X i,k =1, then statements (1), (2), and (3) are true. If X i,k =1, then y is in the right spine of the left subtree of x . Since y is ina subtree of x , y must have been inserted after x ,so p riority [y ]> priority [x ], proving(1). Since y is in the left subtree of x , key [y ]< key [x ], proving (2). We prove (3) by contradiction: suppose that X i,k =1and there exists a z such that key [y ]< key [z ]< key [x ]and priority [z ]< priority [y ]. In other words, z was inserted before y . There arethree possible cases that satisfy the condition key [z ]< key [x ]:1. Suppose z is in the right spine of the left subtree of x .For y to be inserted into theright spine of the left subtree of x , it will have to be inserted into the right subtreeof z . Since key [y ]< key [z ], this leads to a contradiction.2. Suppose z is in the left subtree of x but not in the right spine. This implies thatz is in the left subtree of some node z in the right spine of x . Therefore for y tobe inserted into the right spine of the left subtree of x , it must be inserted into theright subtree of z . This leads to a contradiction by reasoning similar to case 1.3. Suppose that z is not in one of the subtrees of x . Then z and x have a commonancestor z such that z is in the left subtree of z and x is in the right subtree of x .This implies key [z ] < key [z ] < key [x ]. Since key [y ]< key [z ]< key [z ], y cannotbe inserted into the right subtree of z . Therefore it cannot be inserted in a subtreeof x , which is a contradiction.Therefore there can be no z such that key [y ] < key [z ] < key [x ] and priority [z ] < priority [y ]. This proves statement (3). We have proven both the “if” and “only if” directions, proving the claim.(g) Show that(k − i − 1)! 1 Pr {X i,k =1} == . (k − i +1)! (k − i +1)(k − i )Solution: We showed in the previous part that X i,k =1if and only if the priorities of the items between y and x are ordered in a certain way. Since all orderings are equally likely, to calculate the probability we count the number of permutations of priorities that obey this order and divide by the number of total number of priority permutations. We proved in (e) that whether or not X i,k =1depends only on the relative ordering of the priorities of y , x , and all z such that key [y ] < key [z ] < key [x ]. Since we assumed that the keys of the items come from {1,...,n }, the keys of the items in question are i,i +1,i +2,...,k − 1,k . There are (k − i +1)!permutations of the priorities of these items. Of these permutations, the ones for which X i,k =1are those where i has minimum priority, k has the second smallest priority, and the priorities of the remaining k − i − 1 items follow in any order. There are (k − i − 1)! of these permutations. Thus the probability that we get a “bad” order is (k −i −1)!/(k −i +1)!= 1/(k − i )(k − i +1).(h) Show thatk −1 � 1 1 E [C ]= =1− . j (j +1)k j =1 Solution:X For a node x with key k , E [C ]is the expected number of nodes in the right spine of the left subtree of x . This equals the sum of the expected value of the random variables i,k for all i in the tree. Since X i,k =0for all nodes i ≥ k , we only need to consider i <k .� � � k −1 �k −1 � E [C ]=E [X i,k ]=E X i,k i =1 i =1 k −1 =Pr {X i,k =1}i =1 k −1 � 1 =(k − i )(k − i +1) i =1 k −1 � 1= j (j +1)j =1 1To simplify this sum, observe that j (j 1+1) = j +1−j = − 1 . Therefore the sumj (j +1) j j +1 telescopes and we have 1 E [C ]=1− . kIf you didn’t see this, you could have proven that the equationk −1 � 1 1 =1− j (j +1)k j =1 holds by induction on k . In the proving the inductive hypothesis you might have 11discovered 1 − = .j j +1 j (j +1)(i) Use a symmetry argument to show that1 E [D ]=1− . n − k +1Solution: The idea is to consider the treap produced if the ordering relationship amongthe keys is reversed. That is, for all items x , leave priority [x ]unchanged but replacekey [x ]with n − key [x ]+1.Let T be the binary tree we get when inserting the items (in increasing order of priority) using the original keys. Once we remap the keys and insert them into a newbinary search tree, we get a tree T whose shape is the mirror image of the shape ofT . (reflected left to right). Consider the item x with key k in T and therefore has key n − k +1 i n T . The length of the left spine of x ’s right subtree in T has becomethe length of the right spine of x ’s left subtree in T . We know by part (g) that the expected length of the right spine of a left subtree of a node y is 1− 1/idkey [y ],so the expected length of the right spine of the left subtree of x in T is 1− 1/(n − k +1), which means that 1 E [D ]=1− . n − k +1(j) Conclude that the expected number of rotations performed when inserting a node into a treap is less than 2.Solution:11 E [number of rotations ]= E [C +D ]=E [C ]+E [D ]=1− +1− k n − k +1 ≤ 1+1=2. Problem 4-2. Being balancedCall a family of trees balanced if every tree in the family has height O (lg n ), where n is the number of nodes in the tree. (Recall that the height of a tree is the maximum number of edges along any path from the root of the tree to a leaf of the tree. In particular, the height of a tree with just one node is 0.)For each property below, determine whether the family of binary trees satisfying that property is balanced. If you answer is “no”, provide a counterexample. If your answer is “yes”, give a proof (hint: it should be a proof by induction). Remember that being balanced is an asymptotic property, so your counterexamples must specify an infinite set of trees in the family, not just one tree. (a) Every node of the tree is either a leaf or it has two children.Solution: No. Counterexample is a right chain, with each node having a leaf hanging off to the left(b) The size of each subtree can be written as 2k − 1, where k is an integer (k is not the same for each subtree).Solution: Yes.Consider any subtree with root r . We know from the condition that the size of this subtree is 2k 1 − 1. We also know that the size of the subtree rooted at the left child of r is 2k 2 − 1, and the size of the subtree rooted at the right child of r is 2k 3 − 1. But the size of the subtree at r is simply the node r together with the nodes in the left and right subtrees. This leads to the equation 2k 1 − 1=1+(2k 2 − 1)+(2k 3 − 1),or 2k 1 =2k 2 +2k 3. Now we show that k 2 =k 3. This is easy to see if you consider the binary representations of k 1, k 2, and k 3.Otherwise, if we assume WLOG that k 2 ≤ k 3, then we have 2k 1−k 2 =1+2k 3−k 2. 2Now, the only pair of integer powers of 2 that could satisfy this equation are 21 and 0 . Thus k 3 − k 2 =0,or k 2 =k 3, and the left and right subtrees of r have an equal number of nodes. It follows that the tree is perfectly balanced.(c) There is a constant c>0such that, for each node of the tree, the size of the smallerchild subtree of this node is at least c times the size of the larger child subtree.Solution:Yes1. The proof is by induction. Assume that the two subtrees of x with n nodes in itssubtree has two children y and z with subtree sizes n1 and n2. By inductive hypothesis,the height of y’s subtree is at most d lg n1 and the height of z’s subtree is at most d lg n2for some constant d. We now prove that the height of x’s subtree is at most d lg n.Assume wlog that n1 ≥n2. Therefore, by the problem statement, we have n2 ≥cn1.Therefore, we have n=n1 +n2 +1≥(1+c)n1 +1≥(1+c)n1 and the height hof x’s subtree is d lg n1 +1≤d lg(n/(c+1))+1≤d lg n−d lg(1+c)+1≤d lg nif d lg(1+c)≥1. Therefore, for sufficiently large d, the height of a tree with n nodesis at most d lg n.(d) There is a constant c such that, for each node of the tree, the heights of its childrensubtrees differ by at most c.Solution: Yes1. Let n(h)be the minimum number of nodes that a tree of height h thatsatisfies the stated property can have. We show by induction that n(h)≥(1+α)h−1,for some constant 0<α≤1. We can then conclude that for a tree with n nodes,h≤log1+α(n+1)=O(lg n).For the base case, a subtree of height 0has a single node, and 1≥(1+α)0 −1forany constant α≤1.In the inductive step, assume for all trees of height k<h, that the n(k)≥(1+α)k−1.Now consider a tree of height h, and look at its two subtrees. We know one subtreemust have height h−1, and the other must have height at least h−1−c. Therefore,we known(h)≥n(h−1)+n(h−1−c)+1.Using the inductive hypothesis, we getn(h) ≥(1+α)h−1 −1+(1+α)h−1−c−1+1≥2(1+α)h−1−c−1.Suppose we picked αsmall enough so that (1+α)<21/(c+1). Then (1+α)c+1 <2.Therefore, we getn(h)≥2(1+α)h−1−c−1≥(1+α)h−1.1Note that in this problem we assume that a nil pointer is a subtree of size 0, and so a node with only one child has two subtrees, one of which has size 0. If you assume that a node with only one child has only one subtree, then the answer to this problem part is “no”.�11Handout18: Problem Set4SolutionsTherefore, we satisfy the inductive hypothesis.Note that if we plug this value for αback into h≤log1+α(n+1),we getlg(n+1)h≤≤(c+1)l g(n+1).lg(1+2c+1)(e) The average depth of a node is O(lg n). (Recall that the depth of a node x is thenumber of edges along the path from the root of the tree to x.)Solution: No.√Consider a tree with n−n nodes organized as a complete binary tree, and the other √ √n nodes sticking out as a chain of length n from the balanced tree. The height of√√√the tree is lg(n−n)+n=Ω(n), while the average depth of a node is at most�√√ √(1/n)n·(n+lg n)+(n−n)·lg n√√=(1/n)(n+n lg n+n lg n−nlgn)=(1/n)(n+n lg n)=O(lg n)√Thus, we have a tree with average node depth O(lg n), but height Ω(n).。
This file contains the exercises,hints,and solutions for Chapter 9of the book ”Introduction to the Design and Analysis of Algorithms,”2nd edition,byA.Levitin.The problems that might be challenging for at least some students are marked by ;those that might be difficult for a majority of students are marked by .Exercises 9.11.Give an instance of the change-making problem for which the greedy al-gorithm does not yield an optimal solution.2.Write a pseudocode of the greedy algorithm for the change-making prob-lem,with an amount n and coin denominations d 1>d 2>...>d m as its input.What is the time efficiency class of your algorithm?3.Consider the problem of scheduling n jobs of known durations t 1,...,t n for execution by a single processor.The jobs can be executed in any order,one job at a time.You want to find a schedule that minimizes the total time spent by all the jobs in the system.(The time spent by one job in the system is the sum of the time spent by this job in waiting plus the time spent on its execution.)Design a greedy algorithm for this problem. Does the greedy algo-rithm always yield an optimal solution?4.Design a greedy algorithm for the assignment problem (see Section 3.4).Does your greedy algorithm always yield an optimal solution?5.Bridge crossing revisited Consider the generalization of the bridge cross-ing puzzle (Problem 2in Exercises 1.2)in which we have n >1people whose bridge crossing times are t 1,t 2,...,t n .All the other conditions of the problem remain the same:at most two people at the time can cross the bridge (and they move with the speed of the slower of the two)and they must carry with them the only flashlight the group has.Design a greedy algorithm for this problem and find how long it willtake to cross the bridge by using this algorithm.Does your algorithm yields a minimum crossing time for every instance of the problem?If it does–prove it,if it does not–find an instance with the smallest number of people for which this happens.6.Bachet-Fibonacci weighing problem Find an optimal set of n weights {w 1,w 2,...,w n }so that it would be possible to weigh on a balance scale any integer load in the largest possible range from 1to W ,provided a. weights can be put only on the free cup of the scale.b. weights can be put on both cups of the scale.1课后答案网 w w w .k h d a w .c o m7.a.Apply Prim’s algorithm to the following graph.Include in the priority queue all the vertices not already in the tree.b.Apply Prim’s algorithm to the following graph.Include in the priority queue only the fringe vertices (the vertices not in the current tree which are adjacent to at least one tree vertex).8.The notion of a minimum spanning tree is applicable to a connected weighted graph.Do we have to check a graph’s connectivity before ap-plying Prim’s algorithm or can the algorithm do it by itself?9.a.How can we use Prim’s algorithm to find a spanning tree of a connected graph with no weights on its edges?b.Is it a good algorithm for this problem?10. Prove that any weighted connected graph with distinct weights hasexactly one minimum spanning tree.11.Outline an efficient algorithm for changing an element’s value in a min-heap.What is the time efficiency of your algorithm?2课后答案网 w h d a w .c o mHints to Exercises 9.11.As coin denominations for your counterexample,you may use,among a multitude of other possibilities,the ones mentioned in the text:d 1=7,d 2=5,d 3=1.2.You may use integer divisions in your algorithm.3.Considering the case of two jobs might help.Of course,after forming a hypothesis,you will have to either prove the algorithm’s optimality for an arbitrary input or find a specific counterexample showing that it is not the case.4.You can apply the greedy approach either to the entire cost matrix or to each of its rows (or columns).5.Simply apply the greedy approach to the situation at hand.You may assume that t 1≤t 2≤...≤t n .6.For both versions of the problem,it is not difficult to get to a hypothesis about the solution’s form after considering the cases of n =1,2,and 3.It is proving the solutions’optimality that is at the heart of this problem.7.a.Trace the algorithm for the graph given.An example can be found in the text of the section.b.After the next fringe vertex is added to the tree,add all the unseen vertices adjacent to it to the priority queue of fringe vertices.8.Applying Prim’s algorithm to a weighted graph that is not connected should help in answering this question.9.a.Since Prim’s algorithm needs weights on a graph’s edges,some weights have to be assigned.b.Do you know other algorithms that can solve this problem?10.Strictly speaking,the wording of the question asks you to prove two things:the fact that at least one minimum spanning tree exists for any weighted connected graph and the fact that a minimum spanning tree is unique if all the weights are distinct numbers.The proof of the former stems from the obvious observation about finiteness of the number of spanning trees for a weighted connected graph.The proof of the latter can be obtained by repeating the correctness proof of Prim’s algorithm with a minor adjustment at the end.11.Consider two cases:the key’s value was decreased (this is the case needed for Prim’s algorithm)and the key’s value was increased.3课后答案网 w w w .k h d a w .c o mSolutions to Exercises 9.11.Here is one of many such instances:For the coin denominations d 1=7,d 2=5,d 3=1and the amount n =10,the greedy algorithm yields one coin of denomination 7and three coins of denomination 1.The actual optimal solution is two coins of denomination 5.2.Algorithm Change (n,D [1..m ])//Implements the greedy algorithm for the change-making problem //Input:A nonnegative integer amount n and//a decreasing array of coin denominations D//Output:Array C [1..m ]of the number of coins of each denomination //in the change or the ”no solution”messagefor i ←1to m doC [i ]← n/D [i ]n ←n mod D [i ]if n =0return Celse return ”no solution”The algorithm’s time efficiency is in Θ(m ).(We assume that integer di-visions take a constant time no matter how big dividends are.)Note also that if we stop the algorithm as soon as the remaining amount becomes 0,the time efficiency will be in O (m ).3.a.Sort the jobs in nondecreasing order of their execution times and exe-cute them in that order.b.Yes,this greedy algorithm always yields an optimal solution.Indeed,for any ordering (i.e.,permutation)of the jobs i 1,i 2,...,i n ,the total time in the system is given by the formula t i 1+(t i 1+t i 2)+...+(t i 1+t i 2+...+t i n )=nt i 1+(n −1)t i 2+...+t i n .Thus,we have a sum of numbers n,n −1,...,1multiplied by “weights”t 1,t 2,...t n assigned to the numbers in some order.To minimize such a sum,we have to assign smaller t ’s to larger numbers.In other words,the jobs should be executed in nondecreasing order of their execution times.Here is a more formal proof of this fact.We will show that if jobs are ex-ecuted in some order i 1,i 2,...,i n ,in which t i k >t i k +1for some k,then the total time in the system for such an ordering can be decreased.(Hence,no such ordering can be an optimal solution.)Let us consider the other job ordering,which is obtained by swapping the jobs k and k +1.Obvi-ously,the time in the systems will remain the same for all but these two 4课后答案网 w w w .k h d a w .c o mjobs.Therefore,the difference between the total time in the system for the new ordering and the one before the swap will be[(k −1j =1t i j +t i k +1)+(k −1j =1t i j +t i k +1+t i k )]−[(k −1j =1t i j +t i k )+(k −1j =1t i j +t i k +t i k +1)]=t i k +1−t i k <0.4.a.The all-matrix version:Repeat the following operation n times.Select the smallest element in the unmarked rows and columns of the cost matrix and then mark its row and column.The row-by-row version:Starting with the first row and ending with the last row of the cost matrix,select the smallest element in that row which is not in a previously marked column.After such an element is selected,mark its column to prevent selecting another element from the same col-umn.b.Neither of the versions always yields an optimal solution.Here isa simple counterexample:C = 122100 5.Repeat the following step n −2times:Send to the other side the pair of two fastest remaining persons and then return the flashlight with the fastest person.Finally,send the remaining two people together.Assuming that t 1≤t 2≤...≤t n ,the total crossing time will be equal to (t 2+t 1)+(t 3+t 1)+...+(t n −1+t 1)+t n =ni =2t i +(n −2)t 1=n i =1t i +(n −3)t 1.Note:For an algorithm that always yields a minimal crossing time,seeGünter Rote,“Crossing the Bridge at Night,”EATCS Bulletin,vol.78(October 2002),241—246.The solution to the instance of Problem 2in Exercises 1.2shows that the greedy algorithm doesn’t always yield the minimal crossing time for n >3.No smaller counterexample can be given as a simple exhaustive check for n =3demonstrates.(The obvious solution for n =2is the one generated by the greedy algorithm as well.)5课后答案网 w w w .kh d a w .c o m6.a.Let’s apply the greedy approach to the first few instances of the problem in question.For n =1,we have to use w 1=1to balance weight 1.For n =2,we simply add w 2=2to balance the first previously unattainable weight of 2.The weights {1,2}can balance every integral weights up to their sum 3.For n =3,in the spirit of greedy thinking,we take the next previously unattainable weight:w 3=4.The three weights {1,2,4}allow to weigh any integral load l between 1and their sum 7,with l ’s binary expansion indicating the weights needed for load l :Generalizing these observations,we should hypothesize that for any posi-tive integer n the set of consecutive powers of 2{w i =2i −1,i =1,2,...n }makes it possible to balance every integral load in the largest possible range,which is up to and including n i =12i −1=2n −1.The fact that every integral weight l in the range 1≤l ≤2n −1can be balanced with this set of weights follows immediately from the binary expansion of l,which yields the weights needed for weighing l.(Note that we can obtain the weights needed for a given load l by applying to it the greedy algorithm for the change-making problem with denominations d i =2i −1,i =1,2,...n.)In order to prove that no set of n weights can cover a larger range of consecutive integral loads,it will suffice to note that there are just 2n −1nonempty selections of n weights and,hence,no more than 2n −1sums they yield.Therefore,the largest range of consecutive integral loads they can cover cannot exceed 2n −1.[Alternatively,to prove that no set of n weights can cover a larger range of consecutive integral loads,we can prove by induction on i that if any mul-tiset of n weights {w i ,i =1,...,n }–which we can assume without loss of generality to be sorted in nondecreasing order–can balance every integral load starting with 1,then w i ≤2i −1for i =1,2,...,n.The basis checks out immediately:w 1must be 1,which is equal to 21−1.For the general case,assume that w k ≤2k −1for every 1≤k <i.The largest weight the first i −1weights can balance is i −1k =1w k ≤ i −1k =12k −1=2i −1−1.If w i were larger than 2i ,then this load could have been balanced neither with the first i −1weights (which are too light even taken together)nor with the weights w i ≤...≤w n (which are heavier than 2i even individ-ually).Hence,w i ≤2i −1,which completes the proof by induction.This immediately implies that no n weights can balance every integral load up to the upper limit larger than n i =1w i ≤ n i =12i −1=2n −1,the limit attainable with the consecutive powers of 2weights.]b.If weights can be put on both cups of the scale,then a larger range can 6课后答案网 w w w .k h d a w .be reached with n weights for n >1.(For n =1,the single weight still needs to be 1,of course.)The weights {1,3}enable weighing of every integral load up to 4;the weights {1,3,9}enable weighing of every inte-gral load up to 13,and,in general,the weights {w i =3i −1,i =1,2,...,n }enable weighing of every integral load up to and including their sum of n i =13i −1=(3n −1)/2.A load’s expansion in the ternary system indicates the weights needed.If the ternary expansion contains only 0’s and 1’s,the load requires putting the weights corresponding to the 1’s on the opposite cup of the balance.If the ternary expansion of load l,l ≤(3n −1)/2,contains one or more 2’s,we can replace each 2by (3-1)to represent it in the form l =n i =1βi 3i −1,where βi ∈{0,1,−1},n = log 3(l +1) .In fact,every positive integer can be uniquely represented in this form,obtained from its ternary expansion as described above.For example,5=123=1·31+2·30=1·31+(3−1)·30=2·31−1·30=(3−1)·31−1·30=1·32−1·31−1·30.(Note that if we start with the rightmost 2,after a simplification,the new rightmost 2,if any,will be at some position to the left of the starting one.This proves that after a finite number of such replacements,we will be able to eliminate all the 2’s.)Using the representation l = n i =1βi 3i −1,we can weigh load l by placing all the weights w i =3i −1for negative βi ’s along with the load on one cup of the scale and all the weights w i =3i −1for positive βi ’s on the opposite cup.Now we’ll prove that no set of n weights can cover a larger range of con-secutive integral loads than (3n −1)/2.Each of the n weights can be either put on the left cup of the scale,or put on the right cup,or not to be used at all.Hence,there are 3n −1possible arrangements of the weights on the scale,with each of them having its mirror image (where all the weights are switched to the opposite pan of the scale).Eliminating this symmetry,leaves us withjust (3n −1)/2arrangements,which can weight at most (3n −1)/2different integral loads.Therefore,the largest range of consecutive integral loads they can cover cannot exceed (3n −1)/2.7.a.Apply Prim’s algorithm to the following graph:7课后答案网 w w w.k h d a w .c o mthe edges ae,eb,ec,and cd.b.Apply Prim’s algorithm to the following graph:the edges ab,be,ed,dc,ef,ei,ij,cg,gh,il,gk.8.There is no need to check the graph’s connectivity because Prim’s algo-rithm can do it itself.If the algorithm reaches all the graph’s vertices (via edges of finite lengths),the graph is connected,otherwise,it is not.9.a.The simplest and most logical solution is to assign all the edge weights to 1.8课a w .c o mb.Applying a depth-first search (or breadth-first search)traversal to get a depth-first search tree (or a breadth-first search tree),is conceptually simpler and for sparse graphs represented by their adjacency lists faster.10.The number of spanning trees for any weighted connected graph is a pos-itive finite number.(At least one spanning tree exists,e.g.,the one obtained by a depth-first search traversal of the graph.And the number of spanning trees must be finite because any such tree comprises a subset of edges of the finite set of edges of the given graph.)Hence,one can always find a spanning tree with the smallest total weight among the finite number of the candidates.Let’s prove now that the minimum spanning tree is unique if all the weights are distinct.We’ll do this by contradiction,i.e.,by assuming that there exists a graph G =(V,E )with all distinct weights but with more than one minimum spanning tree.Let e 1,...,e |V |−1be the list of edges com-posing the minimum spanning tree T P obtained by Prim’s algorithm with some specific vertex as the algorithm’s starting point and let T be an-other minimum spanning tree.Let e i =(v,u )be the first edge in the list e 1,...,e |V |−1of the edges of T P which is not in T (if T P =T ,such edge must exist)and let (v,u )be an edge of T connecting v with a vertex not in the subtree T i −1formed by {e 1,...,e i −1}(if i =1,T i −1consists of vertex v only).Similarly to the proof of Prim’s algorithms correctness,let us replace (v,u )by e i =(v,u )in T .It will create another spanning tree,whose weight is smaller than the weight of T because the weight of e i =(v,u )is smaller than the weight of (v,u ).(Since e i was chosen by Prim’s algorithm,its weight is the smallest among all the weights on the edges connecting the tree vertices of the subtree T i −1and the vertices adjacent to it.And since all the weights are distinct,the weight of (v,u )must be strictly greater than the weight of e i =(v,u ).)This contradicts the assumption that T was a minimum spanning tree.11.If a key’s value in a min-heap was decreased,it may need to be pushedup (via swaps)along the chain of its ancestors until it is smaller than or equal to its parent or reaches the root.If a key’s value in a min-heap was increased,it may need to be pushed down by swaps with the smaller of its current children until it is smaller than or equal to its children or reaches a leaf.Since the height of a min-heap with n nodes is equal to log 2n (by the same reason the height of a max-heap is given by this formula–see Section 6.4),the operation’s efficiency is in O (log n ).(Note:The old value of the key in question need not be known,of paring the new value with that of the parent and,if the min-heap condition holds,with the smaller of the two children,will suffice.)9课后答案网 w w w.k h d a w .c o mExercises 9.21.Apply Kruskal’s algorithm to find a minimum spanning tree of the follow-ing graphs.a.b.2.Indicate whether the following statements are true or false:a.If e is a minimum-weight edge in a connected weighted graph,it must be among edges of at least one minimum spanning tree of the graph.b.If e is a minimum-weight edge in a connected weighted graph,it must be among edges of each minimum spanning tree of the graph.c.If edge weights of a connected weighted graph are all distinct,the graph must have exactly one minimum spanning tree.d.If edge weights of a connected weighted graph are not all distinct,the graph must have more than one minimum spanning tree.3.What changes,if any,need to be made in algorithm Kruskal to make it find a minimum spanning forest for an arbitrary graph?(A minimum spanning forest is a forest whose trees are minimum spanning trees of the graph’s connected components.)10课后答案网h d a w .c o m4.Will either Kruskal’s or Prim’s algorithm work correctly on graphs that have negative edge weights?5.Design an algorithm for finding a maximum spanning tree –a spanning tree with the largest possible edge weight–of a weighted connected graph.6.Rewrite the pseudocode of Kruskal’s algorithm in terms of the operations of the disjoint subsets’ADT.7. Prove the correctness of Kruskal’s algorithm.8.Prove that the time efficiency of find (x )is in O (log n )for the union-by-size version of quick union.9.Find at least two Web sites with animations of Kruskal’s and Prim’s al-gorithms.Discuss their merits and demerits..10.Design and conduct an experiment to empirically compare the efficienciesof Prim’s and Kruskal’s algorithms on random graphs of different sizes and densities.11. Steiner tree Four villages are located at the vertices of a unit squarein the Euclidean plane.You are asked to connect them by the shortest network of roads so that there is a path between every pair of the villages along those roads.Find such a network.11课后答案网ww w.kh d aw .c omHints to Exercises 9.21.Trace the algorithm for the given graphs the same way it is done for another input in the section.2.Two of the four assertions are true,the other two are false.3.Applying Kruskal’s algorithm to a disconnected graph should help to an-swer the question.4.The answer is the same for both algorithms.If you believe that the algorithms work correctly on graphs with negative weights,prove this assertion;it you believe this is not to be the case,give a counterexample for each algorithm.5.Is the general trick of transforming maximization problems to their mini-mization counterparts (see Section6.6)applicable here?6.Substitute the three operations of the disjoint subsets’ADT–makeset (x ),find (x ),and union (x,y )–in the appropriate places of the pseudocode given in the section.7.Follow the plan used in Section 9.1to prove the correctness of Prim’s algorithm.8.The argument is very similar to the one made in the section for the union-by-size version of quick find.9.You may want to take advantage of the list of desirable characteristics in algorithm visualizations,which is given in Section 2.7.10.n/a11.The question is not trivial because introducing extra points (called Steinerpoints )may make the total length of the network smaller than that of a minimum spanning tree of the square.Solving first the problem for three equidistant points might give you an indication how a solution to the problem in question could look like.12课后答案网ww w.kh d aw .c omSolutions to Exercises9.21.a.后课13b.⇒⇒⇒⇒⇒⇒14课c⇒⇒⇒⇒⇒课2.a.True.(Otherwise,Kruskal’s algorithm would be invalid.)b.False.As a simple counterexample,consider a complete graph withthree vertices and the same weight on its three edgesc.True(Problem10in Exercises9.1).15d.False (see,for example,the graph of Problem 1a).3.Since the number of edges in a minimum spanning forest of a graph with |V |vertices and |C |connected components is equal to |V |−|C |(this for-mula is a simple generalization of |E |=|V |−1for connected graphs),Kruskal (G )will never get to |V |−1tree edges unless the graph is con-nected.A simple remedy is to replace the loop while ecounter <|V |−1with while k <|E |to make the algorithm stop after exhausting the sorted list of its edges.4.Both algorithms work correctly for graphs with negative edge weights.One way of showing this is to add to all the weights of a graph with negative weights some large positive number.This makes all the new weights positive,and one can “translate”the algorithms’actions on the new graph to the corresponding actions on the old one.Alternatively,you can check that the proofs justifying the algorithms’correctness do not depend on the edge weights being nonnegative.5.Replace each weight w (u,v )by −w (u,v )and apply any minimum spanning tree algorithm that works on graphs with arbitrary weights (e.g.,Prim’s or Kruskal’s algorithm)to the graph with the new weights.6.Algorithm Kruskal (G )//Kruskal’s algorithm with explicit disjoint-subsets operations //Input:A weighted connected graph G = V,E//Output:E T ,the set of edges composing a minimum spanning tree of G sort E in nondecreasing order of the edge weights w (e i 1)≤...≤w (e i |E |)for each vertex v ∈V make (v )E T ←∅;ecounter ←0//initialize the set of tree edges and its size k ←0//the number of processed edges while ecounter <|V |−1k ←k +1if find (u )=find (v )//u,v are the endpoints of edge e i kE T ←E T ∪{e i k };ecounter ←ecounter +1union (u,v )return E T 7.Let us prove by induction that each of the forests F i ,i =0,...,|V |−1,of Kruskal’s algorithm is a part (i.e.,a subgraph)of some minimum span-ning tree.(This immediately implies,of course,that the last forest in the sequence,F |V |−1,is a minimum spanning tree itself.Indeed,it contains all vertices of the graph,and it is connected because it is both acyclic and has |V |−1edges.)The basis of the induction is trivial,since F 0is16课后答案网ww w.kh d aw .c ommade up of |V |single-vertex trees and therefore must be a subgraph of any spanning tree of the graph.For the inductive step,let us assume that F i −1is a subgraph of some minimum spanning tree T .We need to prove that F i ,generated from F i −1by Kruskal’s algorithm,is also a part of a minimum spanning tree.We prove this by contradiction by assuming that no minimum spanning tree of the graph can contain F i .Let e i =(v,u )be the minimum weight edge added by Kruskal’s algorithm to forest F i −1to obtain forest F i .(Note that vertices v and u must belong to different trees of F i −1–otherwise,edge (v,u )would’ve created a cycle.)By our assumption,e i cannot belong to T .Therefore,if we add e i to T ,a cycle must be formed (see the figure below).In addition to edge e i =(v,u ),this cycle must contain another edge (v ,u )connecting a vertex v in the same tree of F i −1as v to a vertex u not in that tree.(It is possible that v coincides with v or u coincides with u but not both.)If we now delete the edge (v ,u )from this cycle,we will obtain another spanning tree of the entire graph whose weight is less than or equal to the weight of T since the weight of e i is less than or equal to the weight of (v ,u ).Hence,this spanning tree is a minimum spanning tree,which contradicts the assumption that no minimum spanning tree contains F i .This com-pletes the correctness proof of Kruskal’s algorithm.8.In the union-by-size version of quick-union ,each vertex starts at depth 0of its own tree.The depth of a vertex increases by 1when the tree it is in is attached to a tree with at least as many nodes during a union operation.Since the total number of nodes in the new tree containing the node is at least twice as much as in the old one,the number of such increases cannot exceed log 2n.Therefore the height of any tree (which is the largest depth of the tree’s nodes)generated by a legitimate sequence of unions will not exceed log 2n.Hence,the efficiency of find (x )is in O (log n )because find (x )traverses the pointer chain from the x ’s node to the tree’s root.9.n/a10.n/a17课后答案.kh d aw .c om11.The minimum Steiner tree that solves the problem is shown below.(Theother solution can be obtained by rotating the figure 90◦.)A popular discussion of Steiner trees can be found in “Last Recreations:Hydras,Eggs,and Other Mathematical Mystifications”by Martin Gard-ner.In general,no polynomial time algorithm is known for finding a minimum Steiner tree;moreover,the problem is known to be NP -hard (see Section 11.3).For the state-of-the-art information,see,e.g.,The Steiner Tree Page at /steiner/.18课后答案网ww w.kc omExercises 9.31.Explain what adjustments if any need to be made in Dijkstra’s algorithm and/or in an underlying graph to solve the following problems.a.Solve the single-source shortest-paths problem for directed weighted graphs.b.Find a shortest path between two given vertices of a weighted graph or digraph.(This variation is called the single-pair shortest-path prob-lem .)c.Find the shortest paths to a given vertex from each other vertex of a weighted graph or digraph.(This variation is called the single-destination shortest-paths problem .)d.Solve the single-source shortest-path problem in a graph with nonneg-ative numbers assigned to its vertices (and the length of a path defined as the sum of the vertex numbers on the path).2.Solve the following instances of the single-source shortest-paths problem with vertex a as the source:a.b.3.Give a counterexample that shows that Dijkstra’s algorithm may not work for a weighted connected graph with negative weights.19课案w w.kh d aw .c om4.Let T be a tree constructed by Dijkstra’s algorithm in the process of solving the single-source shortest-path problem for a weighted connected graph G .a.True or false:T is a spanning tree of G ?b.True or false:T is a minimum spanning tree of G ?5.Write a pseudocode of a simpler version of Dijkstra’s algorithm that finds only the distances (i.e.,the lengths of shortest paths but not shortest paths themselves)from a given vertex to all other vertices of a graph represented by its weight matrix.6. Prove the correctness of Dijkstra’s algorithm for graphs with positive weights.7.Design a linear-time algorithm for solving the single-source shortest-paths problem for dags (directed acyclic graphs)represented by their adjacency lists.8.Design an efficient algorithm for finding the length of a longest path in a dag.(This problem is important because it determines a lower bound on the total time needed for completing a project composed of precedence-constrained tasks.)9.Shortest-path modeling Assume you have a model of a weighted con-nected graph made of balls (representing the vertices)connected by strings of appropriate lengths (representing the edges).a.Describe how you can solve the single-pair shortest-path problem with this model .b.Describe how you can solve the single-source shortest-paths problem with this model .10.Revisit Problem 6in Exercises 1.3about determining the best route fora subway passenger to take from one designated station to another in a well-developed subway system like those in Washington,DC and London,UK.Write a program for this task.20课后答案网ww w.kh d aw .c om。
Algorithm Design Techniques and Analysis: English VersionExercise with AnswersIntroductionAlgorithms are an essential aspect of computer science. As such, students who are part of this field must master the art of algorithm design and analysis. Algorithm design refers to the process of creating algorithms that solve computational problems. Algorithm analysis, on the other hand, focuses on evaluating the resources required to execute those algorithms. This includes computational time and memory consumption.This document provides students with helpful algorithm design and analysis exercises. The exercises are in the formof questions with step-by-step solutions. The document is suitable for students who have completed the English versionof the Algorithm Design Techniques and Analysis textbook. The exercises cover various algorithm design techniques, such as divide-and-conquer, dynamic programming, and greedy approaches.InstructionEach exercise comes with a question and its solution. Read the question carefully and try to find a solution withoutlooking at the answer first. If you get stuck, look at the solution. Lastly, try the exercise agn without referring to the answer.Exercise 1: Divide and ConquerQuestion:Given an array of integers, find the maximum possible sum of a contiguous subarray.Example:Input: [-2, -3, 4, -1, -2, 1, 5, -3]Output: 7 (the contiguous subarray [4, -1, -2, 1, 5]) Solution:def max_subarray_sum(arr):if len(arr) ==1:return arr[0]mid =len(arr) //2left_arr = arr[:mid]right_arr = arr[mid:]max_left_sum = max_subarray_sum(left_arr)max_right_sum = max_subarray_sum(right_arr)max_left_border_sum =0left_border_sum =0for i in range(mid-1, -1, -1):left_border_sum += arr[i]max_left_border_sum =max(max_left_border_sum, left_b order_sum)max_right_border_sum =0right_border_sum =0for i in range(mid, len(arr)):right_border_sum += arr[i]max_right_border_sum =max(max_right_border_sum, righ t_border_sum)return max(max_left_sum, max_right_sum, max_left_border_s um+max_right_border_sum)Exercise 2: Dynamic ProgrammingQuestion:Given a list of lengths of steel rods and a corresponding list of prices, determine the maximum revenue you can get by cutting these rods into smaller pieces and selling them. Assume the cost of each cut is 0.Lengths: [1, 2, 3, 4, 5, 6, 7, 8]Prices: [1, 5, 8, 9, 10, 17, 17, 20]If the rod length is 4, the maximum revenue is 10.Solution:def max_revenue(lengths, prices, n):if n ==0:return0max_val =float('-inf')for i in range(n):max_val =max(max_val, prices[i] + max_revenue(length s, prices, n-i-1))return max_valExercise 3: Greedy AlgorithmQuestion:Given a set of jobs with start times and end times, find the maximum number of non-overlapping jobs that can be scheduled.Start times: [1, 3, 0, 5, 8, 5]End times: [2, 4, 6, 7, 9, 9]Output: 4Solution:def maximum_jobs(start_times, end_times):job_list =sorted(zip(end_times, start_times))count =0end_time =float('-inf')for e, s in job_list:if s >= end_time:count +=1end_time = ereturn countConclusionThe exercises presented in this document provide a practical way to master essential algorithm design and analysis techniques. Solving the problems without looking at the answers will expose students to the type of problems they might encounter in real life. The document’s solutionsprovide step-by-step instructions to ensure that students can approach the problems with confidence.。
在过去的一年中，我一直在数学的海洋中游荡，research进展不多，对于数学世界的阅历算是有了一些长进。
为什么要深入数学的世界作为计算机的学生，我没有任何企图要成为一个数学家。
我学习数学的目的，是要想爬上巨人的肩膀，希望站在更高的高度，能把我自己研究的东西看得更深广一些。
说起来，我在刚来这个学校的时候，并没有预料到我将会有一个深入数学的旅程。
我的导师最初希望我去做的题目，是对appearance 和motion建立一个unified的model。
这个题目在当今Computer Vision中百花齐放的世界中并没有任何特别的地方。
事实上，使用各种Graphical Model把各种东西联合在一起framework，在近年的论文中并不少见。
我不否认现在广泛流行的Graphical Model是对复杂现象建模的有力工具，但是，我认为它不是panacea（万应灵药），并不能取代对于所研究的问题的深入的钻研。
如果统计学习包治百病，那么很多“下游”的学科也就没有存在的必要了。
事实上，开始的时候，我也是和Vision中很多人一样，想着去做一个Graphical Model——我的导师指出，这样的做法只是重复一些标准的流程，并没有很大的价值。
经过很长时间的反复，另外一个路径慢慢被确立下来——我们相信，一个图像是通过大量“原子”的某种空间分布构成的，原子群的运动形成了动态的可视过程。
微观意义下的单个原子运动，和宏观意义下的整体分布的变换存在着深刻的联系——这需要我们去发掘。
在深入探索这个题目的过程中，遇到了很多很多的问题，如何描述一个一般的运动过程，如何建立一个稳定并且广泛适用的原子表达，如何刻画微观运动和宏观分布变换的联系，还有很多。
在这个过程中，我发现了两个事情：我原有的数学基础已经远远不能适应我对这些问题的深入研究。
在数学中，有很多思想和工具，是非常适合解决这些问题的，只是没有被很多的应用科学的研究者重视。
各种算法英文表示array algorithm数组算法bounded variable algorithm边界变量算法dynamic programming algorithm 动态规划算法enumerative algorithm枚举算法Euclid's algorithm欧几里得算法FFT algorithm快速傅里叶算法fuzzy algorithm模糊算法game playing algorithm博奕[对策]算法Gauss-Newton algorithm高斯-牛顿算法iterative algorithm迭代算法knapsack algorithm渐缩算法Markov algorithm马尔可夫算法minimization algorithm极小化算法optimal estimation algorithm最佳估计算法optimization algorithm最优化算法partan algorithm平行切线(算)法recursive algorithm递归算法statistcal algorithm统计算法steepest descent algorithm最速下降算法steplength algorithms步长算法universal algorithm通用算法algorithm for coding images编码图象的算法Hew to 服从，遵守各种分布的表示：泊松分布：Poisson distribution指数分布：Exponential distribution高斯分布：Gaussian distribution几何分布：Geometric distribution全球性分布：Global distribution均匀分布：Homogeneous distribution 对数分布：Logarithmic distribution。
