模式识别课程实验报告学院专业班级姓名学号指导教师提交日期1 Data PreprocessingThe provide dataset includes a training set with 3605 positive samples and 10055 negative samples, and a test set with 2043 positive samples and 4832 negative samples. A 2330-dimensional Haar-like feature was extracted for each sample. For high dimensional data, we keep the computation manageable by sampling , feature selection, and dimension reduction.1.1 SamplingIn order to make the samples evenly distributed, we calculate the ratio of negative samples and positive samples int test set. And then randomly select the same ratio of negative samples in training set. After that We use different ratios of negative and positive samples to train the classifier for better training speed.1.2 Feature SelectionUnivariate feature selection can test each feature, measure the relationship between the feature and the response variable so as to remove unimportant feature variables. In the experiment, we use the method sklearn.feature_selection.SelectKBest to implement feature selection. We use chi2 which is chi-squared stats of non-negative features for classification tasks to rank features, and finally we choose 100 features ranked in the top 100.2 Logistic regressionIn the experiment, we choose the logistic regression model, which is widely used in real-world scenarios. The main consideration in our work is based on the binary classification logic regression model.2.1 IntroductionA logistic regression is a discriminant-based approach that assumes that instances of a class are linearly separable. It can obtain the final prediction model by directly estimating the parameters of the discriminant. The logistic regression model does not model class conditional density, but rather models the class condition ratio.2.2 processThe next step is how to obtain the best evaluation parameters, making the training of the LR model can get the best classification effect. This process can also be seen as a search process, that is, in an LR model of the solution space, how to find a design with our LR model most match the solution. In order to achieve the best available LR model, we need to design a search strategy.The intuitive idea is to evaluate the predictive model by judging the degree of matching between the results of the model and the true value. In the field of machine learning, the use of loss function or cost function to calculate the forecast. For the classification, logistic regression uses the Sigmoid curve to greatly reduce the weight of the points that are far from the classification plane through the nonlinear mapping, which relatively increases the weight of the data points which is most relevant to the classification.2.3 Sigmoid functionWe should find a function which can separate two in the two classes binary classification problem. The ideal function is called step function. In this we use the sigmoid function.()z e z -+=11σWhen we increase the value of x, the sigmoid will approach 1, and when we decrease the value of x, the sigmoid will gradually approaches 0. The sigmoid looks like a step function On a large enough scale.2.4 gradient descenti n i i T i x x y L ∑=+--=∂∂-=1t t1t ))(()(θσαθθθαθθThe parameter α called learning rate, in simple terms is how far each step. this parameter is very critical. parameter σis sigmoid function that we introduce in 2.3.3 Train classifierWe use the gradient descent algorithm to train the classifier. Gradient descent is a method of finding the local optimal solution of a function using the first order gradient information. In optimized gradient rise algorithm, each step along the gradient negative direction. The implementation code is as follows:# calculate the sigmoid functiondef sigmoid(inX):return 1.0 / (1 + exp(-inX))#train a logistic regressiondef trainLogisticRegression(train_x, train_y, opts):numSamples, numFeatures = shape(train_x)alpha = opts['alpha'];maxIter = opts['maxIter']weights = ones((numFeatures, 1))# optimize through gradient descent algorilthmfor k in range(maxIter):if opts['optimizeType'] == 'gradDescent': # gradient descent algorilthm output = sigmoid(train_x * weights)error = train_y - outputweights = weights + alpha * train_x.transpose() * error return weightsIn the above program, we repeat the following steps until the convergence:(1) Calculate the gradient of the entire data set(2) Use alpha x gradient to update the regression coefficientsWhere alpha is the step size, maxIter is the number of iterations.4 evaluate classifier4.1 optimizationTo find out when the classifier performs best, I do many experiments. The three main experiments are listed below.First.I use two matrix types of data to train a logistic regression model using some optional optimize algorithm, then use some optimization operation include alpha and maximum number of iterations. These operations are mainly implemented by this function trainLogisticRegression that requires three parameters train_x, train_y, opts, of which two parameters are the matrix type of training data set, the last parameter is some optimization operations, including the number of steps and the number of iterations. And the correct rate as follows：When we fixed the alpha, change the number of iterations, we can get the results as shown below：Red: alpha = 0.001Yellow: alpha = 0.003Blue: alpha = 0.006Black: alpha = 0.01Green: alpha = 0.03Magenta: alpha = 0.3This line chart shows that if the iterations too small, the classifier will perform bad. As the number of iterations increases, the accuracy will increase. And we can konw iterations of 800 seems to be better. The value of alpha influences the result slightly. If the iterations is larger then 800, the accuracy of the classifier will stabilize and the effect of the step on it can be ignored.Second.When we fixed the iterations, change the alpha, we can get the results as shown below：This line chart shows when the number of iterations is larger 800, the accuracy is relatively stable, with the increase of the alpha, the smaller the change. And when the number of iterations is small, for example, the number of times is 100, the accuracy is rather strange, the accuracy will increase with the increase of alpha. We can know that when the number of iterations is small, even if the alpha is very small, its accuracy is not high. The number of iterations and the alpha is important for the accuracy of the logical regression model training.Third.The result of the above experiment is to load all the data to train the classifier, but there are 3605 pos training data but 10055 neg training data. So I am curious about whether can I use less neg training data to train the classifier for better training speed.I use the test_differentNeg.py to does this experiment:Red: iterations = 100 Yellow: iterations = 300 Blue: iterations = 800 Black: iterations = 1000The line chart shows that the smaller the negative sample, the lower the accuracy. However, as the number of negative samples increases, the accuracy of the classifier.I found that when the number of negative samples is larger 6000, the accuracy was reduced and then began to increase.4.2 cross-validationWe divided the data set into 10 copies on average，9 copies as a training set and 1 copies as a test set.we can get the results as shown below:Red: alpha = 0.001Yellow: alpha = 0.003Blue: alpha = 0.006Black: alpha = 0.01Green: alpha = 0.03Magenta: alpha = 0.3This figure shows that if the iterations too small, the classifier will perform bad. Asthe number of iterations increases, the accuracy will increase. And we can konw iterations of 1000 seems to be better. The value of alpha influences the result slightly.If the iterations is larger then 1000, the accuracy of the classifier will achieve 84% and stable. And the effect of the step on it can be ignored.5 ConclusionIn this experiment, We notice that no matter before or after algorithm optimization,the test accuracy does not achieve our expectation. At first the accuracy of LR can reach 80%. After optimization, the accuracy of LR can reach 88%. I think there are several possible reasons below：(1) The training set is too small for only about 700 samples for active training sets, while the negative training set has only 800 samples;(2) We did not use a more appropriate method to filter the samples;(3) The method of selecting features is not good enough because it is too simple;(4) Dimension reduction does not be implemented, so that The high dimension of features increases the computation complexity and affects the accuracy;(5) The test sets may come from different kinds of video or images which are taken from different areas.(6) The number of iterations is set too small or the alpha is set too small or too large. These questions above will be improved in future work.In this project, we use Logistic regression to solve the binary classification problem. Before the experiment, we studied and prepared the algorithm of the LR model and reviewed other machine learning models studied in the pattern recognition course. In this process, we use python with a powerful machine learning library to implement, continue to solve the various problems, and optimize the algorithm to make the classifier perform better. At the same time, we also collaborate and learn from each other. Although the project takes us a lot of time, We have consolidated the theoretical knowledge and have fun in practice.。
