Capital Normal University论文编码:TP181首都师范大学学士学位论文基于Web的文本分类挖掘的研究院系信息工程学院专业计算机科学与技术系年级2001学号1011000047指导老师刘丽珍论文作者王雪完成日期2005年6月6日中文提要文本分类最初是应文本信息检索的要求出现的,但是随着文本数据的激增,传统的研究方法己经不适合大规模文本分类,文本数据挖掘应运而生。
作为文本数据挖掘的一个重要功能,文本分类技术日益成为研究热点。
文本分类目的是对文本集有序组织,便于文本信息高效管理,为人的决策提供支持。
但是传统的人工分类的做法存在许多弊端,不仅是耗费大量人力、物和精力,而且受人为因素影响较大,分类结果一致性不高。
与之相比,文本自动分类具有快速、高效的特点,且分类准确率较高。
对文本分类技术进行研究,介绍文本分类的基本过程,论述文本特征提取方法,讨论朴素贝叶斯、K近邻、支持向量机、投票等常用的文本分类原理与方法,探讨中文文本分类技术。
采用支持向量机技术,设计并实现了一个开放的中文文档自动分类系统。
实验表明,它不仅具有较高的训练效率,同时能得到很高的分类准确率和查全率。
关键词:文本挖掘文本分类支持向量机向量空间模型外文提要Text categorization appears initially for text information retrieval system; however text data increases so fast that traditional research methods have been improper for large-scale text categorization. So text data mining emerges, and text categorization becomes more and more important as a major research field of it.The purpose of text categorization is to organize text by order,so as to manage text information efficiently and support decisions of human being. However categorization by hand not only consumes plenty of manpower, material resources and energy, but also makes categorization accuracy inconsistent. Compared with categorization by hand, automatic text categorization classifies texts faster and its categorization accuracy rates higher.Introduces the techniques of text categorization, including its basic process ,the algorithms of text feature extraction ,the theories and technologies such as Naïve bayes, KNN, SVM, Voted and so on. Chinese text classification is discussed.An open Chinese document classification system using support is designed and implemented.The experiment shows that it not only improves training efficiency, but also has good precision and recall.Key wordt ext mining Text categorization Support Vector Machine(SVM)vector space model目录中文提要 ..................................................................................................................... 1外文提要 ..................................................................................................................... 3目录 ........................................................................................................................... 4第一章绪论 ........................................................................................................... 51.1文本自动分类研究的背景和意义 ............................................................. 51.2问题的描述 ................................................................................................. 71.3国内外文本自动分类研究动态 ................................................................. 7第二章中文文本分类技术研究 ............................................................................. 92.1文本预处理 ................................................................................................. 92.1.1文本半结构化 ................................................................................... 92.1.2自动分词 ........................................................................................... 92.1.3特征选择[12]....................................................................................... 92.2分类模型 ................................................................................................. 102.2.1贝叶斯(Naive Bayes)方法[14] .................................................. 102.2.2K-近邻(KNN)方法 .................................................................. 102.2.3决策树(Decision Tree)分类..................................................... 112.2.4基于投票的方法 ........................................................................... 112.2.5支持向量机(SVM)方法[17] ...................................................... 122.3分类性能评价 ......................................................................................... 12第三章基于支持向量机的中文文本分类 ......................................................... 133.1 统计学习理论.......................................................................................... 133.2支持向量机原理 ..................................................................................... 153.3支持向量机的特点 ................................................................................. 17第四章基于支持向量机的中文文本分类器的实现 ......................................... 184.1 系统体系结构.......................................................................................... 184.1.1文本训练模块设计 .......................................................................... 194.1.2文本分类模块设计 .......................................................................... 19第五章系统的性能测试 ..................................................................................... 205.1开发环境和数据集 ................................................................................. 205.2测试结果及分析 ..................................................................................... 20第六章总结与展望 ............................................................................................... 226.1全文总结 ................................................................................................. 226.2进一步工作及展望 ................................................................................. 22附录(附图) ......................................................................................................... 23参考文献 ................................................................................................................. 26致谢 ..................................................................................................................... 27第一章绪论1.1文本自动分类研究的背景和意义分类最初是应信息检索(Information Retrieval,简称IR)系统的要求而出现的,也是数据挖掘应用领域的重要技术之一[1].随着全球计算机与通讯技术的飞速发展、互联网的普及与应用,信息爆炸的现实使人们越来越注重对自动分类的研究,文本自动分类及其相关技术的研究也日益成为一项研究热点。