Hadoop大数据平台的搭建与测试-论文
- 格式:pdf
- 大小:315.08 KB
- 文档页数:6


Hadoop平台搭建及实例运行本文搭建hadoop平台的硬件环境是虚拟机上安装2个ubuntu系统,具体信息如下:表1.平台硬件环境名字操作系统IP地址内存ubuntu2Ubuntu 8.04192.168.28.13020Gubuntu3Ubuntu 8.04192.168.28.13120G一、平台搭建1.安装必需软件(1)安装java$ sudo apt-get install SUN-java5-jdk系统会自动分析软件的依赖关系,并推荐安装下列相关软件:java-common odbcinst1debian1 sun-java5-bin sun-java5-demo sun-java5-jdk sun-java5-jre unixodbc安装过程中需要回答是否同意使用协议,此时按tab键至OK,再按回车即可正常安装。
安装完这三个之后还需要写入系统变量:$ sudo vi /etc/environment写入下面两行内容:CLASSPATH=.:/usr/lib/jvm/java-1.5.0-sun/libJAVA_HOME=/usr/lib/jvm/java-1.5.0-sun还要将系统虚拟机的优先顺序也调整一下:$ sudo vi /etc/jvm将下面一句写在最顶部:/usr/lib/jvm/java-1.5.0-sun测试接下来在终端中输入命令:$java -version终端如果返回以下信息:图1-1.Java版本信息则说明java已经安装好了。
(2)安装其它软件$ sudo apt-get install ssh$ sudo apt-get install rsync注:在每个节点上都要执行以上相同的过程。
2.系统配置(1)修改主机名root@ubuntu3:/hadoop$ sudo vi /etc/hostname把新的主机名写入hostname即可,这里我写入:ubuntu3(2)修改hosts文件root@ubuntu3:/hadoop$ sudo vi /etc/hosts将以下内容添加到该文件中:127.0.1.1 ubuntu3192.168.28.130 ubuntu2192.168.28.131 ubuntu3注:把第一行的127.0.0.1 localhost用#注掉(3)配置sshHadoop启动以后,Namenode通过SSH来启动和停止各个节点上的各种守护进程,需要在节点之间执行指令的时候是不输入密码,所以需要用无密码公钥认证的方式配置SSH。
(此文档为word格式,下载后您可任意编辑修改!)毕业设计(论文)中文题目:基于hadoop的分布式存储平台的搭建与验证英文题目:Setuping and verification distributed storage platform based on the principle of Google file system developed and implemented by the greatconcern of the IT industry, and widely used.The thesis aims to set up Hadoop multi-node distributed storage platform and analyze its security mechanisms to be implemented on a separate computer.The thesis first introduces the research background knowledge of the subject, and detailed description of the study and the principle of the of the platform, and its performance were verified, further security mechanisms. First the industry generally accepted user requirements and the architecture of the distributed file system model are introduced。
Then for HDFS architecture to achieve the Hadoop security mechanisms and the corresponding security policy. In addition,the advantages of HDFS in the field of cloud computing applications and the security problem are summarized. At last thedesign and application recommendations are presented.The experimental platform installed virtualbox ubuntu10.10 of application is a the this experiment platform.Keywords: ,HDFS, MapReduce,ZooKeeper,Avro,Chukwa,HBase,Hive,Mahout,Pig 在内的10 个子项目。