数学之路-分布式计算-hadoop、python与R(3)
二、ubuntu 12.04hadoop2.6配置与安装
1、java7配置
#su
#addusermyhaspl
以下添加su权限
使得myhaspl用户属于sudo组
#vi/etc/group
sudo:x:27:,myhaspl
然后又一次启动
#reboot
转到myhaspl用户。開始下载解压java。
本博客全部内容是原创,假设转载请注明来源
http://blog.csdn.net/myhaspl/
#sumyhaspl
$wgethttp://download.oracle.com/otn-pub/java/jdk/7u71-b14/jdk-7u71-linux-x64.tar.gz
myhaspl@aaayun:~$tar -xzvf jdk-7u71-linux-x64.tar.gz
myhaspl@aaayun:~$sudo mv jdk1.7.0_71 /usr/lib/java
root@aaayun:/home/myhaspl#vi /etc/profile
exportJAVA_HOME=/usr/lib/java
exportJRE_HOME=${JAVA_HOME}/jre
exportCLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
exportPATH=${JAVA_HOME}/bin:$PATH
#reboot
myhdp@myhdp:~$sudo update-alternatives --install /usr/bin/java java /usr/lib/java/bin/java300
update-alternatives:using /usr/lib/java/bin/java to provide /usr/bin/java (java) in auto mode
myhdp@myhdp:~$sudo update-alternatives --install /usr/bin/javac javac /usr/lib/java/bin/javac300
update-alternatives:using /usr/lib/java/bin/javac to provide /usr/bin/javac (javac) in auto mode
myhaspl@aaayun:~$ java -version
javaversion "1.7.0_71"
Java(TM)SE Runtime Environment (build 1.7.0_71-b14)
JavaHotSpot(TM) 64-Bit Server VM (build 24.71-b01, mixed mode)
myhaspl@aaayun:~$
myhaspl@aaayun:~$vi HelloWorldjava
publicclass HelloWorld{
publicstatic void main (String[] args){
System.out.println("HelloWorld");
}
}
myhaspl@aaayun:~$javac HelloWorld.java
myhaspl@aaayun:~$java HelloWorld
HelloWorld
myhdp@myhdp:~$sudo apt-get install ssh
myhdp@myhdp:~$sudo apt-get install rsync
2、hadoop2.6
下载解压hadoop
myhaspl@aaayun:~$wget http://mirrors.cnnic.cn/apache/hadoop/common/stable/hadoop-2.6.0.tar.gz
myhaspl@aaayun:~$tar -xzvf hadoop-2.6.0.tar.gz
将hadoop移动到系统目录,并授予相关权限。
myhaspl@aaayun:~$sudo mv hadoop-2.6.0 /usr/local/
myhaspl@aaayun:~$cd /usr/local/
myhaspl@aaayun:~$sudo chown -R root:root hadoop-2.6.0/
root@aaayun:/home/myhaspl#vi /etc/profile
exportHADOOP_PREFIX=/usr/local/hadoop-2.6.0
myhdp@myhdp:/usr/local/hadoop-2.6.0$ export HADOOP_PREFIX=/usr/local/hadoop
myhdp@myhdp:/usr/local/hadoop-2.6.0$cd ~
显示hadoop文档
myhdp@myhdp:~$cd/usr/local/hadoop-2.6.0
myhdp@myhdp:~$bin/hadoop
Usage:hadoop [--config confdir] COMMAND
where COMMAND is one of:
fs run a generic filesystemuser client
version print the version
jar <jar> run a jar file
checknative [-a|-h] check native hadoop and compression librariesavailability
distcp <srcurl> <desturl> copyfile or directories recursively
archive -archiveName NAME -p <parentpath> <src>* <dest> create a hadoop archive
classpath prints the class path needed to get the
credential interact with credential providers
Hadoop jar and therequired libraries
daemonlog get/set the log level for eachdaemon
trace view and modify Hadoop tracingsettings
or
CLASSNAME run the class named CLASSNAME
Mostcommands print help when invoked w/o parameters.
伪分布式
hdfs的地址及端口号
myhdp@myhdp:~$sudo vi /usr/local/hadoop-2.6.0/etc/hadoop/core-site.xml <configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
~
hdfs备份数量,改为1
etc/hadoop/hdfs-site.xml:
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
相关文章
- Python 中调用 R 语言 | rpy2 的安装和使用 | 完美解决各种问题并成功配置好
- Python玩转emoji表情 一行代码的事儿!
- 【列表复制】详解python中list列表复制的几种方法(赋值、切片、copy(),deepcopy())
- python 设计模式 单例模式
- 12月23日科技资讯|华为证实 P40 将没有 Google 服务;Python 3.8.1 发布
- 做Python网络爬虫需要掌握哪些核心技术?
- Python数据科学指南
- python基础小白题3
- 成功解决:Can‘t find Python executable “python“, you can set the PYTHON env variable.
- 从零起步(无需数学和Python基础)编码实现AI框架之第六节课:使用Matrix编写AI框架实战及测试
- shell脚本调用python脚本的路径问题
- PyQt(Python+Qt)学习随笔:Qt Designer中主窗口对象unifiedTitleAndToolBarOnMac属性
- Python中判断字符串是否为数字的三个方法isdecimal 、isdigit、isnumeric的差别
- Python Pandas pandas.DataFrame.to_html函数方法的使用
- 利用Python合并指定行列excel文件
- IMPLEMENTING A GRU/LSTM RNN WITH PYTHON AND THEANO - 学习笔记
- statistics of Python
- 新出python基础到项目整套练习册及文章密码获取