准备
准备三台虚拟机(CentOS 7.6)，网络选择NAT模式

master 192.168.10.11

slave1 192.168.10.12

slave2 192.168.10.13

 

第一步 设置主机名
[root@localhost ~]# hostnamectl set-hostname master

[root@localhost ~]# hostnamectl set-hostname slave1

[root@localhost ~]# hostnamectl set-hostname slave2

用命令[root@localhost ~]# hostname 查看

 

第二步 设置网络IP
vi /etc/sysconfig/network-scripts/ifcfg-ens33

TYPE=Ethernet

PROXY_METHOD=none

BROWSER_ONLY=no

BOOTPROTO=none

DEFROUTE=yes

IPV4_FAILURE_FATAL=no

IPV6INIT=yes

IPV6_AUTOCONF=yes

IPV6_DEFROUTE=yes

IPV6_FAILURE_FATAL=no

IPV6_ADDR_GEN_MODE=stable-privacy

NAME=ens33

UUID=de2680b4-14ec-46cc-845b-212aebc713c7

DEVICE=ens33

ONBOOT=yes

IPADDR=192.168.10.11

GATEWAY=192.168.10.1

NETMASK=255.255.255.0

DNS1=192.168.10.1

 

[root@master ~]# systemctl restart network

 

分别设置为：192.168.10.11、192.168.10.12、192.168.10.13

第三步 添加主机名和IP的映射关系
每台机器都添加

[root@master ~]# vi /etc/hosts

192.168.10.11 master

192.168.10.12 slave1

192.168.10.13 slave2

 

第四步 关闭防火墙
每台机器设置

查看：systemctl status firewalld.service

关闭：systemctl stop firewalld.service

启动：systemctl start firewalld.service

 

第五步 创建hadoop用户
每台机器都创建

（1） 统一用hadoop 用户名密码

用户名hadoop,密码hadoop：

#useradd hadoop;

#passwd hadoop 输入密码 hadoop

(2) 也可直接用root 用户

 

第六步 在master主机上下载hadoop3.1.4、jdk1.8
下载hadoop

# wget http://mirrors.hust.edu.cn/apache/hadoop/core/hadoop-3.1.4/hadoop-3.1.4.tar.gz

# tar -zvxf hadoop-3.1.4.tar.gz

# mv hadoop-3.1.4 /home/hadoop/

# chown hadoop:hadoop -R /home/hadoop/

 

下载jdk-8u291-linux-x64.tar.gz

# tar -zvxf jdk-8u291-linux-x64.tar.gz

# mv jdk1.8.0_291 /usr/local/jdk

# chmod 755 -R /usr/local/jdk

 

 

第七步 添加环境变量
现只在matser下添加

# vi /etc/profile

export JAVA_HOME=/usr/local/jdk

export JRE_HOME=/$JAVA_HOME/jre

export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin

export HADOOP_HOME=/home/hadoop/hadoop-3.1.4

export HADOOP_COMMON_HOME=$HADOOP_HOME

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native

export HADOOP_OPTS=”-Djava.library.path=$HADOOP_HOME/lib”

export PATH=.:${JAVA_HOME}/bin:${HADOOP_HOME}/bin:$PATH

 

—————————————————-

下面的内容也可以配置在 hadoop-env.sh中

对应第五步中的用户

export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_SECONDARYNAMENODE_USER=root
export YARN_RESOURCEMANAGER_USER=root
export YARN_NODEMANAGER_USER=root

 

# source /etc/profile

 

第八步 设置免密登录
[root@master ~]# su – hadoop

[hadoop@master ~]$ ssh-keygen

[root@slave1~]# su – hadoop

[hadoop@slave1 ~]$ ssh-keygen

[root@slave2 ~]# su – hadoop

[hadoop@slave2 ~]$ ssh-keygen

[hadoop@master ~]$ ssh-copy-id master

[hadoop@master ~]$ ssh-copy-id slave1

[hadoop@master ~]$ ssh-copy-id slave2

[hadoop@slave1 ~]$ ssh-copy-id master

[hadoop@slave1 ~]$ ssh-copy-id slave1

[hadoop@slave1 ~]$ ssh-copy-id slave2

[hadoop@slave2 ~]$ ssh-copy-id master

[hadoop@slave2 ~]$ ssh-copy-id slave1

[hadoop@slave2 ~]$ ssh-copy-id slave2

 

第九步 hadoop配置
先在matser配置

[hadoop@master ~]$ cd /home/hadoop/hadoop-3.1.4/etc/hadoop/

[hadoop@master ~]$ mkdir /home/hadoop/hadoop-3.1.4/hdfs/tmp -p

[hadoop@master ~]$ mkdir /home/hadoop/hadoop-3.1.4/hdfs/name -p

[hadoop@master ~]$ mkdir /home/hadoop/hadoop-3.1.4/hdfs/data -p

[hadoop@master hadoop]$ vi hadoop-env.sh

export JAVA_HOME=/usr/local/jdk

export HADOOP_OPTS=”-Djava.library.path=${HADOOP_HOME}/lib/native”

 

[hadoop@master hadoop]$ vi core-site.xml

<configuration>

<property>

<name>fs.defaultFS</name>

<value>hdfs://master:9000</value>

</property>

<property>

<name>hadoop.tmp.dir</name>

<value>/home/hadoop/hadoop-3.1.4/hdfs/tmp</value>

</property>

</configuration>

 

 

[hadoop@master hadoop]$ vi hdfs-site.xml

<configuration>

<property>

<name>dfs.namenode.name.dir</name>

<value>/home/hadoop/hadoop-3.1.4/hdfs/name</value>

</property>

<property>

<name>dfs.datanode.data.dir</name>

<value>/home/hadoop/hadoop-3.1.4/hdfs/data</value>

</property>

<property>

<name>dfs.replication</name>

<value>3</value>

</property>

<property>

<name>dfs.secondary.http.address</name>

<value>master:50090</value>

</property>

</configuration>

 

[hadoop@master hadoop]$ vi mapred-site.xml

<property>

<name>yarn.resourcemanager.hostname</name>

<value>master</value>

</property>

<property>

<name>yarn.nodemanager.aux-services</name>

<value>mapreduce_shuffle</value>

</property>

 

[hadoop@master hadoop]$ vi yarn-site.xml

<property>

<name>yarn.resourcemanager.hostname</name>

<value>master</value>

</property>

<property>

<name>yarn.nodemanager.aux-services</name>

<value>mapreduce_shuffle</value>

</property>

 

[hadoop@master hadoop]$ vi workers

slave1

slave2

 

第十步 拷贝hadoop文件和/etc/profile和jdk文件
[hadoop@master hadoop]$ scp -rq /home/hadoop/hadoop-3.1.4 slave1:/home/hadoop/

[hadoop@master hadoop]$ scp -rq /home/hadoop/hadoop-3.1.4 slave2:/home/hadoop/

[hadoop@master hadoop]$ su

[root@master hadoop]# scp /etc/profile slave1:/etc/

[root@master hadoop]# scp /etc/profile slave2:/etc/

[root@master hadoop]# scp -rq /usr/local/jdk/ slave1:/usr/local/

[root@master hadoop]# scp -rq /usr/local/jdk/ slave2:/usr/local/

[root@slave1 ~]$ source /etc/profile

[root@slave2 ~]$ source /etc/profile

scp 之后记得改一下slave的权限：

每个机器都要改：

chown hadoop:hadoop -R /home/hadoop/

chmod 755 -R /usr/local/jdk

第十一步 格式化及启动hadoop
切换至hadoop用户启动：

[root@master sbin]# su – hadoop

[hadoop@master hadoop]# cd /home/hadoop/hadoop-3.1.4/sbin/

[hadoop@sbin ~]$ hdfs namenode -format

出现***has been successfully formatted.表示格式化成功。

启动：start-all.sh

查看对应进程是否启动了

[hadoop@master sbin]$ jps

61376 Jps

48002 SecondaryNameNode

48227 ResourceManager

47690 NameNode

 

[root@slave1 ~]# jps

23411 DataNode

29013 Jps

23611 NodeManager

 

[root@slave2 ~]# jps

4347 DataNode

5116 Jps

4460 NodeManager

 

测试

查看集群，活动节点是否有2个

http://192.168.10.11:8088/cluster

 

查看集群容量

[hadoop@master sbin]$ hadoop fs -df -h /

 

查看各个机器状态报告

[hadoop@master sbin]$ hadoop dfsadmin -report
————————————————
版权声明：本文为CSDN博主「^果然好^」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/allensandy/article/details/116082426