大规模集群自动化部署SSH无密码登陆
2023-09-14 09:00:23 时间
expect的使用请看我的另一篇文章：
http://tianxingzhe.blog.51cto.com/3390077/1687661
spawn命令激活一个Unix程序来进行交互式的运行。
send命令向进程发送字符串。
expect命令等待进程的某些字符串
set timeout 1 设置超时时间 timeout -1 为永不超时
expect eof
只有spawn执行的命令结果才会被expect捕捉到，因为spawn会启动一个进程，只有这个进程的相关信息才会被捕捉到，主要包括：标准输入的提示信息，eof和timeout。
这里，eof是必须去匹配的，在spawn进程结束后会向expect发送eof；如果不去匹配，有时也能运行，比如sleep多少秒后再去spawn下一个命令，但是不要依赖这种行为，很有可能今天还可以，明天就不能用了。
expect \"#\" 期待返回shell提示符(是#或者$)
interact 命令
执行完成后保持交互状态，把控制权交给控制台，这个时候就可以手工操作了。如果没有这一句登录完成后会退出，而不是留在远程终端上。如果你只是登录过去执行一段命令就退出，可改为 expect eof
id_dsa/ id_dsa.pub：你用openssh工具生成的私钥公钥对
authorized_keys ：你使用ssh连接的linux服务器需要认证你的身份，所以你需要在连接的linux服务器上安装自已的公钥，authorized_keys这里面就是存放你自己的id_dsa.pub的内容
scp是有Security的文件copy，基于ssh登录。操作起来比较方便，比如要把当前一个文件copy到远程另外一台主机上，可以如下命令。
scp /home/daisy/full.tar.gz
大体思路
1、首先在一个文本文件中保存1000台机器的hadoop用户名和密码
2、用shell遍历这个文件 写一个循环用namenode的去循环登陆其他的999个节点，执行生成密钥的工作，然后把生成的公钥写回namenode
3、在namenode上生成密钥 写入这个文件
4、把第三部生成的文件拷贝到剩下的机器上
5、用循环遍历验证免密的效果
本解决方法主要包括两个脚本： sshpass.sh和ssh4slaves
1. sshpass.sh
#!/bin/bash
# Name : sshpass.sh
# Time : 17/09/2012
# Author : simplestone@dbinterest.com
# Purpose : For fast and easy setup of the SSH Passwordless access among all the nodes
# in a cluster.
# User : Any user you are performing the test! Better to settup a separate user from your
# working env to avoid troubles!!! "root" is used in this example, and you can change it
# via the export virable "USER=root"
# Attention: The test env is assuming that each $USER on each $HOST is usring the same password!
# And this likely makes sense as no body want to put more trouble on this.
# Usage : 1st, make sure the script has the execute permisison "chmod +x ssh_pass.sh"
# ./ssh_pass.sh password
# : 2nd, ensure the "ssh4slaves.sh" script is with ssh_pass.sh for all nodes setup!!!
# : 3rd, "expect" has to be installed on all the nodes for the SSH config
export FILELOC="/root"
export SLAVESFILE="$FILELOC/sshslaves"
export HOSTS=`cat $FILELOC/sshhosts`
export SLAVES=`cat $FILELOC/sshslaves`
export SSH4SLAVESCRIPT="$FILELOC/ssh4slaves.sh"
export MASTER=hdp01
export USER=root
export PASSWD=$1
export SSHLOC="$FILELOC/.ssh/"
export RSAFILE="$FILELOC/.ssh/id_rsa"
export RSAPUBFILE="$FILELOC/.ssh/id_rsa.pub"
export AUTHFILE="$FILELOC/.ssh/authorized_keys"
export EXPECTCHK=`rpm -qa expect | wc -l`
if [ $EXPECTCHK != 1 ]
then
echo
echo "########################################################################################"
echo "Please install the \"expect\" package first on all nodes to allow the script to run!!!"
echo "yum -y install expect"
echo "########################################################################################"
if [ -e $RSAFILE ]
then
echo "########################################################################################"
echo "Attention: This is for TEST ONLY, please fully test it before applying it to PROD"
echo "environment!!! OR you might get in trouble!!!"
echo
echo "BETTER TO HAVE A NEW USER FOR THE TEST TO AVOID DESTROYING YOUR ENVIRONMENT!"
echo
echo "Please manually delete the ssh related file on each host before executing the script!!!"
echo
for host in $HOSTS
do
echo "Please run command on $host: rm -rf $SSHLOC"
done
echo "########################################################################################"
else
# Just generate
for host in $HOSTS
do
if [ $host = "$MASTER" ]
then
echo
echo "###########################################################"
echo "Generating RSA keys for MASTER host $MASTER"
echo "###########################################################"
echo
expect -c "
set timeout 1
spawn ssh $USER@$host
expect \"yes/no\"
send -- \"yes\r\"
expect \"password:\"
send -- \"$PASSWD\r\"
expect \"#\"
send \"ssh-keygen -t rsa -P -f $RSAFILE\r\"
expect \"#\"
send \"ssh-copy-id -i $RSAPUBFILE $MASTER\r\"
expect \"password:\"
send -- \"$PASSWD\r\"
expect eof
"
else
echo
echo "###########################################################"
echo "Generating RSA keys for all OTHER hosts..."
echo "hostname is $host"
echo "###########################################################"
echo
expect -c "
set timeout 1
spawn ssh $USER@$host
expect \"yes/no\"
send -- \"yes\r\"
expect \"password:\"
send -- \"$PASSWD\r\"
expect \"#\"
send \"ssh-keygen -t rsa -P -f $RSAFILE\r\"
expect \"#\"
send \"ssh-copy-id -i $RSAPUBFILE $MASTER\r\"
expect \"yes/no\"
send -- \"yes\r\"
expect \"password:\"
send -- \"$PASSWD\r\"
expect eof
"
fi
done
###
for host in $SLAVES
do
echo
echo "############################################################################"
echo "Copying authorized_keys to host $host from the MASTER host $MASTER..."
echo "############################################################################"
echo
expect -c "
set timeout 1
spawn scp $AUTHFILE "$USER@$host:$SSHLOC"
expect \"password:\"
send -- $PASSWD\r
expect eof
"
done
for host in $SLAVES
do
echo
echo "############################################################################"
echo "Distributing the $SLAVESFILE file to slave host $host..."
echo "############################################################################"
echo
scp $SLAVESFILE "$host:$FILELOC"
echo
echo "############################################################################"
echo "Distributing the $SSH4SLAVESCRIPT script to slave host $host..."
echo "############################################################################"
echo
scp $SSH4SLAVESCRIPT "$host:$FILELOC"
done
for host in $SLAVES
do
echo
echo "############################################################################"
echo "Working on the slaves node $host to ensure no prompt for the "yes/no" question..."
echo "############################################################################"
echo
ssh -q $USER@$host $SSH4SLAVESCRIPT
done
### Check whether the Passwordless ssh works ###
for host in $HOSTS
do
echo
echo "############################################################################"
echo "Check whether the Passwordless SSH works for $host..."
echo "############################################################################"
echo
ssh $host uname -a date
done
# rm -rf /root/.ssh
# mv /root/.ssh /root/sshlogin
#{ eval "$GET_ID" ; } | ssh $1 "umask 077; test -d .ssh || mkdir .ssh ; cat .ssh/authorized_keys; test -x /sbin/restorecon /sbin/restorecon .ssh .ssh/authorized_keys" || exit 1
#cat /root/.ssh/id_rsa.pub | ssh hdp01 "umask 077; test -d .ssh || mkdir .ssh ; cat .ssh/authorized_keys; test -x /sbin/restorecon /sbin/restorecon .ssh .ssh/authorized_keys" || exit 1
#/root/.ssh/id_rsa.pub
./ssh_pass.sh password （password替换程序里的$1参数），本例中密码为stonetest
2. ssh4slaves
#!/bin/bash # Name : ssh4slaves.sh # Time : 17/09/2012 # Author : simplestone@dbinterest.com # Purpose : For fast and easy setup of the SSH Passwordless access among all the slave nodes # in a cluster. Mainly to ensure no prompt for "yes/no" again!!! # User : Any user you are performing the test! Better to settup a separate user from your # working env to avoid troubles!!! "root" is used in this example, and you can change it # via the export virable "USER=root" # Attention: The test env is assuming that each $USER on each $HOST is usring the same password! # And this likely makes sense as no body want to put more trouble on this. # Usage : This script is called by the main script "ssh_pass.sh" # 1st, make sure the script has the execute permisison "chmod +x ssh4slaves.sh" before # distributing it to other slaves node. # 2nd, Remember to change variable "PASSWORD" before start the main script "sshpass.sh" export FILELOC="/root" export SLAVES=`cat $FILELOC/sshslaves` export USER=root export PASSWD=stonetest for host in $SLAVES echo echo "Ensure ssh passwordless works among all slave nodes..." echo expect -c " set timeout 1 spawn ssh $USER@$host expect \"yes/no\" send -- \"yes\r\" expect eof " done
3. 其他配置
[root@hdp01 ~]# pwd /root [root@hdp01 ~]# cat sshhosts hdp01 hdp02 hdp03 [root@hdp01 ~]# cat sshslaves hdp02 hdp03 [root@hdp01 ~]# ls -lrth | tail -2 -rwxr-xr-x 1 root root 1.3K Sep 18 02:08 ssh4slaves.sh -rwxr-xr-x 1 root root 6.5K Sep 18 02:11 ssh_pass.sh
4. 测试输出
[root@hdp01 ~]# ./ssh_pass.sh stonetest ########################################################### Generating RSA keys for MASTER host hdp01 ########################################################### spawn ssh root@hdp01 The authenticity of host hdp01 (192.168.1.121) cant be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added hdp01,192.168.1.121 (RSA) to the list of known hosts. root@hdp01s password: Last login: Tue Sep 18 02:09:29 2012 from hdp02.dbinterest.local [root@hdp01 ~]# ssh-keygen -t rsa -P -f /root/.ssh/id_rsa Generating public/private rsa key pair. Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: 3a:c3:98:b3:e4:39:fa:fe:87:c6:22:90:16:57:4e:47 root@hdp01.dbinterest.local The keys randomart image is: +--[ RSA 2048]----+ | .E | | o . | | + . | | . . . | | .o S | |o. + . | |.. =.=. | | .oo++o. | | .=*=.. | +-----------------+ [root@hdp01 ~]# ssh-copy-id -i /root/.ssh/id_rsa.pub hdp01 root@hdp01s password: Now try logging into the machine, with "ssh hdp01", and check in: .ssh/authorized_keys to make sure we havent added extra keys that you werent expecting. [root@hdp01 ~]# ########################################################### Generating RSA keys for all OTHER hosts... hostname is hdp02 ########################################################### spawn ssh root@hdp02 The authenticity of host hdp02 (192.168.1.122) cant be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added hdp02,192.168.1.122 (RSA) to the list of known hosts. root@hdp02s password: Last login: Tue Sep 18 02:09:23 2012 from hdp02.dbinterest.local [root@hdp02 ~]# ssh-keygen -t rsa -P -f /root/.ssh/id_rsa Generating public/private rsa key pair. Created directory /root/.ssh. Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: a9:89:fe:40:8a:8e:21:55:da:3b:6b:68:4f:3e:8f:fc root@hdp02.dbinterest.local The keys randomart image is: +--[ RSA 2048]----+ | | | | | . | | + . | | o o S | | o o o o | |+ ..* o | |+.o=o= | |.o oB=E | +-----------------+ [root@hdp02 ~]# ssh-copy-id -i /root/.ssh/id_rsa.pub hdp01 The authenticity of host hdp01 (192.168.1.121) cant be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added hdp01,192.168.1.121 (RSA) to the list of known hosts. root@hdp01s password: Now try logging into the machine, with "ssh hdp01", and check in: .ssh/authorized_keys to make sure we havent added extra keys that you werent expecting. ########################################################### Generating RSA keys for all OTHER hosts... hostname is hdp03 ########################################################### spawn ssh root@hdp03 The authenticity of host hdp03 (192.168.1.123) cant be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added hdp03,192.168.1.123 (RSA) to the list of known hosts. root@hdp03s password: Last login: Tue Sep 18 02:09:19 2012 from hdp02.dbinterest.local [root@hdp03 ~]# ssh-keygen -t rsa -P -f /root/.ssh/id_rsa Generating public/private rsa key pair. Created directory /root/.ssh. Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: a4:3d:dd:54:42:c0:45:ec:ed:ae:d6:bd:14:a0:9b:16 root@hdp03.dbinterest.local The keys randomart image is: +--[ RSA 2048]----+ | ..*= . | | . .o | | . ..o | | + . oo o | | . S .E.. . | | . + . .| | + o o | | . . + .| | ... ..| +-----------------+ [root@hdp03 ~]# ssh-copy-id -i /root/.ssh/id_rsa.pub hdp01 The authenticity of host hdp01 (192.168.1.121) cant be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added hdp01,192.168.1.121 (RSA) to the list of known hosts. root@hdp01s password: Now try logging into the machine, with "ssh hdp01", and check in: .ssh/authorized_keys to make sure we havent added extra keys that you werent expecting. [root@hdp03 ~]# ############################################################################ Copying authorized_keys to host hdp02 from the MASTER host hdp01... ############################################################################ spawn scp /root/.ssh/authorized_keys root@hdp02:/root/.ssh/ root@hdp02s password: authorized_keys 100% 1227 1.2KB/s 00:00 ############################################################################ Copying authorized_keys to host hdp03 from the MASTER host hdp01... ############################################################################ spawn scp /root/.ssh/authorized_keys root@hdp03:/root/.ssh/ root@hdp03s password: authorized_keys 100% 1227 1.2KB/s 00:00 ############################################################################ Distributing the /root/sshslaves file to slave host hdp02... ############################################################################ sshslaves 100% 12 0.0KB/s 00:00 ############################################################################ Distributing the /root/ssh4slaves.sh script to slave host hdp02... ############################################################################ ssh4slaves.sh 100% 1277 1.3KB/s 00:00 ############################################################################ Distributing the /root/sshslaves file to slave host hdp03... ############################################################################ sshslaves 100% 12 0.0KB/s 00:00 ############################################################################ Distributing the /root/ssh4slaves.sh script to slave host hdp03... ############################################################################ ssh4slaves.sh 100% 1277 1.3KB/s 00:00 ############################################################################ Working on the slaves node hdp02 to ensure no prompt for the yes/no question... ############################################################################ Ensure ssh passwordless works among all slave nodes... spawn ssh root@hdp02 The authenticity of host hdp02 (192.168.1.122) cant be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added hdp02,192.168.1.122 (RSA) to the list of known hosts. Last login: Tue Sep 18 02:11:54 2012 from hdp01.dbinterest.local [root@hdp02 ~]# Ensure ssh passwordless works among all slave nodes... spawn ssh root@hdp03 The authenticity of host hdp03 (192.168.1.123) cant be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added hdp03,192.168.1.123 (RSA) to the list of known hosts. Last login: Tue Sep 18 02:11:55 2012 from hdp01.dbinterest.local [root@hdp03 ~]# ############################################################################ Working on the slaves node hdp03 to ensure no prompt for the yes/no question... ############################################################################ Ensure ssh passwordless works among all slave nodes... spawn ssh root@hdp02 The authenticity of host hdp02 (192.168.1.122) cant be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added hdp02,192.168.1.122 (RSA) to the list of known hosts. Last login: Tue Sep 18 02:11:58 2012 from hdp02.dbinterest.local [root@hdp02 ~]# Ensure ssh passwordless works among all slave nodes... spawn ssh root@hdp03 The authenticity of host hdp03 (192.168.1.123) cant be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added hdp03,192.168.1.123 (RSA) to the list of known hosts. Last login: Tue Sep 18 02:11:59 2012 from hdp02.dbinterest.local ############################################################################ Check whether the Passwordless SSH works for hdp01... ############################################################################ Linux hdp01.dbinterest.local 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux Tue Sep 18 02:12:05 PDT 2012 ############################################################################ Check whether the Passwordless SSH works for hdp02... ############################################################################ Linux hdp02.dbinterest.local 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux Tue Sep 18 02:12:05 PDT 2012 ############################################################################ Check whether the Passwordless SSH works for hdp03... ############################################################################ Linux hdp03.dbinterest.local 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux Tue Sep 18 02:12:06 PDT 2012
5. 其他节点测试
[root@hdp02 ~]# [root@hdp02 ~]# ssh hdp02 Last login: Tue Sep 18 02:12:00 2012 from hdp03.dbinterest.local [root@hdp02 ~]# exit logout Connection to hdp02 closed. [root@hdp02 ~]# ssh hdp03 Last login: Tue Sep 18 02:12:02 2012 from hdp03.dbinterest.local [root@hdp03 ~]# exit logout Connection to hdp03 closed. [root@hdp02 ~]# ---------- [root@hdp03 ~]# [root@hdp03 ~]# ssh hdp01 Last login: Tue Sep 18 02:12:22 2012 from hdp02.dbinterest.local [root@hdp01 ~]# exit logout Connection to hdp01 closed. [root@hdp03 ~]# ssh hdp02 Last login: Tue Sep 18 02:12:25 2012 from hdp02.dbinterest.local [root@hdp02 ~]# exit logout Connection to hdp02 closed. [root@hdp03 ~]# ssh hdp03 Last login: Tue Sep 18 02:12:30 2012 from hdp02.dbinterest.local [root@hdp03 ~]# exit logout Connection to hdp03 closed. [root@hdp03 ~]#
代码下载见附件
参考文章：
http://www.cnblogs.com/iloveyoucc/archive/2012/05/11/2496433.html
http://f.dataguru.cn/thread-19920-1-1.html
本文出自 “点滴积累” 博客，请务必保留此出处http://tianxingzhe.blog.51cto.com/3390077/1687600
相关文章
- ssh2登录方式_ssh端口号
- A Visual Guide to SSH Tunnels (with labs)
- H3C | telnet & SSH实验
- GitHub 仓库中 SSH 与 HTTPS 的区别
- 利用反向ssh从外网访问内网主机
- Coding配置SSH到提交代码（linux）
- ssh 无密码登录详解程序员
- Linux无密码SSH安全连接的极致体验（linux无密码ssh）
- 登录实现Linux的免密码SSH登录（linux免密码ssh）
- 配置 Linux 服务器 SSH 远程登录（linux配置ssh）
- 轻松玩转Linux服务器：SSH远程连接指南（ssh远程连接linux）
- 探索Linux：极致SSH体验（linuxssh版本）
- Linux下快速安装SSH服务（linux下安装ssh）
- 数据库搭建SSH框架：深入MySQL数据库（ssh框架mysql）
- Linux之神秘：SSH的奥秘（linux软件ssh）
- Linux下开启SSH远程管理：一步一步学习（linux打开ssh）
- 使用SSH登录Redis服务器（ssh 登陆redis）
- 精彩融合SSH框架与Redis的整合（ssh框架整合redis）
- SSH实现Redis集群整合赋予资源集群新的生命力（ssh整合redis集群）