Redis Cluster集群维护:动态扩容
集群运行时间长久之后,难免由于硬件故障、网络规划、业务增长等原因对已有集群进行相应的调整, 比如增加
Redis node节点、减少节点、节点迁移、更换服务器等。
增加节点和删除节点会涉及到已有的槽位重新分配及数据迁移。
实战案例:
因公司业务发展迅猛,现有的三主三从的redis cluster架构可能无法满足现有业务的并发写入需求,因此公司紧急采购两台服务器10.0.0.68,10.0.0.78,需要将其动态添加到集群当中,但不能影响业务使用和数据丢失。
添加节点准备
增加Redis node节点,需要与之前的Redis node版本相同、配置一致,然后分别再启动两台Redis node,应为一主一从。
#配置node7节点 [root@redis-node7 ~]#dnf -y install redis [root@redis-node7 ~]#sed -i.bak -e s/bind 127.0.0.1/bind 0.0.0.0/ -e /masterauth/a masterauth 123456 -e /# requirepass/a requirepass 123456 -e /# cluster-enabled yes/a cluster-enabled yes -e /# cluster-config-file nodes-6379.conf/a cluster-config-file nodes-6379.conf -e /appendonly no/c appendonly yes /etc/redis.conf [root@redis-node7 ~]#systemctl start redis #配置node8节点 [root@redis-node8 ~]#dnf -y install redis [root@redis-node8 ~]#sed -i.bak -e s/bind 127.0.0.1/bind 0.0.0.0/ -e /masterauth/a masterauth 123456 -e /# requirepass/a requirepass 123456 -e /# cluster-enabled yes/a cluster-enabled yes -e /# cluster-config-file nodes-6379.conf/a cluster-config-file nodes-6379.conf -e /appendonly no/c appendonly yes /etc/redis.conf [root@redis-node8 ~]#systemctl start redis
添加节点到集群
使用以下命令添加新节点,要添加的新redis节点IP和端口添加到的已有的集群中任意节点的IP:端口
add-node new_host:new_port existing_host:existing_port new_host:new_port #为新添加的主机的IP和端口 existing_host:existing_port #为已有的集群中任意节点的IP和端口
Redis 4添加方式:
把新的Redis 节点192.168.7.107添加到当前Redis集群当中。
[root@redis-node1 ~]# redis-trib.rb add-node 10.0.0.68:6379 10.0.0.58:6379
Redis 5添加方式:
#将第一台新的主机都加入集群 [root@redis-node1 ~]#redis-cli -a 123456 --cluster add-node 10.0.0.68:6379 10.0.0.58:6379 Warning: Using a password with -a or -u option on the command line interface may not be safe. > Adding node 10.0.0.68:6379 to cluster 10.0.0.58:6379 > Performing Cluster Check (using node 10.0.0.58:6379) S: 9875b50925b4e4f29598e6072e5937f90df9fc71 10.0.0.58:6379 slots: (0 slots) slave replicates d34da8666a6f587283a1c2fca5d13691407f9462 M: d04e524daec4d8e22bdada7f21a9487c2d3e1057 10.0.0.48:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s) M: d34da8666a6f587283a1c2fca5d13691407f9462 10.0.0.28:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s) S: 99720241248ff0e4c6fa65c2385e92468b3b5993 10.0.0.18:6379 slots: (0 slots) slave replicates d04e524daec4d8e22bdada7f21a9487c2d3e1057 S: f9adcfb8f5a037b257af35fa548a26ffbadc852d 10.0.0.38:6379 slots: (0 slots) slave replicates cb028b83f9dc463d732f6e76ca6bbcd469d948a7 M: cb028b83f9dc463d732f6e76ca6bbcd469d948a7 10.0.0.8:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. > Check for open slots... > Check slots coverage... [OK] All 16384 slots covered. > Send CLUSTER MEET to node 10.0.0.68:6379 to make it join the cluster. [OK] New node added correctly. #观察到已经加入成功,但无没有slot位,而且新的主机是master [root@redis-node1 ~]#redis-cli -a 123456 --cluster info 10.0.0.8:6379 Warning: Using a password with -a or -u option on the command line interface may not be safe. 10.0.0.8:6379 (cb028b83...) - 6672 keys | 5461 slots | 1 slaves. 10.0.0.78:6379 (f67f1c02...) - 0 keys | 0 slots | 0 slaves. 10.0.0.68:6379 (d6e2eca6...) - 0 keys | 0 slots | 0 slaves. 10.0.0.48:6379 (d04e524d...) - 6679 keys | 5462 slots | 1 slaves. 10.0.0.28:6379 (d34da866...) - 6649 keys | 5461 slots | 1 slaves. [OK] 20000 keys in 5 masters. 1.22 keys per slot on average. [root@redis-node1 ~]#redis-cli -a 123456 --cluster check 10.0.0.8:6379 Warning: Using a password with -a or -u option on the command line interface may not be safe. 10.0.0.8:6379 (cb028b83...) - 6672 keys | 5461 slots | 1 slaves. 10.0.0.78:6379 (f67f1c02...) - 0 keys | 0 slots | 0 slaves. 10.0.0.68:6379 (d6e2eca6...) - 0 keys | 0 slots | 0 slaves. 10.0.0.48:6379 (d04e524d...) - 6679 keys | 5462 slots | 1 slaves. 10.0.0.28:6379 (d34da866...) - 6649 keys | 5461 slots | 1 slaves. [OK] 20000 keys in 5 masters. 1.22 keys per slot on average. > Performing Cluster Check (using node 10.0.0.8:6379) M: cb028b83f9dc463d732f6e76ca6bbcd469d948a7 10.0.0.8:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s) M: d6e2eca6b338b717923f64866bd31d42e52edc98 10.0.0.68:6379 slots: (0 slots) master S: 9875b50925b4e4f29598e6072e5937f90df9fc71 10.0.0.58:6379 slots: (0 slots) slave replicates d34da8666a6f587283a1c2fca5d13691407f9462 S: f9adcfb8f5a037b257af35fa548a26ffbadc852d 10.0.0.38:6379 slots: (0 slots) slave replicates cb028b83f9dc463d732f6e76ca6bbcd469d948a7 M: d04e524daec4d8e22bdada7f21a9487c2d3e1057 10.0.0.48:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s) S: 99720241248ff0e4c6fa65c2385e92468b3b5993 10.0.0.18:6379 slots: (0 slots) slave replicates d04e524daec4d8e22bdada7f21a9487c2d3e1057 M: d34da8666a6f587283a1c2fca5d13691407f9462 10.0.0.28:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. > Check for open slots... > Check slots coverage... [OK] All 16384 slots covered. [root@redis-node1 ~]#cat /var/lib/redis/nodes-6379.conf d6e2eca6b338b717923f64866bd31d42e52edc98 10.0.0.68:6379@16379 master - 0 1582356107260 8 connected 9875b50925b4e4f29598e6072e5937f90df9fc71 10.0.0.58:6379@16379 slave d34da8666a6f587283a1c2fca5d13691407f9462 0 1582356110286 6 connected f9adcfb8f5a037b257af35fa548a26ffbadc852d 10.0.0.38:6379@16379 slave cb028b83f9dc463d732f6e76ca6bbcd469d948a7 0 1582356108268 4 connected d04e524daec4d8e22bdada7f21a9487c2d3e1057 10.0.0.48:6379@16379 master - 0 1582356105000 7 connected 5461-10922 99720241248ff0e4c6fa65c2385e92468b3b5993 10.0.0.18:6379@16379 slave d04e524daec4d8e22bdada7f21a9487c2d3e1057 0 1582356108000 7 connected d34da8666a6f587283a1c2fca5d13691407f9462 10.0.0.28:6379@16379 master - 0 1582356107000 3 connected 10923-16383 cb028b83f9dc463d732f6e76ca6bbcd469d948a7 10.0.0.8:6379@16379 myself,master - 0 1582356106000 1 connected 0-5460 vars currentEpoch 8 lastVoteEpoch 7 #和上面显示结果一样 [root@redis-node1 ~]#redis-cli -a 123456 CLUSTER NODES Warning: Using a password with -a or -u option on the command line interface may not be safe. d6e2eca6b338b717923f64866bd31d42e52edc98 10.0.0.68:6379@16379 master - 0 1582356313200 8 connected 9875b50925b4e4f29598e6072e5937f90df9fc71 10.0.0.58:6379@16379 slave d34da8666a6f587283a1c2fca5d13691407f9462 0 1582356311000 6 connected f9adcfb8f5a037b257af35fa548a26ffbadc852d 10.0.0.38:6379@16379 slave cb028b83f9dc463d732f6e76ca6bbcd469d948a7 0 1582356314208 4 connected d04e524daec4d8e22bdada7f21a9487c2d3e1057 10.0.0.48:6379@16379 master - 0 1582356311182 7 connected 5461-10922 99720241248ff0e4c6fa65c2385e92468b3b5993 10.0.0.18:6379@16379 slave d04e524daec4d8e22bdada7f21a9487c2d3e1057 0 1582356312000 7 connected d34da8666a6f587283a1c2fca5d13691407f9462 10.0.0.28:6379@16379 master - 0 1582356312190 3 connected 10923-16383 cb028b83f9dc463d732f6e76ca6bbcd469d948a7 10.0.0.8:6379@16379 myself,master - 0 1582356310000 1 connected 0-5460 #查看集群状态 [root@redis-node1 ~]#redis-cli -a 123456 CLUSTER INFO Warning: Using a password with -a or -u option on the command line interface may not be safe. cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:7 cluster_size:3 cluster_current_epoch:8 cluster_my_epoch:1 cluster_stats_messages_ping_sent:17442 cluster_stats_messages_pong_sent:13318 cluster_stats_messages_fail_sent:4 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_sent:30765 cluster_stats_messages_ping_received:13311 cluster_stats_messages_pong_received:13367 cluster_stats_messages_meet_received:7 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:26687 [root@redis-node1 ~]#
重新分配槽位
新的node节点加到集群之后默认是master节点,但是没有slots数据,需要重新分配
添加主机之后需要对添加至集群种的新主机重新分片否则其没有分片也就无法写入数据。
验证当前状态:
redis4的命令
[root@redis-node1 ~]# redis-trib.rb check 10.0.0.68:6379 #当前状态 [root@redis-node1 ~]# redis-trib.rb reshard 10.0.0.68:6379 #重新分片
Redis 5:
[root@redis-node1 ~]#redis-cli -a 123456 --cluster reshard 10.0.0.68:6379 Warning: Using a password with -a or -u option on the command line interface may not be safe. > Performing Cluster Check (using node 10.0.0.68:6379) M: d6e2eca6b338b717923f64866bd31d42e52edc98 10.0.0.68:6379 slots: (0 slots) master M: d34da8666a6f587283a1c2fca5d13691407f9462 10.0.0.28:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s) M: d04e524daec4d8e22bdada7f21a9487c2d3e1057 10.0.0.48:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s) M: cb028b83f9dc463d732f6e76ca6bbcd469d948a7 10.0.0.8:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s) S: 99720241248ff0e4c6fa65c2385e92468b3b5993 10.0.0.18:6379 slots: (0 slots) slave replicates d04e524daec4d8e22bdada7f21a9487c2d3e1057 M: f67f1c02c742cd48d3f48d8c362f9f1b9aa31549 10.0.0.78:6379 slots: (0 slots) master S: f9adcfb8f5a037b257af35fa548a26ffbadc852d 10.0.0.38:6379 slots: (0 slots) slave replicates cb028b83f9dc463d732f6e76ca6bbcd469d948a7 S: 9875b50925b4e4f29598e6072e5937f90df9fc71 10.0.0.58:6379 slots: (0 slots) slave replicates d34da8666a6f587283a1c2fca5d13691407f9462 [OK] All nodes agree about slots configuration. > Check for open slots... > Check slots coverage... [OK] All 16384 slots covered. How many slots do you want to move (from 1 to 16384)?4096 #分配多少个槽位=16384/master个数 What is the receiving node ID? d6e2eca6b338b717923f64866bd31d42e52edc98 #10.0.0.68的ID Please enter all the source node IDs. Type all to use all the nodes as source nodes for the hash slots. Type done once you entered all the source nodes IDs. Source node #1: all #将哪些源主机的槽位分配给新的节点,all是自动在所有的redis node选择划分,如果是从redis cluster删除某个主机可以使用此方式将指定主机上的槽位全部移动到别的redis主机 ...... Do you want to proceed with the proposed reshard plan (yes/no)? yes #确认分配 ...... Moving slot 12280 from 10.0.0.28:6379 to 10.0.0.68:6379: . Moving slot 12281 from 10.0.0.28:6379 to 10.0.0.68:6379: . Moving slot 12282 from 10.0.0.28:6379 to 10.0.0.68:6379: Moving slot 12283 from 10.0.0.28:6379 to 10.0.0.68:6379: .. Moving slot 12284 from 10.0.0.28:6379 to 10.0.0.68:6379: Moving slot 12285 from 10.0.0.28:6379 to 10.0.0.68:6379: . Moving slot 12286 from 10.0.0.28:6379 to 10.0.0.68:6379: Moving slot 12287 from 10.0.0.28:6379 to 10.0.0.68:6379: .. [root@redis-node1 ~]# #确定slot分配成功 [root@redis-node1 ~]#redis-cli -a 123456 --cluster check 10.0.0.8:6379 Warning: Using a password with -a or -u option on the command line interface may not be safe. 10.0.0.8:6379 (cb028b83...) - 5019 keys | 4096 slots | 1 slaves. 10.0.0.68:6379 (d6e2eca6...) - 4948 keys | 4096 slots | 0 slaves. 10.0.0.48:6379 (d04e524d...) - 5033 keys | 4096 slots | 1 slaves. 10.0.0.28:6379 (d34da866...) - 5000 keys | 4096 slots | 1 slaves. [OK] 20000 keys in 5 masters. 1.22 keys per slot on average. > Performing Cluster Check (using node 10.0.0.8:6379) M: cb028b83f9dc463d732f6e76ca6bbcd469d948a7 10.0.0.8:6379 slots:[1365-5460] (4096 slots) master 1 additional replica(s) M: d6e2eca6b338b717923f64866bd31d42e52edc98 10.0.0.68:6379 slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master #可看到4096个slots S: 9875b50925b4e4f29598e6072e5937f90df9fc71 10.0.0.58:6379 slots: (0 slots) slave replicates d34da8666a6f587283a1c2fca5d13691407f9462 S: f9adcfb8f5a037b257af35fa548a26ffbadc852d 10.0.0.38:6379 slots: (0 slots) slave replicates cb028b83f9dc463d732f6e76ca6bbcd469d948a7 M: d04e524daec4d8e22bdada7f21a9487c2d3e1057 10.0.0.48:6379 slots:[6827-10922] (4096 slots) master 1 additional replica(s) S: 99720241248ff0e4c6fa65c2385e92468b3b5993 10.0.0.18:6379 slots: (0 slots) slave replicates d04e524daec4d8e22bdada7f21a9487c2d3e1057 M: d34da8666a6f587283a1c2fca5d13691407f9462 10.0.0.28:6379 slots:[12288-16383] (4096 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. > Check for open slots... > Check slots coverage... [OK] All 16384 slots covered.
为新的master添加slave节点
需要再向当前的Redis集群中添加一个Redis单机服务器10.0.0.78,用于解决当前10.0.0.68单机的潜在宕机问题,即实现响应的高可用功能,有两种方式:
方法1:在新加节点到集群时,直接将之设置为slave
查看当前状态 [root@redis-node1 ~]#redis-cli -a 123456 --cluster check 10.0.0.8:6379 Warning: Using a password with -a or -u option on the command line interface may not be safe. 10.0.0.8:6379 (cb028b83...) - 5019 keys | 4096 slots | 1 slaves. 10.0.0.68:6379 (d6e2eca6...) - 4948 keys | 4096 slots | 0 slaves. 10.0.0.48:6379 (d04e524d...) - 5033 keys | 4096 slots | 1 slaves. 10.0.0.28:6379 (d34da866...) - 5000 keys | 4096 slots | 1 slaves. [OK] 20000 keys in 4 masters. 1.22 keys per slot on average. > Performing Cluster Check (using node 10.0.0.8:6379) M: cb028b83f9dc463d732f6e76ca6bbcd469d948a7 10.0.0.8:6379 slots:[1365-5460] (4096 slots) master 1 additional replica(s) M: d6e2eca6b338b717923f64866bd31d42e52edc98 10.0.0.68:6379 slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master S: 9875b50925b4e4f29598e6072e5937f90df9fc71 10.0.0.58:6379 slots: (0 slots) slave replicates d34da8666a6f587283a1c2fca5d13691407f9462 S: f9adcfb8f5a037b257af35fa548a26ffbadc852d 10.0.0.38:6379 slots: (0 slots) slave replicates cb028b83f9dc463d732f6e76ca6bbcd469d948a7 M: d04e524daec4d8e22bdada7f21a9487c2d3e1057 10.0.0.48:6379 slots:[6827-10922] (4096 slots) master 1 additional replica(s) S: 99720241248ff0e4c6fa65c2385e92468b3b5993 10.0.0.18:6379 slots: (0 slots) slave replicates d04e524daec4d8e22bdada7f21a9487c2d3e1057 M: d34da8666a6f587283a1c2fca5d13691407f9462 10.0.0.28:6379 slots:[12288-16383] (4096 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. > Check for open slots... > Check slots coverage... [OK] All 16384 slots covered. #直接加为slave节点 [root@redis-node1 ~]#redis-cli -a 123456 --cluster add-node 10.0.0.78:6379 10.0.0.8:6379 --cluster-slave --cluster-master-id d6e2eca6b338b717923f64866bd31d42e52edc98 #验证是否成功 [root@redis-node1 ~]#redis-cli -a 123456 --cluster check 10.0.0.8:6379 Warning: Using a password with -a or -u option on the command line interface may not be safe. 10.0.0.8:6379 (cb028b83...) - 5019 keys | 4096 slots | 1 slaves. 10.0.0.68:6379 (d6e2eca6...) - 4948 keys | 4096 slots | 1 slaves. 10.0.0.48:6379 (d04e524d...) - 5033 keys | 4096 slots | 1 slaves. 10.0.0.28:6379 (d34da866...) - 5000 keys | 4096 slots | 1 slaves. [OK] 20000 keys in 4 masters. 1.22 keys per slot on average. > Performing Cluster Check (using node 10.0.0.8:6379) M: cb028b83f9dc463d732f6e76ca6bbcd469d948a7 10.0.0.8:6379 slots:[1365-5460] (4096 slots) master 1 additional replica(s) M: d6e2eca6b338b717923f64866bd31d42e52edc98 10.0.0.68:6379 slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master 1 additional replica(s) S: 36840d7eea5835ba540d9b64ec018aa3f8de6747 10.0.0.78:6379 slots: (0 slots) slave replicates d6e2eca6b338b717923f64866bd31d42e52edc98 S: 9875b50925b4e4f29598e6072e5937f90df9fc71 10.0.0.58:6379 slots: (0 slots) slave replicates d34da8666a6f587283a1c2fca5d13691407f9462 S: f9adcfb8f5a037b257af35fa548a26ffbadc852d 10.0.0.38:6379 slots: (0 slots) slave replicates cb028b83f9dc463d732f6e76ca6bbcd469d948a7 M: d04e524daec4d8e22bdada7f21a9487c2d3e1057 10.0.0.48:6379 slots:[6827-10922] (4096 slots) master 1 additional replica(s) S: 99720241248ff0e4c6fa65c2385e92468b3b5993 10.0.0.18:6379 slots: (0 slots) slave replicates d04e524daec4d8e22bdada7f21a9487c2d3e1057 M: d34da8666a6f587283a1c2fca5d13691407f9462 10.0.0.28:6379 slots:[12288-16383] (4096 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. > Check for open slots... > Check slots coverage... [OK] All 16384 slots covered.
方法2:先将新节点加入集群,再修改为slave
为新的master添加slave节点
命令格式:
Redis 3/4:
[root@redis-node1 ~]#redis-trib.rb add-node 10.0.0.78:6379 10.0.0.8:6379
Redis 5版本:
把10.0.0.78:6379添加到集群中:
[root@redis-node1 ~]#redis-cli -a 123456 --cluster add-node 10.0.0.78:6379 10.0.0.8:6379
更改新节点更改状态为slave:
需要手动将其指定为某个master的slave,否则其默认角色为master。
[root@redis-node1 ~]#redis-cli -h 10.0.0.78 -p 6339 -a 123456 #登录到新添加节点 10.0.0.78:6380 CLUSTER NODES #查看当前集群节点,找到目标master 的ID 10.0.0.78:6380 CLUSTER REPLICATE 886338acd50c3015be68a760502b239f4509881c #将其设置slave,命令格式为cluster replicate MASTERID 10.0.0.78:6380 CLUSTER NODES #再次查看集群节点状态,验证节点是否已经更改为指定master 的slave
本文链接:http://www.yunweipai.com/35538.html
原创文章,作者:ItWorker,如若转载,请注明出处:https://blog.ytso.com/52787.html
Redis相关文章
- Java连接redis集群操作存储、删除以及获取值详解编程语言
- 如何解决Redis空闲连接(redis空闲连接)
- 快速查找数据:Redis实现范围查找(redis查找范围)
- key处理Redis Java实现自动删除过期key(redisjava过期)
- 【面试必备】Redis缓存面试题攻略(redis缓存面试题)
- 避免陷入坑中:深入理解 Redis 集群中的缺陷与挑战(redis集群的坑)
- 利用循环加速Redis数据获取速度(循环redis取数据)
- 库存膨胀如何利用Redis解决高并发问题(库存高并发redis)
- 利用Redis实现精准的标签系统(标签系统redis)
- 如何快捷地在服务器上安装Redis(服务器怎么装redis)
- 大优点Redis集群架构五大显著优势(redis集群架构五)
- Redis集群攻克最大的挑战卡槽限制(redis集群最大卡槽)
- Redis集群支持中文字符的乱码神器(redis集群支持乱码)
- Redis集群实现多租户安全管理(redis集群多租户)
- 如何成功地配置 Redis集群与哨兵(redis集群和哨兵配置)
- Redis集群实现分区优化(redis 集群 分区)
- Redis集群基于Resharding协议的使用介绍(redis集群使用的协议)
- 如何构建Redis集群从节点的角度(redis 集群 从节点)
- Redis集群中从节点之间的数据同步(redis集群从从同步)
- 重新定义Redis动态键值规划(redis 键值规划)
- Redis退出集群终止命令(redis退出集群命令)
- 红色的远程性能低劣Redis远程调用的性能分析(redis远程性能差)
- 设置适当的Redis过期时间关键策略(redis 过期时间策略)
- Redis过去与未来相融的成功之道(redis过去策略)
- Redis集群SETNX实现多节点协同管理示例(redis集群setnx)
- Redis精准计算过期时间(redis 过期时间计算)
- 中过期数据重新洗牌Redis在集群中清除过期数据(redis 清理掉集群)