• 幕客老师召集小伙伴
  • python自动化运维项目实战
  • nginx从入门到实战
  • 阿里云与Centos7实战

Shell实现Redis重新分片脚本

一、实现Redis的重新分片步骤

步骤一、redis-trib对目标节点发送CLUSTER SETSLOT <slot> IMPORTING <source_id> 命令,让目标节点准备好从源节点导入(import)属于槽slot的键值对。

步骤二、redis-trib对源节点发送CLUSTER SETSLOT <slot> MIGRATING <target_id> 命令,让源节点准备好将属于槽slot的键值对迁移(migrate)至目标节点。

步骤三、redis-trib向源节点发送CLUSTER SETKEYSINSLOT <slot> <count>命令,获得最多count个属于槽slot的键值对的键名(key_name)。

步骤四、对于步骤3获得每个键名,redis-strib都向源节点发送一个MIGRATE <target_ip> <target_port> <key_name> 0 <timeout> 将被选中的键原子地从源节点迁移到目标节点。

步骤五、重复执行步骤3和步骤4,直到源节点保存的所有属性slot的键值都被迁移至目标节点为止。

步骤六、向集群中任意一个节点发送CLUSTER SETSLOT <slot> NODE <target_id>命令,将slot指派給目标节点,指派信息会通过消息发送到整个集群,最终集群中所有节点都会知道槽slot已经指派给了目标节点。

 

二、脚本操作步骤

如,现在我们已有一组集群分别为:www.iaskjob.com:6379\www.iaskjob.com:6380\www.iaskjob.com:6381

需要加入一组服务:www.iaskjob.com:6382

1)第一步先加入节点

redis-cli -h www.iaskjob.com -p 6379 CLUSTER MEET www.iaskjob.com 6382

2)按照如下格式执行脚本

./redisshared.sh www.iaskjob.com 6379 www.iaskjob.com 6382 7001 10000

3)验证

www.iaskjob.com:6379> CLUSTER NODES
81d1ee4a5ced0cd5df48baac6e931d97e99a514b www.iaskjob.com:6381 master - 0 1458877723064 2 connected 10001-16383
7b66b3a23e762e0b1b018a931398afb2b36b4527 www.iaskjob.com:6382 master - 0 1458877722062 4 connected 7001-10000
15bf370e8b37ee9d742c5f19051f9a0e1e5ab6f4 www.iaskjob.com:6379 myself,master - 0 0 3 connected 0-5000
8ddd09065eec9c796b93608003b400e5e913fcb5 www.iaskjob.com:6380 master - 0 1458877721059 1 connected 5001-7000

三、脚本内容如下:

#/bin/sh
#########################################################################
# File Name: redisshared.sh
# Program function:
# Author:Jeson
# mail:iaskjob@163.com
# Created Time: 3/24 10:50:25 2016201
#========================================================================
#!/bin/bash
#
# Moves/Shards Redis slots
#

shopt -s -o nounset

declare -rx SCRIPT=${0##*/}

declare -rx redis='/usr/local/bin/redis-cli'

declare -x SRC_IP=${1}
declare -x SRC_PORT=${2}
declare -x DEST_IP=${3}
declare -x DEST_PORT=${4}
declare -x HSLOT_START=${5}
declare -x HSLOT_END=${6}
# redis-shard.sh <src-ip> <src-port> <dest-ip> <dest-port> <hslot-start> <hslot-end>

if test ! -x "${redis}"; then
    printf "${SCRIPT}:${LINENO}: the command ${redis} is not available - aborting\n" >&2
    exit 197
fi

function getClusterNodes() {
# Gathers node ip:port and node ID of all known cluster nodes
echo "Discovering cluster nodes"
sleep 5
declare -a cluster_nodes=$(redis-cli -h ${SRC_IP} -p ${SRC_PORT} CLUSTER NODES | awk '{print $2"|"$1}');
echo "Cluster Nodes:";
echo "IP:PORT | NODE_ID"
for node in ${cluster_nodes[@]}; do
echo "${node}"
done
return 0
}

function clusterMeet() {
# Make all the nodes aware of each other. It is not necessary to
# execute a CLUSTER MEET on all of the other nodes, as all nodes exchange information
# about other nodes.
echo "Meeting with cluster ${DEST_IP}:${DEST_PORT}"
echo "redis-cli -h ${SRC_IP} -p ${SRC_PORT} CLUSTER MEET ${DEST_IP} ${DEST_PORT}"
}

function clusterSlotImport() {
# Changes the hash slot state to importing. It must be executed at
# the node that is going to receive the hash slot, and the node ID of the current
# slot owner must be passed in.
# CLUSTER SETSLOT <hash-slot> IMPORTING <source-id>
local hslot=${1}
local src_id=${2}
echo "redis-cli -h ${DEST_IP} -p ${DEST_PORT} CLUSTER SETSLOT ${hslot} IMPORTING ${src_id}"
redis-cli -h ${DEST_IP} -p ${DEST_PORT} CLUSTER SETSLOT ${hslot} IMPORTING ${src_id}
return 0
}

function clusterSlotMigrate() {
# Changes the hash slot state to migrating. It is the opposite of the importing
# subcommand. It must me executed at the node that owns the hash slot, and the node ID
# of the new slot owner must be passed in.
# CLUSTER SETSLOT <hash-slot> MIGRATING <destination-id>
local hslot=${1}
local dest_id=${2}
echo "redis-cli -h ${SRC_IP} -p ${SRC_PORT} CLUSTER SETSLOT ${hslot} MIGRATING ${dest_id}"
redis-cli -h ${SRC_IP} -p ${SRC_PORT} CLUSTER SETSLOT ${hslot} MIGRATING ${dest_id}
return 0
}

function clusterSetSlotNode() {
# Associates a hash slot with a node. It must be executed on the source and destination nodes.
# Executing it on all master nodes is also recommended to avoid wrong redirects while the
# propagation takes place.
# When executed on destination node, the importing state is cleared and then the configuration
# epoch is updated.
# When executed on the source node, the migrating state is cleared as long as no keys exist
# in that slot. Otherwise and error is thrown.
# CLUSTER SETSLOT <hash-slot> MIGRATING <owner-id>
local hslot=${1}
local dest_id=${2}
master_nodes=$(redis-cli -h ${SRC_IP} -p ${SRC_PORT} CLUSTER NODES | grep master | awk '{print$2}')
if [ -z "${master_nodes}" ]; then
echo "No other masters found"
else
for master in ${master_nodes[@]}; do
ip=$(echo "${master}" | awk -F: '{print $1}')
port=$(echo "${master}" | awk -F: '{print $2}')
echo "redis-cli -h ${ip} -p ${port} CLUSTER SETSLOT ${hslot} NODE ${dest_id}"
redis-cli -h ${ip} -p ${port} CLUSTER SETSLOT ${hslot} NODE ${dest_id}
done
fi
#echo "redis-cli -h ${SRC_IP} -p ${SRC_PORT} CLUSTER SETSLOT ${hslot} NODE ${dest_id}"
#echo "redis-cli -h ${DEST_IP} -p ${DEST_PORT} CLUSTER SETSLOT ${hslot} NODE ${dest_id}"
return 0
}

function clusterSetSlotStable() {
# Clears any state of a hash slot (importing or migrating). It is useful wen a rollback in a resharding
return 0
}

function migrateKey() {
# CLUSTER COUNTKEYSINSLOT <slot>
# CLUSTER GETKEYSINSLOT
# MIGRATE <host> <port> <key> <db> <timeout>
local hslot=${1}
key_cnt=$(redis-cli -h ${SRC_IP} -p ${SRC_PORT} CLUSTER COUNTKEYSINSLOT ${hslot} | awk '{print$1}')
if [[ ${key_cnt} -ne 0 ]]; then
keys=$(redis-cli -h ${SRC_IP} -p ${SRC_PORT} CLUSTER GETKEYSINSLOT ${hslot} ${key_cnt} | awk '{print$1}')
for key in ${keys[@]}; do
echo "Migrating ${key} from ${hslot} to ${DEST_IP} ${DEST_PORT}"
#echo "redis-cli -h ${SRC_IP} -p ${SRC_PORT} MIGRATE ${DEST_IP} ${DEST_PORT} ${key} 0 2000"
redis-cli -h ${SRC_IP} -p ${SRC_PORT} MIGRATE ${DEST_IP} ${DEST_PORT} ${key} 0 2000
done
else
echo "No keys found in ${hslot}"
fi
return 0
}

function slotPusher() {
# Main executor of slot commands
local src_id=$(redis-cli -h ${SRC_IP} -p ${SRC_PORT} CLUSTER NODES | grep "${SRC_IP}:${SRC_PORT}" | awk '{print$1}')
local dest_id=$(redis-cli -h ${DEST_IP} -p ${DEST_PORT} CLUSTER NODES | grep "${DEST_IP}:${DEST_PORT}" | awk '{print$1}
')
echo "${DEST_IP}:${DEST_PORT}"
echo "dest_id............"${dest_id}
for slot in $( seq $HSLOT_START $HSLOT_END); do
echo ${slot}
clusterSlotImport ${slot} ${src_id}
clusterSlotMigrate ${slot} ${dest_id}
migrateKey ${slot}
clusterSetSlotNode ${slot} ${dest_id}
#clusterSetSlotStable
done
return 0
}

function execute_main() {
# Main program
clusterMeet
slotPusher
}

execute_main

exit 0

Shell实现Redis重新分片脚本

Pingbacks已打开。

引用地址

暂无评论

发表评论