RedisRoad - redis 使用 unlink 删除大 key
2023-02-18 16:39:35 时间
什么是 Redis 大 Key
string
类型中的值大于10kb
hash, list, set, zset
中的元素超过5000个
如何查找大 Key
string
通过命令直接查找
redis-cli -h 127.0.0.1 -p6379 -a "YourPassword" --bigkeys
- 使用
RdbTools
工具
rdb dump.rdb -c memory --bytes 10240 -f redis.csv
怎么删除 Redis 中的 大 Key
风险点: 直接删除大 Key 会造成阻塞。由于 redis 是 单线程 执行, 阻塞可能造成其他所有请求超时。如果超时越来越多,则可能会造成 redis 链接耗尽, 引发其他异常。
因此, 解决方案可以有如下几种选择
- 业务低峰期删除 :配合 redis 的监控, 在业务低峰期进行删除。但这也是治标不治本, 没有真正解决 阻塞 问题。
- 分批次删除 : 大事化小
hash
使用hsacn
扫描法。set
使用 srandmember 每次随机抽取数据删除。zset
使用zremrangebyrank
删除list
使用pop
删除
- 异步删除: 使用
unlink
命令 代替del
命令。使用unlink
后, redis 会将 key 放入到一个 异步线程 中进行删除, 这样就不会阻塞主线程了。
redis 中的删除方法 del
和 unlink
- del命令使用同步删除,unlink使用异步删除。
- 在删除数据体量很小的简单类型时建议使用del命令,在删除大key时应该使用unlink命令。
- 删除小key使用del的原因是:虽然del是同步删除,会阻塞主线程,但是unlink同样会在主线程执行一些判断和其它操作。而这些操作可能带来的开销比实际删除一个小key还略大。所以能直接删的key就没必要使用异步删除了。
############################# LAZY FREEING ####################################
# Redis has two primitives to delete keys. One is called DEL and is a blocking
# deletion of the object. It means that the server stops processing new commands
# in order to reclaim all the memory associated with an object in a synchronous
# way. If the key deleted is associated with a small object, the time needed
# in order to execute the DEL command is very small and comparable to most other
# O(1) or O(log_N) commands in Redis. However if the key is associated with an
# aggregated value containing millions of elements, the server can block for
# a long time (even seconds) in order to complete the operation.
#
# For the above reasons Redis also offers non blocking deletion primitives
# such as UNLINK (non blocking DEL) and the ASYNC option of FLUSHALL and
# FLUSHDB commands, in order to reclaim memory in background. Those commands
# are executed in constant time. Another thread will incrementally free the
# object in the background as fast as possible.
#
# DEL, UNLINK and ASYNC option of FLUSHALL and FLUSHDB are user-controlled.
# It's up to the design of the application to understand when it is a good
# idea to use one or the other. However the Redis server sometimes has to
# delete keys or flush the whole database as a side effect of other operations.
# Specifically Redis deletes objects independently of a user call in the
# following scenarios:
#
# 1) On eviction, because of the maxmemory and maxmemory policy configurations,
# in order to make room for new data, without going over the specified
# memory limit.
# 2) Because of expire: when a key with an associated time to live (see the
# EXPIRE command) must be deleted from memory.
# 3) Because of a side effect of a command that stores data on a key that may
# already exist. For example the RENAME command may delete the old key
# content when it is replaced with another one. Similarly SUNIONSTORE
# or SORT with STORE option may delete existing keys. The SET command
# itself removes any old content of the specified key in order to replace
# it with the specified string.
# 4) During replication, when a replica performs a full resynchronization with
# its master, the content of the whole database is removed in order to
# load the RDB file just transferred.
#
# In all the above cases the default is to delete objects in a blocking way,
# like if DEL was called. However you can configure each case specifically
# in order to instead release memory in a non-blocking way like if UNLINK
# was called, using the following configuration directives:
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
相关文章
- 第 11 篇:基于 drf-haystack 的文章搜索接口
- 第 10 篇 评论接口
- 第 9 篇:实现分类、标签、归档日期接口
- 第 8 篇:内容支持 Markdown 语法,接口返回包含解析后的 HTML
- 第 7 篇:文章详情的 API 接口
- 第 5 篇:用视图集,简化你的代码
- 第 4 篇:用类视图实现首页 API
- 第 3 篇:实现博客首页文章列表 API
- 初始化 RESTful API 风格的博客系统
- Chrome 离线安装 Proxy SwitchyOmega 重启后消失解决办法
- 程序员必备之终端模拟器,让你的终端世界多一抹“颜色”
- 搭建博客、自己的小窝?快来看看这些开源静态网站生成器
- 有了这些开源动效项目,设计和开发不再相杀只剩相爱
- 优雅地使用 C++ 制作表格:tabulate
- 真·程序员必修书单
- 命令行版的斗地主你玩过没?
- 统计各个分类和标签下的文章数
- 缩写名词解释
- 用这些免费开源的图标库,为你的项目画龙点睛吧!
- 快速搭建 SpringCloud 微服务开发环境的脚手架