Redis-audit工具使用(转)

在我的线上环境中，由于应用上对redis数据没有做冷热处理，所以经常会出现redis内存使用率居高不下的情况，一直以来都想知道都是什么样的数据比较消耗redis内存，就好比写一个sql语句放在数据库中运行，看各表占空间的大小情况然后排序下，这看上去蛮简单的一个功能，貌似在redis上还不容易实现，redis-audit工具基本上可以满足类似的需求。下面是简单的安装和使用介绍！

1:安装ruby

1 2 3 4 5 6 7 8 9 10

[root@db1 ~]# wget ftp://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.2-p0.tar.gz [root@db1 ~]# tar -zxvpf ruby-1.9.2-p0.tar.gz [root@db1 ~]# cd ruby-1.9.2-p0 [root@db1 ruby-1.9.2-p0]# ./configure  --prefix=/usr/local/ruby [root@db1 ruby-1.9.2-p0]# make && make install [root@db1 ruby-1.9.2-p0]# cd [root@redis2 ~]# grep PATH /root/.bash_profile PATH=$PATH:$HOME/bin:/usr/local/ruby/bin:$PATH export PATH [root@redis2 ~]# source /root/.bash_profile

2:安装rubygems

1 2 3 4

[root@db1 ~]# wget http://production.cf.rubygems.org/rubygems/rubygems-2.0.7.tgz [root@db1 ~]# tar zxvf rubygems-2.0.7.tgz [root@db1 ~]# cd rubygems-2.0.7 [root@db1 rubygems-2.0.7]# ruby setup.rb

3:使用gem安装bundler

1 2 3 4 5 6 7

[root@db1 rubygems-2.0.7]# cd [root@db1 ~]#  wget http://tokyo-m.rubygems.org/gems/bundler-1.3.5.gem [root@db1 ~]# gem install bundler -l Successfully installed bundler-1.3.5 1 gem installed Installing ri documentation for bundler-1.3.5... Installing RDoc documentation for bundler-1.3.5...

4:安装git

1 2 3 4 5 6 7

[root@db1 ~]# cat /etc/yum.repos.d/git.repo [puias-computational] name=PUIAS Computational baseurl=http://puias.math.ias.edu/data/puias/computational/5/x86_64/ enabled=1 gpgcheck=0 [root@db1 ~]# yum -y install git

5:使用git迁出redis-audit源代码，查看redis数据情况

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139

[root@db1 ~]#  git clone https://github.com/snmaynard/redis-audit.git Cloning into redis-audit... remote: Counting objects: 117, done. remote: Compressing objects: 100% (79/79), done. remote: Total 117 (delta 39), reused 113 (delta 36) Receiving objects: 100% (117/117), 17.63 KiB, done. Resolving deltas: 100% (39/39), done. [root@db1 ~]# cd redis-audit/ [root@db1 redis-audit]# bundle install Fetching source index from http://rubygems.org/ Installing redis (3.0.2) Using bundler (1.3.5) Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed. [root@db1 redis-audit]# bundle exec ruby redis-audit.rb 192.168.1.13 65430 0 1000 Auditing 192.168.1.13:65430 db:0 sampling 1000 keys Getting a list of all 358 keys... Auditing 358 keys... 35 keys sampled - 10% complete - 2013-08-23 14:46:15 +0800 70 keys sampled - 20% complete - 2013-08-23 14:46:15 +0800 105 keys sampled - 29% complete - 2013-08-23 14:46:15 +0800 140 keys sampled - 39% complete - 2013-08-23 14:46:15 +0800 175 keys sampled - 49% complete - 2013-08-23 14:46:16 +0800 210 keys sampled - 59% complete - 2013-08-23 14:46:16 +0800 245 keys sampled - 68% complete - 2013-08-23 14:46:16 +0800 280 keys sampled - 78% complete - 2013-08-23 14:46:16 +0800 315 keys sampled - 88% complete - 2013-08-23 14:46:17 +0800 350 keys sampled - 98% complete - 2013-08-23 14:46:17 +0800 DB has 358 keys Sampled 464.34 MB of Redis memory Found 12 key groups ============================================================================== Found 23 keys containing strings, like: ShoppingCart_6d7cfd49f68043669f30133dc5d983a9, ShoppingCart_177600d6b78d46b9990ab5f6dc5711a6, ShoppingCart_a58d6bd82e8b4e7eb55d874c831b66de, ShoppingCart_6405daf3e0b042f580417e3100887871, ShoppingCart_8d6ff3326eaf46bc8b2e24fb9fad6f87, ShoppingCart_0179c7f4f7c84ab5bd387b068dd1632f, ShoppingCart_93637af307fa47038db62e146351f373, ShoppingCart_0c2f7ed9663e432f8d9cdba89a186650, ShoppingCart_c08edc03c3db43c580d66593211da03f, ShoppingCart_76069203a18c4ecba40f713b85ac24d9 These keys use 0.0% of the total sampled memory (4.33 kB) None of these keys expire Average last accessed time: 47 minutes, 40 seconds - (Max: 47 minutes, 40 seconds Min:47 minutes, 40 seconds) ============================================================================== Found 83 keys containing strings, like: Class:TrainingOnline;UsrId:c99b93261e5642e4995e313174bc1c46;trnId:9f0ee4743e7f40c085247c14eb5eeae9;, Class:TrainingOnline;UsrId:d6a7adaea2ea4b30907de24de0047ea6;trnId:4fb1eedb88b14d45aba927492d1c8713;, Class:TrainingOnline;UsrId:32cdc62b1cdb4307888c6ad2fd53b996;trnId:9f0ee4743e7f40c085247c14eb5eeae9;, Class:TrainingOnline;UsrId:7d4d033f68374ee49f12c60620ffe651;trnId:1b0ae6754ccf4733add782f9a11bd0ef;, Class:TrainingOnline;UsrId:ecfe8652fc9d4177bf782701d764f19d;trnId:26b9d13727694fc69d3fb31a6a8d3e0b;, Class:TrainingOnline;UsrId:7f36ff25369b4e6ba088c235f05b557c;trnId:8b3d8c8c2ab448d585ed7c4909292948;, Class:TrainingOnline;UsrId:9d2e29e4a765495f99f27f8c703f298b;trnId:4fb1eedb88b14d45aba927492d1c8713;, Class:TrainingOnline;UsrId:9b8a5fede6b049a0921ba093ba8009a6;trnId:5a7bcfa6c4c24e42b0e43b5c58f70fc1;, Class:TrainingOnline;UsrId:76069203a18c4ecba40f713b85ac24d9;trnId:cda6b6e1c3fc417bb7e2e27c998a5862;, Class:TrainingOnline;UsrId:785340fe2d1b4136ad6eb2cb4c85d601;trnId:ec44de45aae64487a93da3b522760d4c; These keys use 0.0% of the total sampled memory (7.21 kB) None of these keys expire Average last accessed time: 47 minutes, 40 seconds - (Max: 47 minutes, 40 seconds Min:47 minutes, 40 seconds) ============================================================================== Found 1 keys containing hashs, like: yd.mvc.Public.SsoModels.SsoTemp These keys use 0.01% of the total sampled memory (28.46 kB) None of these keys expire Average last accessed time: 47 minutes, 40 seconds - (Max: 47 minutes, 40 seconds Min:47 minutes, 40 seconds) ============================================================================== Found 2 keys containing hashs, like: yd.Model.Public.OnlineCustomer, yd.Model.Public.UserHasAction These keys use 0.02% of the total sampled memory (74.29 kB) None of these keys expire Average last accessed time: 24 minutes, 10 seconds - (Max: 47 minutes, 40 seconds Min:40 seconds) ============================================================================== Found 1 keys containing hashs, like: yd.Common.Monitoring.MvcRun These keys use 0.13% of the total sampled memory (597.7 kB) None of these keys expire Average last accessed time: 10 seconds - (Max: 10 seconds Min:10 seconds) ============================================================================== Found 2 keys containing hashs, like: yd.Util.MethodRun, yd.Util.WcfKeep These keys use 0.26% of the total sampled memory (1.21 MB) None of these keys expire Average last accessed time: 15 seconds - (Max: 20 seconds Min:10 seconds) ============================================================================== Found 13 keys containing hashs, like: yd.Model.Stu.StudyCourseHistory, yd.Model.Sso.Userlink, yd.Model.Sys.GlobalConfig, yd.Model.Sys.ConfigService, yd.Model.Sys.ServiceType, yd.Model.Sso.Domain, yd.Model.Stu.TestTotal, yd.Model.Stu.TestRecord, yd.Model.Stu.StudyNote, yd.Model.Sta.StarStudent These keys use 3.54% of the total sampled memory (16.42 MB) None of these keys expire Average last accessed time: 47 minutes, 40 seconds - (Max: 47 minutes, 40 seconds Min:47 minutes, 40 seconds) ============================================================================== Found 7 keys containing hashs, like: yd.Model.Wot.WorkStandardConfig, yd.Model.Wot.TeacherAttention, yd.Model.Wot.TrainingWork, yd.Model.Wot.WorkStandard, yd.Model.Wot.WorkStudent, yd.Model.Wot.WorkType, yd.Model.Wot.StudentWork These keys use 4.51% of the total sampled memory (20.92 MB) None of these keys expire Average last accessed time: 47 minutes, 40 seconds - (Max: 47 minutes, 40 seconds Min:47 minutes, 40 seconds) ============================================================================== Found 19 keys containing hashs, like: yd.Model.Cla.AssessTemplateConfig, yd.Model.Cla.WorkStandard, yd.Model.Cla.NewpaperTemplate, yd.Model.Cla.TeacherWorkDetail, yd.Model.Cla.AssessConfig, yd.Model.Cla.OnlineDiscuss, yd.Model.Cla.StudentAssess, yd.Model.Cla.GrowArchive, yd.Model.Cla.LearningResource, yd.Model.Cla.StudentFileConfig These keys use 11.94% of the total sampled memory (55.43 MB) None of these keys expire Average last accessed time: 47 minutes, 40 seconds - (Max: 47 minutes, 40 seconds Min:47 minutes, 40 seconds) ============================================================================== Found 28 keys containing hashs, like: yd.Model.Asm.SysinfoConfigType, yd.Model.Asm.User, yd.Model.Ame.CardUseApply, yd.Model.Asm.BranchExpand, yd.Model.Ame.CardOperate, yd.Model.Ame.CardMoney, yd.Model.Ame.CardSellerDetail, yd.Model.Asm.FavoritesType, yd.Model.Ame.Card, yd.Model.Ame.CardSeller These keys use 12.77% of the total sampled memory (59.32 MB) None of these keys expire Average last accessed time: 46 minutes, 5 seconds - (Max: 47 minutes, 40 seconds Min:3 minutes, 30 seconds) ============================================================================== Found 22 keys containing hashs, like: yd.Model.Tra.InvoicePrintDetail, yd.Model.Tra.TrainingYear, yd.Model.Tra.InvoicePrint, yd.Model.Tra.TrainingConfigType, yd.Model.Tra.StudentStudy, yd.Model.Tra.CourseAndTeacher, yd.Model.Tra.SendProcess, yd.Model.Tra.StudentDredge, yd.Model.Tra.StudentDredgeHistory, yd.Model.Tra.TrainingMenu These keys use 15.74% of the total sampled memory (73.09 MB) None of these keys expire Average last accessed time: 41 minutes, 36 seconds - (Max: 47 minutes, 40 seconds Min:5 minutes, 10 seconds) ============================================================================== Found 157 keys containing hashs, like: yd.Model.Crm.LibraryType, yd.Model.Wot.StandardConfig, yd.Model.Tra.TrainingTemplate, yd.Model.Ctl.CodingSysinfo, yd.Model.Bbs.UserDynamicReply, yd.Model.Asm.Contact, yd.Model.Buy.OfflineRecord, yd.Model.Cts.TimingTaskLog, yd.Model.Cla.Assess, yd.Model.Sys.ArgeService These keys use 51.09% of the total sampled memory (237.25 MB) None of these keys expire Average last accessed time: 43 minutes, 18 seconds - (Max: 47 minutes, 50 seconds Min:10 seconds) ============================================================================== Summary ---------------------------------------------------+--------------+-------------------+--------------------------------------------------- Key                                                | Memory Usage | Expiry Proportion | Last Access Time                               ---------------------------------------------------+--------------+-------------------+--------------------------------------------------- yd.Model.Crm.LibraryType                           | 51.09%       | 0.0%              | 10 seconds                                     yd.Model.Tra.InvoicePrintDetail                    | 15.74%       | 0.0%              | 5 minutes, 10 seconds                          yd.Model.Asm.SysinfoConfigType                     | 12.77%       | 0.0%              | 3 minutes, 30 seconds                          yd.Model.Cla.AssessTemplateConfig                  | 11.94%       | 0.0%              | 47 minutes, 40 seconds                         yd.Model.Wot.WorkStandardConfig                    | 4.51%        | 0.0%              | 47 minutes, 40 seconds                         yd.Model.Stu.StudyCourseHistory                    | 3.54%        | 0.0%              | 47 minutes, 40 seconds                         yd.Util.MethodRun                                  | 0.26%        | 0.0%              | 10 seconds                                     yd.Common.Monitoring.MvcRun                        | 0.13%        | 0.0%              | 10 seconds                                     yd.Model.Public.OnlineCustomer                     | 0.02%        | 0.0%              | 40 seconds                                     yd.mvc.Public.SsoModels.SsoTemp                    | 0.01%        | 0.0%              | 47 minutes, 40 seconds                         Class:TrainingOnline;UsrId:c99b93261e5642e4995e313 | 0.0%         | 0.0%              | 47 minutes, 40 seconds                         ShoppingCart_6d7cfd49f68043669f30133dc5d983a9      | 0.0%         | 0.0%              | 47 minutes, 40 seconds                         ---------------------------------------------------+--------------+-------------------+---------------------------------------------------

 

从实际使用的情况上看，当要分析采样的数据较多时，会出现timeout报错信息，目前尚不清楚如何处理！